Network Traffic Sampling via Hash-Based Quasi-Random Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network traffic sampling methods, such as trajectory-based and threshold-based sampling, fail to consistently select subsets of flow records generated by different routers with a selection probability that depends on the fields of the flow record, leading to inconsistent sampling across routers.
Innovation Solution
A method that calculates a hash for each flow record based on invariant parts, generates a quasi-random number, and samples flow records with a probability P, allowing multiple routers to uniformly calculate hashes and generate the same quasi-random number, with P potentially depending on flow size, to achieve consistent sampling across routers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If trajectory-based sampling is used to observe trajectories at each router, then packet traversal tracking is improved, but consistent selection probability across different routers cannot be achieved
Solution Approach 1:
The patent changes the parameter from router-specific trajectory observation to hash-based flow identification. By computing a hash of flow identifiers (source/destination IP addresses, ports, protocol) at each router, the system transforms variable trajectory data into a consistent parameter (hash value) that can be uniformly compared across routers, enabling consistent selection probability while maintaining trajectory observation capabilities.
Solution Approach 2:
The patent introduces a hash function as an intermediary between the raw flow records and the sampling decision. This intermediary transforms the complex trajectory information into a standardized form (hash value) that can be consistently processed by all routers, mediating between the need for trajectory observation and the requirement for consistent selection probability across different network nodes.
2Quantity of substance
If threshold-based sampling is used to select flow records based on flow size, then large flow detection is improved, but consistent selection across multiple routers cannot be achieved
Solution Approach 1:
The patent transforms the sampling parameter from flow size threshold (which varies by router capacity and traffic patterns) to a hash-based probability parameter. The hash value provides a consistent basis for sampling decisions across all routers, while still allowing the sampling probability to be adjusted based on flow characteristics, thereby maintaining both flow size detection capability and cross-router consistency.
Solution Approach 2:
The hash-based sampling mechanism serves multiple functions simultaneously: it enables flow identification, provides consistent sampling probability across routers, and allows for size-dependent sampling rates. This universal approach replaces the need for separate threshold-based mechanisms at each router, achieving multi-functionality that resolves the contradiction between flow size detection and sampling consistency.
3Productivity
If different routers independently sample flow records, then local traffic analysis is improved, but coordinated selection probability across the network cannot be achieved
Solution Approach 1:
The patent changes the sampling parameter from router-specific local criteria to a network-wide hash-based parameter. Each router independently computes the same hash value for a given flow using the formula h(flow) = hash(flow_id) mod N, where N is the number of routers. This parameter transformation enables both independent local processing and network-wide coordination, as the deterministic hash function ensures consistent sampling decisions across all routers without requiring inter-router communication.
Solution Approach 2:
The patent implements a virtual copy of the sampling logic at each router through the hash function. Instead of requiring centralized coordination or inter-router communication, each router has an identical copy of the sampling algorithm that produces consistent results independently. This copying approach maintains local processing efficiency while achieving network-wide sampling coordination through the deterministic nature of the hash function.
Data Source
AI summary
Disclosed herein are systems, computer-implemented methods, and computer-readable media for sampling network traffic. The method includes receiving a plurality of flow records, calculating a hash for each flow record based on one or more invariant part of a respective flow, generating a quasi-random number from the calculated hash for each respective flow record, and sampling flow records having a quasi-random number below a probability P. Invariant parts of flow records include destination IP address, source IP address, TCP/UDP port numbers, TCP flags, and network protocol. A plurality of routers can uniformly calculate hashes for flow records. Each router in a plurality of routers can generate a same quasi-random number for each respective flow record and uses different values for probability P. The probability P can depend on a flow size. The method can divide the quasi-random number by a maximum possible hash value.