Packet Processing Load Balancer for Heavy Flow Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network interface devices face challenges in efficiently processing large amounts of network traffic due to the risk of a single CPU becoming overburdened, leading to increased latency and reduced performance, especially when handling long-lived high-traffic flows.
Innovation Solution
The implementation of a network interface device that utilizes a count-min sketch to detect heavy flows and a load balancer to distribute packet processing across multiple processing units, allowing for efficient handling of both heavy and non-heavy flows by separating their processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single central CPU is allocated to process traffic, then processing simplicity is maintained, but the CPU becomes overburdened when faced with large amounts of traffic, leading to increased processing time
Solution Approach 1:
The patent divides the packet processing function across multiple CPU cores using Receive Side Scaling (RSS). Each core processes packets for specific flow hashes, segmenting the overall processing workload to prevent any single CPU from becoming a bottleneck while maintaining architectural simplicity through automated load distribution.
2Productivity
If RSS is used to distribute network traffic to multiple cores, then processing load is spread across multiple CPUs, but long-lived high traffic flows can still cause a selected core to become overloaded, increasing latency
Solution Approach 1:
The patent introduces dynamic flow classification that distinguishes between heavy flows (long-lived, high traffic) and other flows. The system dynamically adjusts processing strategies based on flow characteristics, applying specialized handling to heavy flows while maintaining standard RSS for other traffic, thereby preventing latency spikes from overwhelming any single core.
Solution Approach 2:
The patent applies different processing qualities to different flow types. Heavy flows receive specialized handling with dedicated resource allocation and prioritization, while non-heavy flows use standard RSS processing. This local differentiation ensures that critical high-traffic flows receive adequate resources without compromising overall system throughput.
3Ease of operation
If a proxy or distributor core is used to distribute networking flows to multiple cores, then flow distribution is centralized, but a single-point bottleneck is created in the system
Solution Approach 1:
The patent implements self-service flow distribution where the RSS mechanism automatically assigns flows to cores based on hash calculations without requiring a centralized proxy. The system manages its own load distribution through deterministic hashing and indirection tables, eliminating the need for separate distribution management infrastructure.
4Measurement precision
If a smartNIC collects statistics of each flow to identify heavy flows, then flow identification accuracy is improved, but with limited on-board memory, collecting statistics of all flows becomes impossible as the number of flows increases
Solution Approach 1:
The patent applies partial monitoring by using RSS hash buckets to sample and track only the most relevant flow characteristics rather than collecting complete statistics for all flows. This partial action approach enables heavy flow identification with limited memory resources by focusing measurement on flows that matter most for load balancing decisions.
Data Source
AI summary
Examples described herein include a device interface; a first set of one or more processing units; and a second set of one or more processing units. In some examples, the first set of one or more processing units are to perform heavy flow detection for packets of a flow and the second set of one or more processing units are to perform processing of packets of a heavy flow. In some examples, the first set of one or more processing units and second set of one or more processing units are different. In some examples, the first set of one or more processing units is to allocate pointers to packets associated with the heavy flow to a first set of one or more queues of a load balancer and the load balancer is to allocate the packets associated with the heavy flow to one or more processing units of the second set of one or more processing units based, at least in part on a packet receive rate of the packets associated with the heavy flow.