Dynamic Packet Batch Sizing for Throughput and Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing software packet processing systems face challenges in optimizing throughput performance while minimizing latency and jitter, as increasing batch size improves throughput but worsens latency, and decreasing batch size degrades throughput.
Innovation Solution
A network device dynamically adjusts batch sizes based on queue conditions and throughput performance across multiple processing stages using a batch size manager logic unit, employing algorithms to increase or decrease batch sizes to balance latency and throughput, and synchronizes batch sizes across stages to prevent packet loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If batch size is increased to optimize throughput performance, then throughput is improved, but latency and jitter performance deteriorate
Solution Approach 1:
The patent implements dynamic batch size adjustment where the batch size is not fixed but adapts based on current system conditions. The batch size manager logic unit continuously monitors queue conditions and throughput performance, then dynamically modifies the batch size parameter to optimize the trade-off between throughput and latency in real-time operational conditions.
Solution Approach 2:
The patent changes the batch size parameter based on monitored system conditions. By adjusting this key parameter dynamically, the system can shift between prioritizing throughput (larger batches) and reducing latency (smaller batches) depending on current operational needs, thus resolving the contradiction between these two performance metrics.
2Productivity
If batch size is increased, then throughput performance is improved, but jitter performance deteriorates
Solution Approach 1:
The dynamic batch size adjustment mechanism allows the system to respond to changing conditions by modifying batch size, thereby maintaining more stable jitter performance. When conditions indicate potential jitter issues, the system can reduce batch size to stabilize packet processing timing, while still maintaining high throughput when conditions are favorable.
Solution Approach 2:
The system implements feedback control by monitoring throughput performance and queue conditions, then using this information to adjust batch size. This closed-loop control helps maintain stable jitter performance by continuously adapting to system state changes that would otherwise cause jitter variations.
3Loss of time
If batch size is decreased to reduce latency, then latency performance is improved, but throughput performance deteriorates
Solution Approach 1:
Rather than using a statically small batch size, the system dynamically adjusts batch size based on current conditions. This allows the system to achieve low latency when needed (by reducing batch size) while maintaining high throughput when conditions permit (by increasing batch size), thus resolving the contradiction between these two metrics.
4Loss of time
If batch size is decreased to improve latency, then latency is reduced, but packet loss increases due to synchronization issues across processing stages
Solution Approach 1:
The batch size manager logic unit serves multiple functions: it monitors queue conditions, tracks throughput performance, determines optimal batch sizes, and synchronizes batch sizes across multiple processing stages. This centralized multi-functional control ensures that latency optimization does not compromise reliability through packet loss.
Solution Approach 2:
The system uses feedback from throughput monitoring and queue condition assessment to adjust batch size in a way that prevents packet loss. By continuously adapting batch size based on system state, the system can reduce latency without creating the synchronization mismatches that lead to packet loss in multi-stage processing.
Data Source
AI summary
Technologies for dynamically managing a batch size of packets include a network device. The network device is to receive, into a queue, packets from a remote node to be processed by the network device, determine a throughput provided by the network device while the packets are processed, determine whether the determined throughput satisfies a predefined condition, and adjust a batch size of packets in response to a determination that the determined throughput satisfies a predefined condition. The batch size is indicative of a threshold number of queued packets required to be present in the queue before the queued packets in the queue can be processed by the network device.