Vectorized Packet Forwarding Reducing Cache Misses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional forwarding engines in intermediate network nodes experience a high number of cache misses, leading to inefficient packet processing due to the need to load and retrieve forwarding instructions and data frequently from cache and main memory, which slows down data packet forwarding operations.
Innovation Solution
Implementing a vector-based forwarding approach where multiple packets are processed concurrently, reducing cache misses by loading forwarding instructions and data once per vector rather than per packet, and dynamically adjusting vector size and timer intervals to maintain low latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional forwarding engines process packets individually, then each packet receives complete forwarding instructions, but cache misses increase and processing efficiency decreases
Solution Approach 1:
The patent combines multiple individual packet processing operations into a single vectorized processing operation. By merging N packets into a vector and processing them simultaneously with loaded forwarding instructions, the system eliminates repeated cache accesses that would occur if each packet were processed individually, thereby improving efficiency while reducing time loss.
Solution Approach 2:
The patent performs preliminary loading of forwarding instructions into the cache before processing the vector of packets. This preliminary action ensures that the instructions are already resident in the cache when the actual packet processing begins, preventing cache misses during the critical processing phase and reducing overall access time.
2Reliability
If forwarding instructions are loaded per packet, then each packet is processed accurately, but the number of cache misses increases significantly
Solution Approach 1:
The patent merges the processing of N packets into a single vectorized operation where forwarding instructions are loaded once and applied to all packets in the vector. This merging maintains forwarding accuracy for each packet while dramatically improving throughput by eliminating redundant cache loading operations that would occur with individual packet processing.
Solution Approach 2:
The patent creates a copy of the forwarding instructions that resides in the cache and applies this copied version to multiple packets simultaneously. This copying approach ensures that each packet receives accurate forwarding instructions (maintaining reliability) while the single cache load operation serves all packets in the vector (improving productivity).
Data Source
AI summary
An intermediate network node is configured to forward a plurality of packets concurrently, e.g., as a vector, rather than one packet at a time. As such, the node can load a single sequence of forwarding instructions that may be repeatedly executed for packets in the vector. In addition, the intermediate network node adaptively controls the rate at which it processes data packets through a directed forwarding graph. To that end, the intermediate node is configured to dynamically select the number of packets per vector, i.e., vector size, processed at each node of the forwarding graph. Further, the intermediate node also may be configured to dynamically select timing intervals for one or more “node timers” used to control the rate at which packets traverse through the graph. Illustratively, the vector size and node-timer intervals are selected so that the average latency through the forwarding graph is less than a predetermined target latency, e.g., 50 microseconds (μs).