Parallel Processing Paths for Ultra-Low Latency Network Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network devices introduce latency due to serial decision-making processes, which hinder ultra-low latency requirements for High Frequency Trading and other applications.
Innovation Solution
Implementing a network device with multiple processing decision paths, including a fast processing decision path and a second processing decision path that uses a table to make decisions in parallel, allowing for ultra-low latency processing by leveraging a table lookup for packets with matching flow parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a serial decision-making process is used in conventional network devices, then design complexity is reduced, but latency increases
Solution Approach 1:
The network device is divided into multiple parallel processing decision paths (fast path and second path) instead of a single serial path. Each path can independently process packets, with the fast path handling time-critical packets and the second path handling other packets, thereby reducing overall latency while maintaining manageable design complexity through modular organization.
Solution Approach 2:
The patent transitions from a one-dimensional serial processing architecture to a multi-dimensional parallel processing architecture by adding multiple processing paths that operate simultaneously. This dimensional expansion allows the system to process multiple packets in different states at the same time, reducing latency without proportionally increasing design complexity.
2Loss of time
If a parallel processing decision path with table lookup is implemented, then latency is reduced, but device complexity increases
Solution Approach 1:
Processing decisions are pre-computed and stored in tables during system initialization or idle periods. When packets arrive, the fast path performs quick table lookups using packet flow parameters as keys to retrieve pre-determined processing decisions, bypassing complex real-time analysis. This preliminary action significantly reduces latency while keeping the runtime device complexity manageable.
Solution Approach 2:
The patent creates simplified copies of processing logic in the form of lookup tables that contain pre-computed processing decisions. Instead of executing complex processing logic for every packet, the system uses these table copies to quickly retrieve decisions based on packet parameters, reducing latency while maintaining the sophistication of the original processing logic in the table structures.
3Productivity
If multiple processing paths are used, then processing speed is improved, but resource requirements increase
Solution Approach 1:
Different processing paths are assigned different qualities and functions: the fast path is optimized for speed with simple table lookup logic for time-critical packets, while the second path handles more complex processing for other packets. This local differentiation allows the system to achieve high processing speed for urgent packets without requiring all resources to be optimized for speed, thereby managing overall resource requirements efficiently.
Solution Approach 2:
The fast path performs only the essential table lookup operation for packets that require ultra-low latency, without executing the full processing logic. This partial action approach achieves sufficient processing speed for time-critical packets while conserving computational resources that would be required if the complete processing path were executed for every packet.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Presented herein are techniques to achieve ultra low latency determination of processing decisions for packets in a network device. A packet is received at a port of a network device. A processing decision is determined in a first processing decision path based on content of the packet and one or more network policies. A processing decision is determined in a second processing decision path, in parallel with the first processing path, by accessing a table storing processing decisions. The second processing decision path can output a processing decision faster than the first processing decision path for packets that match one or more particular packet flow parameters contained in the table. A processing decision determined by the second processing decision path, if one can be made, is used, and otherwise a processing decision determined by the first processing decision path is used.