Hierarchical Packet Preemption for Latency Jitter Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance computing (HPC) systems face performance bottlenecks due to latency issues in data transfer over interconnects between compute nodes, particularly when mixing small and large messages, which increases latency jitter and reduces fabric efficiency.
Innovation Solution
The implementation of a flexible credit exchange architecture within high-performance fabrics that enables efficient data transfer by using Host Fabric Interfaces, switches, and gateways, along with credit-based flow control and packet preemption mechanisms to manage latency and ensure reliable data transmission across the fabric.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If small messages are transmitted in a fabric with large packets, then fabric efficiency is improved, but latency jitter increases significantly
Solution Approach 1:
The patent segments packets into smaller units called flits (flow control digits) that can be transmitted independently. This allows small messages to be sent without being blocked by large packets, as each flit can be preempted and transmitted at its own pace, thereby reducing latency jitter while maintaining fabric efficiency.
Solution Approach 2:
The patent implements dynamic packet preemption where higher-priority packets can interrupt lower-priority packet transmissions. This dynamic mechanism allows the system to adapt to varying message sizes and priorities in real-time, ensuring that small latency-sensitive messages can pass through even when large packets are present, thus reducing latency jitter without sacrificing overall fabric efficiency.
2Loss of time
If packet preemption is implemented to reduce latency jitter, then latency performance is improved, but device complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-establishing priority levels and preemption rules before data transmission occurs. Packets are classified into priority levels in advance, and the preemption mechanism is pre-configured, allowing the fabric to automatically manage latency without requiring complex real-time decision-making logic during transmission, thus reducing operational complexity.
Solution Approach 2:
The patent employs feedback mechanisms where the fabric monitors packet transmission status and adjusts preemption decisions based on actual conditions. This feedback loop allows the system to learn from previous transmissions and optimize future preemption actions, reducing the need for overly complex control logic while maintaining effective latency management.
Data Source
Figure 1
Figure 2~13
Figure 3~4
AI summary
Methods, apparatus, and systems for implementing hierarchical and lossless packet preemption and interleaving to reduce latency jitter in flow-controller packet-based networks. Fabric packets are divided into a plurality of data units, with data units for different fabric packets buffered in separate buffers. Data units are pulled from the buffers and added to a transmit stream in which groups of data units are interleaved. Upon receipt by a receiver, the groups of data units are separated out and buffered in separate buffers under which data units for the same fabric packets are grouped together. In one aspect, each buffer is associated with a respective virtual lane (VL), and the fabric packets are effectively transferred over fabric links using virtual lanes. VLs may have different levels of priority under which data units for fabric packets in higher-priority VLs may preempt fabric packets in lower-priority VLs. By transferring data units rather than entire packets, transmission of a packet can be temporarily paused in favor of a higher-priority packet. Multiple levels of preemption and interleaving in a nested manner are supported.