Switch In-Switch Delay Compensation for Fabric Clock Sync
Solution Overview
Problem
Synchronizing device clocks over a switched fabric, particularly in distributed computing systems with multiple clock domains, is challenging due to traffic congestion and buffering in switches, making it difficult to achieve accurate time synchronization across different fabric domains.
Innovation Solution
Configuring switches to determine and store in-switch delays related to packet transmission and reception, allowing for more accurate time synchronization by adjusting clock synchronization based on these delays, and using non-transparent bridges to isolate devices and facilitate clock domain crossing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If switches buffer transactions to manage traffic congestion, then traffic flow is stabilized, but time synchronization accuracy deteriorates due to unpredictable delays
Solution Approach 1:
The switch determines and stores the in-switch delay in advance (when the request packet is received) rather than when it is needed for synchronization. This preliminary measurement allows the delay to be compensated later, resolving the contradiction by capturing the delay information before buffering operations can introduce variability.
Solution Approach 2:
The invention compensates for the buffering delay by adding the stored in-switch delay to the completion timestamp. This beforehand cushioning approach anticipates the delay impact and corrects it in advance, maintaining both traffic flow stability and time synchronization accuracy.
2Adaptability or versatility
If multiple clock domains are used in different fabric domains, then device independence and flexibility improve, but clock synchronization difficulty increases
Solution Approach 1:
The switch acts as an intermediary between devices in different clock domains. It measures the delay introduced by crossing clock domain boundaries and communicates this information to enable synchronization, thus facilitating device independence while managing the complexity of multi-domain clock synchronization.
Solution Approach 2:
The switch provides feedback about the in-switch delay to the devices, enabling them to adjust their timing accordingly. This feedback mechanism allows devices in different clock domains to synchronize despite the complexity introduced by domain boundaries.
3Measurement precision
If in-switch delay measurement is implemented, then time synchronization accuracy improves, but switch complexity increases
Solution Approach 1:
The switch performs the delay measurement itself using its own internal timing resources when a request packet arrives. This self-service approach eliminates the need for external measurement infrastructure, improving time synchronization accuracy while minimizing the increase in switch complexity.
Data Source
AI summary
Devices and methods for synchronizing devices over a switched fabric. A switch receives a request packet from a device, transmits a completion packet to the device, determines an in-switch delay, and stores the in-switch delay. Another switch receives a packet from a first device, forwards the packet to a second device, determines an in-switch delay of the packet, and stores the in-switch delay. Storing of in-switch delays may include adding an in-switch delay to values in one or more transaction delay fields of a packet. Storing of in-switch delays may include storing the delays in a storage element of a switch. In-switch delay may be determined as a difference between a receiving time corresponding to reception of a packet and a forwarding or transmittal time corresponding to forwarding or transmitting of a packet.