In-Band Clock Synchronization in Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-node computer systems face inefficiencies in synchronizing real-time clock signals due to dedicated clock wires wasting network bandwidth and requiring static configurations that lead to significant downtime during system changes, with existing solutions being unreliable and inflexible.
Innovation Solution
A method where a designated 'master' node in a multi-node system distributes a global real-time clock value using regular network messages and in-band bandwidth, allowing for self-adjustment and flexibility, even in dynamic partitions and unreliable networks, using standard cables like InfiniBand, and enabling scalable and robust clock synchronization without a spanning tree.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If dedicated RTC wires are used to distribute clock signals, then clock distribution reliability is improved, but network bandwidth is wasted and system complexity increases
Solution Approach 1:
The patent combines clock signal distribution with existing network communication infrastructure. Instead of using separate dedicated RTC wires, the system embeds clock synchronization data within regular network messages that traverse the same InfiniBand fabric, thereby merging two functions into one channel and eliminating wasted bandwidth.
Solution Approach 2:
The network infrastructure is made multi-functional by using the same network cables and message channels for both data communication and clock distribution. The InfiniBand network simultaneously handles application traffic and RTC synchronization, making the system more efficient and reducing overall complexity.
2Device complexity
If a static RTC distributing tree is configured by software, then clock distribution structure is established, but system downtime increases when reconfiguration is needed
Solution Approach 1:
The patent transitions from a static, pre-configured RTC distribution tree to a dynamic system that automatically adapts to network changes. The clock synchronization protocol continuously discovers available nodes and adjusts the distribution topology in real-time, allowing the system to reconfigure without manual intervention or downtime when nodes are added or removed.
Solution Approach 2:
The RTC distribution system becomes self-configuring through automatic node discovery and dynamic topology adjustment. When network changes occur, the protocol autonomously recalculates and establishes new clock distribution paths without requiring external reconfiguration, thereby eliminating system downtime.
3Device complexity
If a spanning tree is used to fan out a single clock signal value, then clock distribution is simplified, but system reliability decreases due to single point of failure
Solution Approach 1:
The patent segments the monolithic clock distribution into multiple independent sources and paths. Instead of relying on a single root clock source that fans out through a spanning tree, the system allows multiple nodes to serve as clock sources, creating redundant distribution paths that can independently sustain clock synchronization if one source or path fails.
Solution Approach 2:
The system dynamically changes the operational parameters of clock distribution by allowing any node to potentially become a clock source based on real-time network conditions. This transforms the fixed hierarchical parameter structure of traditional spanning trees into a flexible, adaptive parameter set where multiple nodes can assume master clock roles, thereby improving reliability.
4Measurement precision
If dedicated RTC wires are used for each node, then clock synchronization accuracy is improved, but cable utilization decreases and cost increases
Solution Approach 1:
The patent merges clock synchronization traffic with regular network communication traffic, allowing both to share the same physical cable infrastructure. This approach maintains clock synchronization accuracy by dedicating specific message channels within the shared network while maximizing cable utilization by eliminating the need for separate dedicated RTC wires for each node.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method of determining a master node in a computer system having a plurality of nodes includes establishing a hierarchy of master nodes from the plurality of nodes, wherein the master node synchronizes the plurality of nodes in the computer system with a clock value and determining the master node from the hierarchy of master nodes. A system and method of synchronizing a plurality of nodes in a computer system includes determining a master node from the plurality of nodes, sending a clock value from the master node to neighbor nodes of the master node, synchronizing a node clock in each node receiving the clock value if a predetermined period of time has elapsed in each receiving node, distributing a node clock value from each synchronized node to neighbor nodes of the synchronized node, and repeating synchronizing and distributing, wherein synchronizing a node clock in each node receiving the clock value includes each node receiving the node clock value.