An open multi-functional low-latency high-stability interconnection method

Abstract

The application discloses an open multifunctional low-delay high-stability interconnection method and belongs to the technical field of communication and network interconnection. The method comprises the following steps: establishing passive point-to-point physical connection based on an industry standard among a plurality of accelerator processor nodes; realizing nanosecond-level precision global virtual clock synchronization among the nodes through a Sideband management channel independent of a main data channel; based on the synchronized clock, adopting a time window-restricted forward credit flow control mechanism to complete the cooperative reservation and authorization of link time periods and receiving buffer resources before data transmission; the sender sends data through the main data channel within the authorized time window, and the receiver verifies the compliance of the data arrival time; all control signaling is transmitted through the Sideband channel and is physically isolated from the high-speed data channel. The application realizes open and compatible, low-power-consumption and high-bandwidth physical interconnection, provides nanosecond-level low-delay and high-deterministic data transmission capacity, and improves the communication efficiency and system stability of a large-scale computing cluster.

Description

Technical Field

[0001] This invention belongs to the field of communication and network interconnection technology, specifically relating to an open, multifunctional, low-latency, and high-stability interconnection method. Background Technology

[0002] With the rapid development of artificial intelligence and high-performance computing, large-scale distributed training and real-time inference systems have placed extremely high performance demands on data exchange between multiple processors. Traditional data center networks typically adopt a hierarchical architecture based on switches, and data communication needs to be forwarded through multiple levels of devices, resulting in long transmission paths, high latency, and limited bandwidth utilization, making it difficult to meet the comprehensive requirements of ultra-large-scale computing clusters for low latency and high throughput.

[0003] To reduce communication latency, switchless direct interconnect technology has become an important research direction. However, most mainstream direct interconnect solutions are currently limited to the closed ecosystems of specific vendors, lacking cross-platform compatibility and restricting flexible deployment in heterogeneous computing environments. In addition, these solutions usually do not fully consider the coordinated design of global time synchronization and flow control mechanisms, which can easily lead to communication jitter or buffer overflow under burst traffic, making it difficult to guarantee the timing consistency required for distributed synchronization operations. Existing flow control mechanisms often rely on reverse acknowledgment or simple credit allocation. The former introduces additional latency, and the latter lacks time constraints, neither of which can achieve deterministic data transmission.

[0004] Therefore, under the premise of eliminating central switching equipment, there is an urgent need for an open, low-power interconnection method that supports cross-platform interoperability, which can combine high-precision global clock synchronization with time-aware flow control mechanisms to provide high-bandwidth communication capabilities while achieving nanosecond-level low latency and highly deterministic collaborative data transmission, thereby supporting the efficient and stable operation of next-generation large-scale computing clusters. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides an open, multifunctional, low-latency, and highly stable interconnection method, specifically comprising the following: An open, multifunctional, low-latency, and highly stable interconnect method is applied to a computing cluster containing multiple accelerator processor nodes, comprising: Establish point-to-point physical connections based on open standards among multiple acceleration processor nodes; A global virtual clock synchronization with nanosecond-level precision is established and maintained among the various accelerated processor nodes through a Sideband management channel independent of the main data channel; Based on the global virtual clock synchronization, a forward credit flow control mechanism with time window constraints is used for collaborative reservation of communication resources, including: before data transmission, the sending node sends a credit request containing future transmission time window information to the receiving node through the Sideband management channel; the receiving node returns a credit token containing an authorized time window through the Sideband management channel based on its local resource status. The sending node sends data packets to the receiving node through the main data channel within the authorized time window; The receiving node performs timeliness verification and processing on the received data packets based on the global virtual clock synchronization and the authorized time window; After the data processing is completed, the receiving node sends a credit recovery notification to the sending node through the Sideband management channel.

[0006] Preferably, establishing a point-to-point physical connection based on open standards includes: A passive copper cable conforming to a preset industry standard is used as the transmission medium, and both ends of the passive copper cable are connected to the acceleration processor node through a standardized physical interface; During the link initialization phase, the acceleration processor node reads the length parameter stored in the passive copper cable and configures the physical layer signal conditioning circuit based on the length parameter.

[0007] Preferably, establishing and maintaining global virtual clock synchronization with nanosecond-level accuracy includes: A distributed election protocol is run to elect a master node from each of the aforementioned accelerated processor nodes as the initial clock source; The master node periodically broadcasts a synchronization frame containing its local timestamp through the Sideband management channel. The remaining slave nodes calibrate their local clocks based on the received synchronization frames and the known delays of their transmission paths.

[0008] Preferably, calibrating the local clock from the node specifically includes: Based on the length parameter L read from the passive copper cable and combined with the preset electrical signal propagation speed, the transmission delay Δt is calculated. The local timestamp broadcast by the master node The transmission delay Δt is added to this, serving as the local calibration target time.

[0009] Preferably, when the computing cluster is deployed in a ring topology and the number of nodes exceeds a preset threshold, establishing and maintaining global virtual clock synchronization further includes: Each slave node simultaneously listens to synchronization frames forwarded from the master node and its neighboring nodes; Each node maintains a synchronization frame delay table and runs a dynamic path selection algorithm to select the node corresponding to the path with the minimum observation delay as an auxiliary synchronization source. The clock information from the master node and the auxiliary synchronization source is weighted and fused to fine-tune the local clock correction parameters.

[0010] Preferably, a credit request containing information about the future sending time window is sent, specifically including: The sending node determines the expected start time of transmission based on the amount of data to be transmitted, the global virtual clock synchronization information, and the upper-layer scheduling information. ; Based on the link bandwidth and the amount of data to be sent, the transmission time required is calculated, and thus the end time of the requested time window is obtained. ; The width of the request time window is dynamically adjusted according to the service type of the communication task. The amount of data to be sent and the request time window The service type identifier is encapsulated into a credit request message and sent through the Sideband management channel.

[0011] Preferably, the receiving node makes credit authorization decisions based on local resource conditions, specifically including: Parse the credit request message to obtain the number of bytes to be requested and the request time window; Query the free capacity of the local receive buffer to determine if it meets the application requirements; If the conditions are met, resources are reserved according to the preset buffer management strategy, and the sender's credit request behavior is monitored to assess congestion tendency. Based on the evaluation results, the final authorization time window boundary is determined, a credit token containing the authorization time window is generated, and returned through the Sideband management channel.

[0012] Preferably, the sending node sends data packets through the main data channel, specifically including: At the start of the authorized time window, data transmission is triggered synchronously based on the global virtual clock; Generate a sending timestamp for each data packet and embed it in its header; The main data channel uses differential signal encoding and multi-channel aggregation technology to achieve high-speed data transmission, and does not generate any reverse acknowledgment or congestion feedback signals during transmission.

[0013] Preferably, the receiving node performs timeliness verification and processing on the data packets, specifically including: Parse the received data packets and extract their header sending timestamp and associated credit token identifier; Read the local synchronous clock to obtain the data packet reception time And query the corresponding authorization time window boundary based on the credit token identifier. ; The receiving time is determined by a hardware comparison circuit. Does it meet the requirements? ; If the conditions are met, the data packet payload will be delivered using a zero-copy method; otherwise, the data packet will be marked as abnormal and an exception handling procedure independent of the normal reception process will be triggered.

[0014] Preferably, a credit recall notification is sent, specifically including: After the receiving node has finished processing the data payload, it caches the credit limit to be reclaimed and the corresponding credit token identifier in a local queue. When the accumulated amount in the queue reaches a preset threshold or a timeout mechanism is triggered, one or more cached credit recovery entries are packaged into a credit recovery notification message. The credit recovery notification message is asynchronously sent to the sender node through the Sideband management channel; The sending node updates its local available credit pool based on the token identifier in the credit recovery notification message.

[0015] In summary, this application includes at least one of the following beneficial technical effects: 1. This invention employs industry-standard passive copper cables and standardized interfaces to establish a point-to-point direct physical link, eliminating the need for dedicated switches or external active devices. This design breaks the limitations of closed ecosystems in existing technologies, supports interoperability across hardware vendors, and improves the flexibility and scalability of system construction. Simultaneously, the passive link design significantly reduces single-path power consumption, providing extremely high unidirectional bandwidth while aligning with the trend of green and low-carbon computing development.

[0016] 2. This invention achieves network-wide virtual clock synchronization based on distributed election and delay compensation through an independent Sideband management channel, providing a unified and high-precision time reference for all nodes. Furthermore, by combining a forward credit flow control mechanism with time window constraints, the collaborative reservation and scheduling of link time slots and receive buffer resources are completed before data transmission.

[0017] 3. This invention completely separates all management signaling (including clock synchronization, credit requests, authorization and eviction) from high-speed data streams on the physical channel. A dedicated low-speed sideband channel is responsible for transmitting control signaling, completely independent of the main data channel carrying user data. This ensures that the control logic is not affected by high-speed data flow fluctuations and sudden congestion, greatly improving the response reliability of the control plane and the overall stability of the system, and laying the foundation for reliable collaborative communication of large-scale clusters under long-term high load. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an open, multifunctional, low-latency, and high-stability interconnection method according to the present invention. Figure 2 This is a flowchart illustrating the forward credit flow control mechanism with global virtual clock synchronization and time window constraints in this invention. Figure 3 This is a schematic diagram illustrating the collaborative operation of point-to-point physical connection and Sideband management channel in this invention; Figure 4 This is a flowchart illustrating the entire process of credit request-authorization-transmission-recovery in this invention. Detailed Implementation

[0019] Currently, in large-scale artificial intelligence training and high-performance computing scenarios, traditional networks based on the Spine-Leaf architecture suffer from long communication paths, high end-to-end latency, limited bandwidth utilization, and high power consumption due to their reliance on top-of-rack switches. Existing direct connection solutions are mostly limited to closed ecosystems, lack cross-vendor compatibility, and generally suffer from structural defects such as time asynchrony, reverse latency introduced by flow control mechanisms, and inability to guarantee communication determinism.

[0020] To address the aforementioned technical issues, this invention proposes to construct a point-to-point interconnection system that is switchless, passive, and compatible with open standards. By combining a forward credit flow control mechanism with global virtual clock synchronization and time window constraints, it achieves multi-xPU collaborative communication with nanosecond-level low latency, 800Gbps-level high bandwidth, and high stability. This system is then applied to an open, multi-functional, low-latency, and high-stability interconnection method.

[0021] To further illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose, the following detailed description is provided in conjunction with the accompanying drawings and preferred embodiments, based on specific implementation methods of the present invention.

[0022] Example 1 This embodiment includes multiple xPU nodes, point-to-point physical connection links, a sideband management channel, and clock synchronization units, credit management units, and data transmission engines integrated within each xPU. The xPU nodes are directly interconnected through standardized interfaces, forming a decentralized communication topology. No TOR switches or active relay devices are required to intervene in the main data path. The sideband channel independently carries synchronization frames and credit control signaling, while the main data channel is dedicated to unidirectional high-speed data stream transmission.

[0023] An open, multifunctional, low-latency, and highly stable interconnection method includes the following steps: Step S1: Establish point-to-point physical connections between multiple xPU nodes, using passive copper cables conforming to IEEE 802.3ck and OIF-CEI-112G-VSR standards as the transmission medium. This includes the following steps: S101. A high-speed passive copper cable based on the above standard is selected. The copper cable adopts PAM4 modulation and 8-pair differential shielded twisted pair aggregation technology, supports unidirectional data transmission rate of 800Gbps and above, bidirectional total bandwidth of up to 1600Gbps, and the power consumption of the entire link is less than 0.1W, requiring no external power supply or heat dissipation device.

[0024] S102. Passive copper cables are directly connected to each xPU node via physical interfaces. The interfaces are compatible with CMIS and MSA industry standards and support hot-swapping and online rate negotiation.

[0025] The cable has built-in EEPROM memory at both ends to record manufacturer information, length, supported speeds, and calibration parameters.

[0026] S103. During the link initialization phase, the xPU controller automatically reads the precise length L and other relevant parameters stored in the EEPROM at both ends of the cable.

[0027] Calculate the transmission delay based on the length L and the preset electrical signal propagation speed (5 ns / m). This is used for subsequent clock synchronization calibration.

[0028] S104. Based on the read cable length L and calibration parameters, the SerDes physical layer module integrated inside the xPU dynamically configures the pre-emphasis and equalization circuits, and adjusts the drive strength and receiver gain to ensure signal integrity.

[0029] This adaptive adjustment mechanism ensures that signal attenuation does not exceed 3dB within a 1-meter distance under different link lengths, and that the eye diagram opening meets the requirements of the OIF-CEI-112G-VSR specification.

[0030] S105. After completing the parameter configuration, the system verifies whether the actual performance meets the specifications through the link training and signal quality detection process. If the detection is successful, the physical link is considered ready and the system can enter the clock synchronization and data transmission stage.

[0031] The system has a fault-tolerant mechanism that can reduce operating speed or report an anomaly when a single channel fails.

[0032] In summary, through the above steps S1, the establishment, identification, configuration, and verification of point-to-point passive copper cable interconnection are completed. This method achieves an open, high-bandwidth, and low-power physical connection through standardized interfaces and adaptive signal adjustment, laying a reliable hardware foundation for subsequent global synchronization and time-constrained flow control mechanisms.

[0033] Step S2: Establish global virtual clock synchronization among multiple xPU nodes through a Sideband management channel independent of the main data channel. This includes the following steps: S201. After the system powers on, each xPU node runs a preset distributed election protocol. For example, each node can broadcast its hardware identifier and priority weight to the Sideband channel; the protocol stipulates that the node with the higher priority weight wins, and if the weights are the same, the node with the larger identifier wins.

[0034] Through this deterministic comparison rule, after all nodes have completed information exchange, a master xPU node will be unanimously elected as the initial clock source.

[0035] S202. The master xPU node periodically broadcasts a synchronization timestamp frame via the Sideband channel. This frame carries the current value of its local high-precision counter. The broadcast period is configurable, for example, it can be set to 1 microsecond.

[0036] S203. Each xPU node receives a synchronization frame through the Sideband channel and calculates the transmission delay based on the acquired cable length L and the known electrical signal propagation speed (5ns / m). .

[0037] The local clock was then calibrated. The calibration process requires each xPU node to be equipped with a highly stable local clock source and extremely low jitter in the timestamp processing path, so that the clock deviation between nodes does not exceed ±1ns.

[0038] S204. Each slave node indirectly achieves heartbeat monitoring by listening to the synchronization frames periodically broadcast by the master node.

[0039] If no valid synchronization frame is received within several consecutive synchronization cycles, each node will trigger a new round of election protocol after a preset timeout threshold (e.g., 2ms).

[0040] The newly elected master node will immediately broadcast a new initial timestamp, which other nodes will use to recalibrate, thereby maintaining the high availability of the clock service.

[0041] S205. In a large-scale ring topology with a large number of nodes, in addition to receiving broadcasts from the master node, each node can also listen to the synchronization frames forwarded by neighboring nodes.

[0042] Each node maintains a delay table, recording the difference between the received time of synchronization frames from different neighbors and the original timestamp within the frame.

[0043] S206. Using the minimum delay path selection algorithm, the node dynamically selects the neighbor with the smallest observation delay as an auxiliary reference source, and uses weighted averaging and other methods to fuse information from multiple time sources and fine-tune the local clock correction parameters.

[0044] This enhancement mechanism is used to compensate for accumulated errors in ring topologies, enabling the maximum clock deviation across the entire network under complex topologies to be controlled within ±1.5ns.

[0045] The S207 and Sideband management channels are implemented using low-speed serial protocols (such as UART or I2C) with an operating frequency of 100MHz. Their physical lines are completely isolated from the main data channels to avoid electromagnetic interference. This channel is dedicated to transmitting management messages such as synchronization frames and credit control signaling.

[0046] In summary, through the above steps S2, a complete clock synchronization process is formed, from master node election, periodic broadcasting, delay compensation, fault tolerance to topology adaptive enhancement. This mechanism achieves high-precision global clock alignment through an independent Sideband channel, providing a unified timing reference for subsequent time window-based deterministic flow control.

[0047] Step S3: Before data transmission is initiated, the sender xPU sends a credit request to the target receiver xPU through the Sideband channel, specifically including the following steps: S301. The sender xPU determines the number of bytes to be requested in this credit request based on the amount of data to be sent; this amount of data is usually informed to the credit management unit by the upper-layer application or task scheduler before initiating communication.

[0048] S302. The sender xPU determines a future, estimated start time T_1 based on the established global synchronization clock and scheduling information from the upper-layer task scheduler (such as "earliest available transmission slot").

[0049] Next, based on the current link bandwidth (e.g., 800Gbps) and the amount of data to be sent, the time required to complete the transmission is calculated, thus obtaining the end time of the time window. This forms a complete time window. .

[0050] S303. The sender xPU dynamically adjusts the width of the time window in the credit request based on the service type of the current communication task; the specific adjustment strategy includes: For synchronous operations such as AllReduce, the window width is set to a fixed value, for example, twice the maximum allowable step time deviation of the cluster (i.e., 40ns). If it is a real-time data frame transmission in the inference pipeline, the window width is set according to the frame period. For example, for a data stream with a frame every 10μs, the window width can be set to 12μs to accommodate a certain amount of timing jitter.

[0051] These adjustment strategies can be pre-set in the strategy lookup table of the credit management unit for quick matching.

[0052] S304. The credit management unit receives task characteristic parameters (such as business type identifier, urgency level, etc.) issued by the upper-layer application or runtime system, and maps these parameters to specific flow control policies, including the selection of time window width and the priority of credit limit; this mapping relationship can be implemented through a pre-configured configuration table or policy engine.

[0053] S305. The sender xPU encapsulates the parameters determined in steps S301-S304 into a credit request message.

[0054] The message format must include a type field, a unique request identifier, the amount of data requested, and the start of the time window. and the end point In addition to core fields such as business type identifier, and It forms the basis for the recipient to authorize and verify within a time window.

[0055] The message is sent to the target receiver xPU via a separate Sideband management channel.

[0056] In summary, step S3 describes the generation and transmission process of credit requests. This process integrates global clock, scheduling information, link bandwidth, and service awareness strategies to construct credit requests with precise time constraints. This lays a crucial foundation for subsequent credit authorization and window-constrained transmission by the receiver, effectively avoiding the deterministic loss problem caused by the lack of a time dimension in traditional flow control.

[0057] Step S4: After receiving the credit request, the receiving xPU queries the free capacity of its local receive buffer and makes a credit authorization decision accordingly. This includes the following steps: S401. The receiver xPU receives the credit request message from the sender through the Sideband management channel and parses out the number of bytes to be requested and the request time window contained therein. Information such as business type identifier.

[0058] S402. The receiver xPU queries the current free capacity of the local receive buffer and determines whether it is greater than or equal to the number of bytes requested in the credit request.

[0059] The buffer status can be obtained in real time through the memory management unit or a dedicated buffer status register.

[0060] S403. If the buffer free capacity meets the requirements, the receiver xPU further executes the buffer reservation strategy: For large data transfers (e.g., greater than 1MB), a contiguous physical memory region is pre-locked in the buffer to prevent memory fragmentation from causing subsequent allocation failures.

[0061] This reservation operation is usually implemented by calling the memory locking interface provided by the operating system kernel. If the reservation fails, the receiver will return a negative response and suggest that the sender adjust the request.

[0062] S404. The receiving xPU also maintains a buffer water level threshold (e.g., setting the low water level threshold to 64KB). When the idle capacity is lower than this threshold, it rejects newly arrived large credit requests (i.e. requests with a request amount greater than 64KB) and only accepts small credit requests (less than 64KB) or data credit requests marked as urgent, thereby achieving priority protection and hierarchical scheduling of receiving resources.

[0063] S405. The receiver's xPU monitors credit request behavior against the same sender: Record the arrival times of historical credit request messages locally, calculate the arrival time intervals between adjacent requests, and calculate the average interval.

[0064] If it is detected that the arrival time interval of N consecutive requests from the sender is less than K% of the historical average (e.g., N=3, K=50%), then it is determined that its sending behavior may have a congestion tendency.

[0065] S406. When congestion tendency is identified, the receiver xPU will perform preventative window compression adjustments when generating the trust token. Specifically: Keep the authorization window start time T_1 unchanged, and adjust the window end time to , where α is a configurable compression factor (e.g., 0.7).

[0066] This adjustment acts as a backpressure signal, prompting the sender to replan its transmission to smooth out traffic. If congestion tendency is not identified, then... Keep as requested .

[0067] S407. After completing the above resource checks and strategy decisions, the receiving party xPU generates a credit token.

[0068] The token contains the maximum number of bytes authorized and the final valid time window. And the token's unique identifier.

[0069] The token is returned to the sender xPU via the Sideband management channel, completing the credit authorization process.

[0070] In summary, step S4 describes the complete process by which the recipient processes a credit request and grants credit. This process not only performs basic resource availability checks, but also ensures the stability of resource allocation through buffer reservation and hierarchical management. Furthermore, it combines time-based behavioral monitoring and preventative window adjustment to achieve intelligent and deterministic credit authorization.

[0071] Step S5: After obtaining the trust token, the sender xPU continuously sends data packets through the main data channel within the valid time window, specifically including the following steps: S501. After receiving the credit token returned by the receiver, the sender xPU parses out the maximum number of authorized bytes and the valid time window. and token identifier.

[0072] Subsequently, based on the authorized number of bytes and the preset maximum transmission unit of the link (e.g., 2KB), the data to be sent is divided into several data packets that conform to the MTU size.

[0073] S502. The sender xPU triggers data transmission precisely at the start time T1 of the time window based on the global synchronization clock. The start time can be aligned with the rising edge of the system clock.

[0074] Generate a sending timestamp for each data packet The timestamp is latched by a high-precision counter of the local synchronized clock at the start of packet header transmission and embedded into the packet header with nanosecond precision.

[0075] S503, the sender xPU sends data packets through the main data channel.

[0076] The main data channel uses differential signal encoding and is composed of 8 physical channels aggregated together. Each channel has a nominal rate of 112Gbps, and the aggregated channel achieves a unidirectional physical layer bandwidth of 800Gbps.

[0077] The main data channel is driven directly by the SerDes module inside the xPU, without the need for external active devices.

[0078] S504. The data packet format follows a predefined structure, which includes, in sequence: preamble, header, payload data area, and CRC checksum.

[0079] The packet header must include a sending timestamp. The data packet sequence number, the associated credit token identifier, and the data type field.

[0080] CRC is calculated using CRC-32.

[0081] S505. Throughout the entire transmission process, the main data channel strictly adheres to the forward transmission principle, carrying only the data stream from the sender to the receiver, and neither generates nor transmits any ACK, PFC pause frames, or other congestion feedback signals.

[0082] All management signaling is transmitted through the Sideband channel, thus achieving absolute "zero reverse latency" on the main channel.

[0083] S506: The payload of the data packet can be directly written into the PCIe or CXL bus address space of the receiving xPU according to the pre-established address mapping, supporting RDMA operation.

[0084] The sender xPU's data transmission engine retrieves data from local memory via DMA and injects it into the main data channel's transmission queue.

[0085] In summary, step S5 describes the complete sending process after the sender obtains credit authorization. This process uses the time window in the credit token as a hard constraint, the global synchronization clock as a precise triggering reference, and achieves high-speed, deterministic data transmission under strict time window through a high-bandwidth physical link, a communication model without any reverse traffic, and a data packet format containing key control information.

[0086] Step S6: After receiving the data packet, the receiving xPU reads the sending timestamp in its header. Determine the receiving time by combining the local synchronization clock. Does it meet the requirements? Specifically, it includes the following steps: S601, the receiver xPU receives the serial data stream through the physical layer interface of the main data channel. After completing clock recovery, frame synchronization and decoding, it stores the data packets in the link layer receive buffer.

[0087] S602, The receiver xPU parses the data packet format and extracts the sending timestamp Tx_Timestamp, sequence number, associated credit token identifier and other control fields embedded by the sender from the packet header.

[0088] S603. The receiver xPU reads the high-precision counter of its local synchronization clock to obtain the precise local time of data packet arrival, and records it as the reception time. .

[0089] S604. The receiving xPU queries the "Active Authorization Table" maintained by the local credit management unit based on the credit token identifier in the data packet header, and obtains the authorization time window boundaries T1 and T2 corresponding to the token.

[0090] Then, the receiving time will be and and Compare and determine whether the conditions are met. .

[0091] This comparison process is performed by the credit management unit or dedicated timing verification logic.

[0092] S605. If the judgment condition in step S604 is met, then the data packet is determined to have arrived on time.

[0093] The receiving xPU then transfers the payload of the data packet directly to the target application memory (such as GPU memory) or the designated receive queue in a zero-copy manner through the DMA engine. The entire process bypasses the operating system kernel to reduce processing latency.

[0094] After the transmission is completed, the credit management unit will mark part of the credit of the corresponding token as "pending recovery".

[0095] S606, If the receiving time Earlier or later If the data packet is not found to be "early" or "late", it is considered an abnormal situation.

[0096] The receiver xPU will trigger an exception handling process, which is designed to run in parallel with the normal receive pipeline, so it will not block the continued reception of the main data channel.

[0097] S607. The exception handling process is completed quickly in the hardware, including setting an exception flag (e.g., an early or late flag) associated with the data packet in the hardware state machine.

[0098] This exception flag and data packet metadata are placed in the exception queue and passed to the upper-layer protocol stack by the backend logic. The upper-layer protocol then decides whether to retransmit, discard, or downgrade the data packet according to a preset policy.

[0099] If a retransmission is required, a new credit request process will be triggered.

[0100] S608. At the same time, the receiving xPU records the detailed information of this abnormal event in the non-volatile exception log.

[0101] Log entries must contain at least: the original send timestamp Actual reception time Authorization window boundaries And link identifiers, used for subsequent offline performance analysis and system tuning.

[0102] S609. To support the high-speed time comparison in step S604, the receiver xPU integrates the time window verification logic into the hardware state machine.

[0103] This state machine uses a dual comparator structure: one comparator determines... Is it earlier than Another comparator determines Is it later than .

[0104] The two comparators work in parallel and can make a judgment within a single clock cycle, outputting one of three states: early, on time, or late, which directly controls the selection of subsequent data paths.

[0105] In summary, step S6 above illustrates the complete process by which the receiver verifies and processes the data packet on time. This process uses a local clock with nanosecond-level precision and a pre-stored authorization window as a benchmark, and achieves rigorous verification of the deterministic nature of data transmission through hardware-accelerated real-time comparison and diversion.

[0106] On-time data is delivered efficiently and triggers credit status updates, while abnormal data is quickly identified and handled flexibly by the upper layer. The two processes run in parallel to ensure the high efficiency and continuity of the main channel and the overall stability of the system.

[0107] Step S7: After completing data processing and releasing the corresponding buffer space, the receiving xPU asynchronously sends a credit reclamation notification to the sending xPU via the Sideband channel. This includes the following steps: S701. Once the receiving xPU has completed the physical transfer (i.e., storage in the target application memory) of the received data packet payload in its data processing unit (such as DMA engine), it considers the data processing to be complete and then triggers the release operation of the corresponding buffer space.

[0108] S702, The receiving xPU determines the original credit token identifier (i.e., which credit token originally authorized the reservation of the buffer space) and the specific number of bytes to be released for the buffer resources to be reclaimed this time.

[0109] S703. The credit management unit of the receiving xPU does not immediately send a recycling notification, but temporarily caches the credit limit (number of bytes) to be recycled and the corresponding token identifier in a local "credit queue to be recycled".

[0110] This queue has a capacity limit.

[0111] S704, The credit management unit continuously monitors the queue of credits to be recovered.

[0112] When the accumulated credit limit to be recovered in the queue reaches a preset threshold (e.g., 128KB), the batch merging and reporting mechanism is triggered.

[0113] In addition, the system also has a periodic timeout timer (e.g., every 100μs). If the timeout occurs and the queue is not empty, an uploading is also triggered to ensure timely credit recovery. If the queue is full, an uploading is triggered immediately.

[0114] S705. When batch reporting is triggered, the credit management unit packages one or more credit recovery entries (each containing a token identifier and the number of release bytes) in the credit recovery queue into a single credit recovery notification message.

[0115] S706. The credit recovery notification message is sent back to the original sender xPU asynchronously through a separate Sideband management channel.

[0116] The transmission process is independent of the main data stream and will not interfere with ongoing high-speed data transmission.

[0117] S707. After receiving the credit recovery notification message through the Sideband channel, the sender xPU parses out the number of released bytes and the corresponding original credit token identifier contained therein.

[0118] S708, the credit management unit of the sender xPU finds the corresponding entry in its maintained "list of used credits" (usually organized in a structure such as a hash table for quick lookup) based on the token identifier, marks the entry as "reclaimed", and returns the corresponding credit amount (number of bytes) to the local "available credit pool".

[0119] S709. The credit management unit supports out-of-order recycling processing, meaning that the order of recycling notifications returned by the recipient may not be the same as the order in which the credits were used.

[0120] The management unit uses token identifiers for matching to ensure the accurate return of credit limits. If an abnormal notification is received that a token identifier cannot be matched, it is discarded and logged to maintain the correctness of the credit pool status.

[0121] In summary, through step S7 above, a closed loop of credit recovery from resource use to release is completed. This mechanism uses an asynchronous, batch merging method to efficiently transmit resource release information through an independent Sideband channel, minimizing control overhead. At the same time, combined with a dual-trigger mechanism and robust token matching, it ensures the timely and accurate recovery of credit resources, supporting efficient and streamlined operation of the entire credit application-transmission-recovery process.

[0122] Example 2 This embodiment is designed for large-scale ring topology deployment scenarios with a large number of nodes (e.g., more than 8 nodes, taking a 32-node single ring as an example). In this scenario, if only the synchronization mechanism broadcast by the master node in Embodiment 1 is relied upon, the clock synchronization error may increase with the accumulation of the number of hops due to factors such as signal attenuation in multi-hop transmission, inconsistent physical wiring lengths of each segment, and connector impedance differences.

[0123] Therefore, based on the global virtual clock synchronization in Example 1, this embodiment introduces a clock synchronization enhancement mechanism under a ring topology (step S8), which specifically includes the following steps: S801. In a ring topology, in addition to the root primary xPU node (elected according to Example 1) continuing to periodically broadcast global synchronization frames through the Sideband channel, each secondary xPU node in the network simultaneously activates the synchronization frame listening function for its two adjacent left and right neighboring nodes.

[0124] That is, each node receives not only synchronization frames from the master node, but also synchronization frames forwarded by its directly connected predecessor and successor nodes.

[0125] S802, Each node maintains a synchronization frame delay table.

[0126] For each received synchronization frame (whether from the master node or a neighbor node), the node records the time of receipt of the frame (according to the local clock) and the original transmission timestamp carried in the frame. (or the forwarding timestamp of a neighbor), and calculate the difference between the two as the observation delay for that path.

[0127] S803. Each node runs a minimum delay path selection algorithm based on its maintained delay table.

[0128] The core of this algorithm is to compare the observation delay values ​​from the master node and each neighbor node, and dynamically select the node corresponding to the path with the smallest observation delay as the auxiliary synchronization source.

[0129] The purpose of selecting the path with the shortest delay is to minimize the impact of uncertainties (such as jitter) during transmission and obtain a more reliable time reference.

[0130] S804. After obtaining information from the auxiliary synchronization source, the node does not simply switch the synchronization source, but merges the information from the primary node source and the auxiliary source.

[0131] Specifically, the nodes calculate clock offset estimates from both sources (e.g., and ).

[0132] One feasible strategy for weight allocation is to dynamically set the weights based on the stability (such as variance) of the observed delay of each source path over a period of time, assigning higher weights to paths with more stable delays; subsequently, the nodes calculate a comprehensive clock correction parameter using a weighted average method.

[0133] For example: Suppose that the clock offset calculated from the synchronization frame received directly by node i from the master node is... The offset calculated from the synchronization frame received from its left neighbor is .like A configurable difference threshold indicates a significant inconsistency in the synchronization information of the two paths. In this case, node i will not completely depend on either single path, but will instead, according to the aforementioned weighting strategy, adjust its path accordingly. and A weighted average is used to obtain a better estimate of the clock offset, and this value is used to fine-tune its local clock correction parameters.

[0134] S805. In this ring topology, the Sideband management channel used for transmitting synchronization frames and credit control signaling also constitutes an independent logical ring network.

[0135] Credit requests and recovery control signaling are transmitted in this ring management network according to a preset shortest path routing algorithm, thereby ensuring low-latency transmission of all management signaling and further supporting the stable operation of the entire interconnection system under the ring topology.

[0136] S806. Through the above enhancement mechanism, in a single-ring topology consisting of 32 xPU nodes, clock drift and accumulated errors caused by factors such as inconsistent physical wiring lengths, connector impedance differences, and jitter introduced by multi-hop forwarding can be effectively compensated.

[0137] The design goal of this multi-source monitoring and fusion mechanism is to ensure that the maximum clock deviation between any two nodes in such a large-scale ring network can be controlled within ±1.5ns, thereby meeting the stringent requirements for nanosecond-level timing consistency in scenarios such as ultra-large-scale distributed AI training.

[0138] In summary, the ring topology clock synchronization enhancement mechanism described in this embodiment improves the clock synchronization accuracy in large-scale ring networks with many nodes and asymmetric links by introducing multi-source monitoring, dynamic path selection, and information fusion. This mechanism is an important supplement and extension to the basic synchronization scheme of Embodiment 1, ensuring that the method of the present invention can provide a high-precision time reference under different network topologies, thereby guaranteeing the reliable implementation of time window-based deterministic communication in all scenarios.

[0139] Example 3 Based on the specific technical means described in Embodiments 1 and 2 above, the present invention constructs and implements a "pre-reservation" deterministic interconnection paradigm, which is fundamentally different from the traditional "post-response" network (such as InfiniBand and Ethernet). The core of this paradigm is to complete the negotiation and scheduling of the resources required for communication (link time window and receiver buffer space) before the start of data transmission, and to ensure the consistency of protocol execution through high-precision global clock synchronization.

[0140] Specifically, the present invention achieves pre-booking and deterministic transmission through the following process: (1) Establishment of a unified timing reference (corresponding to step S2 and Example 2): Through an independent Sideband management channel, delay compensation is performed using the deterministic physical parameter of cable length. Combined with mechanisms such as distributed election and multi-source fusion, a global virtual clock with nanosecond-level precision is established and maintained throughout the entire system to provide a unique and reliable time scale for all reservation and transmission actions.

[0141] (2) Spatiotemporal resource collaborative reservation and scheduling (corresponding to steps S3 and S4): Before transmission, the sender initiates a window containing the precise future transmission time to the receiver through the Sideband channel. Credit request.

[0142] The receiver acts as a local centralized scheduler, examining its buffer capacity and authorized time window table. By granting credit tokens with time window constraints, it pre-allocates non-conflicting link periods and buffer spaces for different data streams at the protocol level, thereby eliminating resource contention at the micro level.

[0143] (3) Deterministic transmission and verification based on reservation (corresponding to steps S5 and S6): The sender sends data packets with embedded precise timestamps through the main data channel within the time window authorized by the credit token, and the main channel prohibits any reverse control traffic to achieve "zero reverse delay".

[0144] The receiver uses a local synchronization clock to perform a hard verification of the arrival time of each data packet to determine whether it meets the authorization window.

[0145] On-time data is processed efficiently, while abnormal data (early / late arrivals) is identified and transferred to an independent process for processing, without affecting the continuity of the main channel.

[0146] (3) Complete decoupling of control and data: Throughout the process, all management signals (clock synchronization, credit request / authorization / revocation) are transmitted through the physically isolated Sideband channel, which is completely separated from the main channel that carries the actual data, ensuring the stability and reliability of the control logic and preventing interference from high-speed data traffic.

[0147] In summary, this invention achieves an open, low-latency, and highly stable interconnection method through open passive physical connections, high-precision global clock synchronization, and credit-based flow control with time window constraints, supplemented by an architecture that physically isolates the control plane and data plane. Specifically, the global clock provides the system's time language, while the time window credit mechanism utilizes this language for resource reservation and scheduling. The two work together to enable large-scale xPU clusters to perform deterministic data exchange as if they were a single, precisely synchronized machine.

[0148] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0149] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment includes only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An open, multifunctional, low-latency, and highly stable interconnection method, applied to a computing cluster containing multiple accelerated processor nodes, characterized in that, include: Establish point-to-point physical connections based on open standards among multiple acceleration processor nodes; A global virtual clock synchronization with nanosecond-level precision is established and maintained among the various accelerated processor nodes through a Sideband management channel independent of the main data channel; Based on the global virtual clock synchronization, a forward credit flow control mechanism with time window constraints is used for collaborative reservation of communication resources, including: before data transmission, the sending node sends a credit request containing future transmission time window information to the receiving node through the Sideband management channel; the receiving node returns a credit token containing an authorized time window through the Sideband management channel based on its local resource status. The sending node sends data packets to the receiving node through the main data channel within the authorized time window; The receiving node performs timeliness verification and processing on the received data packets based on the global virtual clock synchronization and the authorized time window; After the data processing is completed, the receiving node sends a credit recovery notification to the sending node through the Sideband management channel.

2. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 1, characterized in that, Establishing point-to-point physical connections based on open standards includes: A passive copper cable conforming to a preset industry standard is used as the transmission medium, and both ends of the passive copper cable are connected to the acceleration processor node through a standardized physical interface; During the link initialization phase, the acceleration processor node reads the length parameter stored in the passive copper cable and configures the physical layer signal conditioning circuit based on the length parameter.

3. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 1, characterized in that, Establishing and maintaining global virtual clock synchronization with nanosecond-level precision includes: A distributed election protocol is run to elect a master node from each of the aforementioned accelerated processor nodes as the initial clock source; The master node periodically broadcasts a synchronization frame containing its local timestamp through the Sideband management channel. The remaining slave nodes calibrate their local clocks based on the received synchronization frames and the known delays of their transmission paths.

4. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 3, characterized in that, Calibrate the local clock from the node, specifically including: Based on the length parameter L read from the passive copper cable and combined with the preset electrical signal propagation speed, the transmission delay Δt is calculated. The local timestamp broadcast by the master node The transmission delay Δt is added to this, serving as the local calibration target time.

5. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 3, characterized in that, When the computing cluster is deployed in a ring topology and the number of nodes exceeds a preset threshold, establishing and maintaining global virtual clock synchronization also includes: Each slave node simultaneously listens to synchronization frames forwarded from the master node and its neighboring nodes; Each node maintains a synchronization frame delay table and runs a dynamic path selection algorithm to select the node corresponding to the path with the minimum observation delay as an auxiliary synchronization source. The clock information from the master node and the auxiliary synchronization source is weighted and fused to fine-tune the local clock correction parameters.

6. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 1, characterized in that, Send a credit request containing information about the future sending time window, specifically including: The sending node determines the expected start time of transmission based on the amount of data to be transmitted, the global virtual clock synchronization information, and the upper-layer scheduling information. ; Based on the link bandwidth and the amount of data to be sent, the transmission time required is calculated, and thus the end time of the requested time window is obtained. ; The width of the request time window is dynamically adjusted according to the service type of the communication task. The amount of data to be sent and the request time window The service type identifier is encapsulated into a credit request message and sent through the Sideband management channel.

7. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 1, characterized in that, The receiving node makes credit authorization decisions based on its local resource conditions, specifically including: Parse the credit request message to obtain the number of bytes to be requested and the request time window; Query the free capacity of the local receive buffer to determine if it meets the application requirements; If the conditions are met, resources are reserved according to the preset buffer management strategy, and the sender's credit request behavior is monitored to assess congestion tendency. Based on the evaluation results, the final authorization time window boundary is determined, a credit token containing the authorization time window is generated, and returned through the Sideband management channel.

8. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 1, characterized in that, The sending node sends data packets through the main data channel, specifically including: At the start of the authorized time window, data transmission is triggered synchronously based on the global virtual clock; Generate a sending timestamp for each data packet and embed it in its header; The main data channel uses differential signal encoding and multi-channel aggregation technology to achieve high-speed data transmission, and does not generate any reverse acknowledgment or congestion feedback signals during transmission.

9. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 1, characterized in that, The receiving node performs timeliness verification and processing on the data packets, specifically including: Parse the received data packets and extract their header sending timestamp and associated credit token identifier; Read the local synchronous clock to obtain the data packet reception time And query the corresponding authorization time window boundary based on the credit token identifier. ; The receiving time is determined by a hardware comparison circuit. Does it meet the requirements? ; If the conditions are met, the data packet payload will be delivered using a zero-copy method; otherwise, the data packet will be marked as abnormal and an exception handling procedure independent of the normal reception process will be triggered.

10. The open, multifunctional, low-latency, and high-stability interconnection method according to claim 1, characterized in that, Sending credit recall notifications, specifically including: After the receiving node has finished processing the data payload, it caches the credit limit to be reclaimed and the corresponding credit token identifier in a local queue. When the accumulated amount in the queue reaches a preset threshold or a timeout mechanism is triggered, one or more cached credit recovery entries are packaged into a credit recovery notification message. The credit recovery notification message is asynchronously sent to the sender node through the Sideband management channel; The sending node updates its local available credit pool based on the token identifier in the credit recovery notification message.

Images