A multi-card NPU interconnection clock system supporting channel spread spectrum modulation, a multi-card NPU system and a clock control method thereof

Abstract

The application discloses a multi-card NPU interconnection clock system supporting channel spread spectrum modulation, a multi-card NPU system and a clock control method thereof. The multi-card NPU interconnection clock system comprises a first clock module, a cache module and a second clock module. The first clock module is arranged on a mainboard and is used for outputting a first-level reference clock with a spread spectrum characteristic, serving as a unified reference for frequency modulation of the multi-card NPU system. The cache module is arranged on the mainboard and is communicatively connected with the first clock module and each second clock module, and is used for synchronously distributing the first-level reference clock to eliminate frequency and phase differences among NPU subcards. The second clock module is arranged on each NPU subcard and is used for locally generating a second-level reference clock based on the first-level reference clock, serving as a reference clock of an RDMA channel in the NPU subcard. The multi-card NPU interconnection clock system is applied to the multi-card NPU system, can effectively suppress channel EMI, eliminate frequency slip phenomenon, reduce link buffer demand and optimize chip area.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to a multi-card NPU interconnect clock system supporting channel spread spectrum modulation, a multi-card NPU system, and a clock control method thereof. Background Technology

[0002] With the expansion of high-performance computing clusters, the number of interconnecting signal lanes between daughter cards in multi-GPU neural network processors (NPUs) continues to rise, leading to increasingly severe electromagnetic interference (EMI) problems. Introducing spread-spectrum clocking (SSC) technology into these signal interconnects has become the mainstream method in the industry for suppressing EMI radiation. Currently, multi-GPU NPU system interconnects commonly employ Remote Direct Memory Access (RDMA) interfaces and are widely based on separate clock architectures for their physical layer designs, such as... Figure 1 As shown in the diagram. In this architecture, each NPU daughter card is independently configured with a local clock chip to obtain the best clock quality and clock integrity for the chip output signal.

[0003] While the separation clock architecture offers advantages in terms of purity in single-node clocking, it suffers from inherent technical drawbacks in multi-card interconnect applications with simultaneous SSC (Self-Signaled Continuous Clock) functionality. First, when each daughter card independently enables SSC modulation, the lack of a globally unified frequency modulation reference leads to random divergence in the spread spectrum phase and frequency offset of each local clock generator. This significantly increases the instantaneous relative frequency difference between different daughter cards, making it impossible to maintain a stable interconnect clock reference. Second, to absorb cross-card frequency slippage and accumulated timing jitter, the physical layer transceivers in the signal path must be configured with extremely deep elastic buffers or data FIFOs to wait for the transceiver clock phases to realign, increasing the difficulty of timing margin design in the signal link. Finally, the introduction of deep buffer structures not only directly increases the area of ​​the RDMA interface chip module, consuming valuable silicon resources, but also significantly prolongs the dwell time of data frames in the link, causing a sharp increase in data transmission latency. This makes it difficult to simultaneously meet the comprehensive requirements of high-performance interconnects for low latency, high integration, and strong EMI suppression. Summary of the Invention

[0004] To address some or all of the problems in existing technologies, and in order to introduce SSC and reduce EMI without increasing the depth of the cache, the first aspect of this invention provides a multi-card NPU interconnect clock system supporting channel spread spectrum modulation, comprising: The first clock module, located on the motherboard, is used to output a primary reference clock with spread spectrum characteristics, serving as a unified reference for frequency modulation across the entire system. A cache module, which is mounted on the motherboard and communicatively connected to the first clock module and each of the second clock modules, is used to synchronously distribute the first-level reference clock to eliminate the frequency phase difference between each NPU daughter card. The second clock module is installed on each NPU daughter card and is used to generate a second-level reference clock locally based on the first-level reference clock, which serves as the reference clock for the RDMA channel in the NPU daughter card.

[0005] Furthermore, the cache module includes a multi-level cascaded buffer circuit, which is used to perform signal shaping and pre-fanout driving on the first-level reference clock.

[0006] Furthermore, the second clock module includes an independent oscillator and a filter network, wherein the filter network is used to filter out spread spectrum modulation ripple in the primary reference clock.

[0007] Based on the multi-card NPU interconnect clock system described above, a second aspect of the present invention provides a multi-card NPU system, comprising: Motherboard; As described above, a multi-card NPU interconnect clock system; The NPU daughter card includes a system-on-chip (SoC), the clock input of which is connected to the second clock module of the multi-card NPU interconnect clock system.

[0008] Furthermore, the system-on-a-chip includes: The clock input pin of the Remote Direct Memory Access Physical Layer (RDMA PHY) is coupled to the output of the second clock module. A phase-locked loop (PLL) is connected to the output of the second clock module via an I / O port and is used to perform high-frequency multiplication on the secondary reference clock output by the second clock module to generate a high-frequency application clock, which is then sent to the Remote Direct Memory Access Digital Link Layer (RDMA MAC) and the Remote Direct Memory Access Physical Coding Sublayer (RDMA PCS). Remote Direct Memory Access Digital Link Layer (RDMA MAC) is used for data frame encapsulation, parsing, addressing, flow control, and lossless scheduling; The Remote Direct Memory Access Physical Coding Sublayer (RDMA PCS) is used for encoding / decoding, synchronization, and FEC to convert MAC frames into bitstreams.

[0009] Furthermore, a shallow data cache register is provided between the RDMA PHY, RDMA PCS, and RDMA MAC, wherein the depth setting value of the shallow data cache register is lower than the cache threshold of the traditional spread spectrum architecture.

[0010] A third aspect of the present invention provides a clock control method for a multi-card NPU system as described above, comprising: The first clock module oscillates and outputs a first-level reference clock with spread spectrum clock characteristics; The primary reference clock is synchronously transmitted to the physical interface of each NPU daughter card through the caching module; The first-level reference clock is received by the second clock module on each NPU daughter card, and a second-level reference clock is generated based on the first-level reference clock and input to the corresponding NPU SoC chip as the operating reference clock for the RDMA channel.

[0011] Furthermore, the first-level reference clock, which oscillates and outputs a spread-spectrum clock characteristic through the first clock module, includes: Configure the spread spectrum modulation center frequency and modulation bandwidth parameters of the first clock module; The noise shaping circuit of the first clock module is activated to output a first-level reference clock with the target amplitude modulation characteristics.

[0012] Furthermore, using the secondary reference clock as the operating reference clock for the RDMA channel includes: The edge signal of the secondary reference clock is captured through the remote direct memory access physical layer; A data sampling window is established at the physical layer via remote direct memory access based on the secondary reference clock; The sampled data is temporarily stored in a shallow cache register by passing the physical layer through remote direct memory access; The secondary reference clock is transmitted to the internal phase-locked loop via the I / O port. At the same time, the phase-locked loop performs frequency synthesis operations to generate a high-frequency application clock for use by the remote direct memory access digital link layer and the physical coding sublayer.

[0013] Furthermore, the clock control method further includes: The cache module performs fan-out driving and impedance matching processing on the primary reference clock to maintain signal integrity in long-distance motherboard traces.

[0014] Furthermore, the clock control method further includes: The primary reference clock is internally low-pass filtered and phase-aligned by the second clock module on each NPU daughter card to filter out spread spectrum modulation ripple.

[0015] This invention discloses a multi-card NPU interconnect clock system supporting channel spread spectrum modulation, a multi-card NPU system, and its clock control method. A primary clock is set on the motherboard to provide a consistent frequency reference for each daughter card. A secondary clock is generated locally by a clock chip on each daughter card to maximize the quality of the reference clock. Based on this, the spread spectrum characteristics of the RDMA path clock are generated through the primary clock, ensuring that the path clock sees the same frequency for each daughter card while performing SSC modulation to suppress EMI. Furthermore, based on this interconnect clock system, only a shallow buffer is needed between the RDMA PHY and RDMA PCS, and the size of the RDMA module does not increase significantly. This multi-card NPU interconnect clock system effectively suppresses channel EMI, eliminates frequency slippage, reduces link buffer requirements, and optimizes chip area and latency performance. Attached Figure Description

[0016] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the present invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0017] Figure 1 This diagram illustrates the structure of a discrete clock architecture in the prior art. Figure 2 This diagram illustrates the structure of a multi-card NPU interconnect clock system supporting channel spread spectrum modulation according to an embodiment of the present invention. Figure 3 A schematic diagram of the structure of an NPU daughter card according to an embodiment of the present invention is shown; and Figure 4 The diagram shows a flowchart of a clock control method for a multi-card NPU system according to an embodiment of the present invention. Detailed Implementation

[0018] In the following description, the invention is described with reference to various embodiments. However, those skilled in the art will recognize that the embodiments may be practiced without one or more specific details or in conjunction with other alternatives and / or additional methods or components. In other instances, well-known structures or operations are not shown or described in detail so as not to obscure the inventive points of the invention. Similarly, for illustrative purposes, specific numbers and configurations are set forth to provide a comprehensive understanding of embodiments of the invention. However, the invention is not limited to these specific details. Furthermore, it should be understood that the embodiments shown in the drawings are illustrative representations and are not necessarily drawn to scale.

[0019] In this specification, references to "an embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. The phrase "in one embodiment" appearing throughout this specification does not necessarily refer to the same embodiment in all instances.

[0020] It should be noted that the embodiments of the present invention describe the method steps in a specific order; however, this is only for illustrating the specific embodiment and not for limiting the order of the steps. On the contrary, in different embodiments of the present invention, the order of the steps can be adjusted according to actual needs.

[0021] To address the limitations of existing discrete clock architectures in overcoming cross-card frequency mismatch, increased cache depth, and latency-area imbalance when introducing spread spectrum modulation, this invention discloses a multi-card NPU interconnect clock system supporting channel spread spectrum modulation. It employs a two-level clock distribution architecture to reconstruct the timing reference for multi-card NPU interconnection. By injecting a unified SSC modulation source at the motherboard level, each daughter card's local clock chip uses the same modulation frequency as its locking reference, thus eliminating instantaneous frequency differences between cards. Simultaneously, based on this synchronization mechanism, data alignment between the RDMA PHY and PCS modules requires only a shallow cache layer. Specifically, a first-level clock chip is placed at the motherboard level to generate a first-level reference clock with spread spectrum modulation (SSC) characteristics. This clock is synchronously fed to each NPU daughter card through a buffer module on the motherboard. Each daughter card then receives this shared reference signal using a locally placed second-level clock chip, and after re-phase-locking and regeneration, outputs the second-level reference clock to the NPU SoC. Through this architecture replacement, each daughter card can maintain the purity of its local clock signal while completely eliminating cross-card frequency mismatch caused by multiple cards independently enabling SSC. Since the relative frequencies between cards remain consistent, the signal path can transmit stably without configuring a very deep data buffer, thereby achieving a significant reduction in chip module area and an effective reduction in data transmission latency.

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings of the embodiments.

[0023] Figure 2 This diagram illustrates the structure of a multi-card NPU interconnect clock system supporting channel spread spectrum modulation according to an embodiment of the present invention. Figure 2As shown, a multi-card NPU interconnect clock system supporting channel spread spectrum modulation includes a first clock module 201, a cache module 202, and a second clock module 203. The first clock module 201 and cache module 202 are mounted on the motherboard, while the second clock module 203 is mounted on each NPU daughter card. The first clock module 201 outputs a primary reference clock clk1 with spread spectrum characteristics as a unified reference for system-wide frequency modulation. The cache module 202 synchronously distributes the primary reference clock to eliminate frequency phase differences between NPU daughter cards. The second clock module 203, based on the primary reference clock, locally generates a secondary reference clock clk2 as the reference clock for the RDMA channels in each NPU daughter card.

[0024] In one embodiment of the present invention, the cache module 202 is electrically connected to the output of the first clock module 201 and extends to the physical interface of each NPU daughter card. Internally, it includes a multi-active buffer circuit to perform signal shaping and fan-out driving on the primary reference clock clk1, thereby maintaining the integrity of the first reference clock signal during long-distance motherboard routing. In another embodiment of the present invention, the output of the cache module 202 is configured as a differential clock signal format to reduce crosstalk interference during motherboard routing.

[0025] In one embodiment of the present invention, the second clock module 203 adopts an independent oscillator architecture, and its input terminal only receives the first-level reference clock clk1 as a frequency and phase-locked source. In another embodiment of the present invention, the second clock module 203 may also be internally configured with a filtering network to filter out the spread spectrum modulation ripple in the first-level reference clock clk1, so as to output a clean second-level reference clock clk2.

[0026] Based on the multi-card NPU interconnect clock system described above, the present invention also provides a multi-card NPU system, which includes the multi-card NPU interconnect clock system described above. Figure 3 A schematic diagram of the structure of an NPU daughter card according to an embodiment of the present invention is shown. Figure 3As shown, the NPU daughter card also includes a System on Chip (SoC). The clock input of the SoC is connected to the output of the second clock module of the multi-card NPU interconnect clock system, and is used to receive the secondary reference clock clk2 as the channel operating reference, so that the data paths between the daughter cards maintain instantaneous frequency consistency under spread spectrum modulation, thus eliminating the need for deep buffering. The NPU SoCs on each NPU daughter card are interconnected through a high-speed interconnect channel, and the clock domain of the high-speed interconnect channel is uniformly defined by the secondary reference clock clk2. As mentioned above, in one embodiment of the present invention, the high-speed interconnect channel adopts the RDMA interface protocol, and its physical layer clock recovery circuit uses the secondary reference clock clk2 as the synchronization reference.

[0027] In one embodiment of the present invention, the system-on-a-chip integrates a Remote Direct Memory Access Physical Layer (RDMA PHY), a Remote Direct Memory Access Digital Link Layer (RDMA MAC), a Remote Direct Memory Access Physical Coding Sublayer (RDMA PCS), and a Phase-Locked Loop (PPL). The clock input pin of the RDMA PHY is coupled to the output of the second clock module. The RDMA MAC is used for data frame encapsulation, parsing, addressing, flow control, and lossless scheduling, while the RDMA PCS is used for encoding / decoding, synchronization, and FEC to convert MAC frames into bitstreams. As mentioned above, a shallow data buffer register is provided between the RDMA PHY, RDMA MAC, and RDMA PCS, wherein the depth setting value of the shallow data buffer register is lower than the buffer threshold of a traditional spread spectrum architecture. The PPL is electrically connected to the second clock module through an internally integrated I / O port, and is used to perform high-frequency multiplication on the secondary reference clock clk2 to generate a high-frequency application clock clk3 for use by the RDMA PCS and RDMA MAC.

[0028] Since the RDMA PHY layer relies on a reference clock for signal sampling and equalization, while the RDMA PCS and RDMA MAC layers rely on a high-frequency clock for data encoding and protocol processing, a common SSC reference clock can be directly connected to the RDMA PHY, and a high-frequency clock can be derived to the RDMA PCS via an internal PLL. This ensures that the clock phase relationship remains constant throughout the link. If the clock derivation path does not maintain synchronization from the same source, additional jitter may be introduced. However, the PLL frequency division architecture from the same source ensures phase continuity. Based on this, the NPU daughter card structure of this embodiment not only ensures that the SSC characteristics are accurately transmitted to the physical layer transceiver to achieve channel-level EMI suppression, but also meets the rigid requirements of upper-layer protocol modules for high-frequency operating clocks through the PLL derivation mechanism, thereby achieving a balance between suppressing electromagnetic interference and maintaining high throughput.

[0029] Figure 4 This diagram illustrates a clock control method for a multi-GPU NPU system according to an embodiment of the present invention. Figure 4 As shown, a clock control method for a multi-card NPU system as described above includes: First, in step 401, the motherboard generates a primary reference clock. The primary reference clock, exhibiting spread spectrum clock characteristics, is output through the oscillation of the first clock module on the motherboard. In one embodiment of the invention, the spread spectrum modulation center frequency and modulation bandwidth parameters of the first clock module are first configured, and then the noise shaping circuit of the first clock module is activated to output a primary reference clock with the target amplitude modulation characteristics. Specifically, a preset SSC waveform table is loaded into the chip memory, and then the oscillator resonant frequency is controlled according to the waveform table to perform periodic triangular wave modulation, outputting the modulated primary reference clock clk1. Next, in step 402, the reference clock is distributed. The primary reference clock is synchronously transmitted to the physical interfaces of each NPU daughter card through the caching module. In one embodiment of the present invention, when distributing the reference clock, the primary reference clock clk1 is further fan-out driven and impedance matched through the caching module to maintain signal integrity in long-distance motherboard traces; Finally, in step 403, the daughter card generates a secondary reference clock. The second clock module on each NPU daughter card receives the primary reference clock and generates a secondary reference clock based on it, which is then input to the corresponding NPUSoC chip as the operating reference clock for the RDMA channel. In one embodiment of the invention, after receiving the primary reference clock clk1, the second clock module on each NPU daughter card first performs internal low-pass filtering and phase alignment processing to filter out spread spectrum modulation ripple, thereby outputting a stable secondary reference clock clk2.

[0030] After the NPU SoC's RDMA PHY captures the edge signal of the secondary reference clock clk2, it establishes a data sampling window based on clk2 and passes the sampled data to a shallow buffer register for temporary storage. Simultaneously, the secondary reference clock clk2 is transmitted to an internal phase-locked loop (PLL) via an I / O port. The PLL then performs frequency synthesis to generate a high-frequency application clock for use by the remote direct memory access digital link layer and the physical coding sublayer. In one embodiment of the invention, when passing the sampled data to the shallow buffer register for temporary storage, the read / write pointers of the shallow buffer register are uniformly driven by the secondary reference clock clk2 to eliminate the risk of buffer overflow during cross-card transmission.

[0031] In order to ensure that the data transmission paths between the daughter cards maintain frequency synchronization during SSC modulation, it should be ensured that the primary reference clock clk1 received by each NPU daughter card has a completely consistent instantaneous frequency modulation curve. Based on this, in one embodiment of the present invention, the loop bandwidth of each daughter card is set to track the spread spectrum modulation rate of the primary reference clock clk1, thereby ensuring that the instantaneous frequency offset of the secondary reference clock clk2 maintains a linear mapping relationship with the primary reference clock clk1.

[0032] In one embodiment of the present invention, when the data transmission throughput between each NPU daughter card exceeds a preset threshold, the SSC modulation depth of the primary reference clock clk1 is dynamically adjusted to balance the electromagnetic interference suppression index and the channel data throughput.

[0033] In one embodiment of the present invention, during the system power-on initialization phase, phase deviation detection is performed on the secondary reference clock clk2 output by each NPU daughter card. If the deviation value exceeds the tolerance range, the modulation parameter reconfiguration command of the first clock module is triggered.

[0034] This invention discloses a multi-card NPU interconnect clock system, a multi-card NPU system, and a clock control method thereof that support channel spread spectrum modulation. It adopts a two-level clock distribution architecture to reconstruct the timing reference of the multi-card NPU interconnect. While ensuring that the channel clock is subjected to SSC modulation to suppress EMI, the clock frequency seen by each daughter card is the same. While effectively suppressing channel EMI, it can eliminate frequency slippage, reduce link buffer requirements, and optimize chip area and latency performance.

[0035] Although various embodiments of the invention have been described above, it should be understood that they are presented by way of example only and not as limitations. It will be apparent to those skilled in the art that various combinations, modifications, and alterations can be made without departing from the spirit and scope of the invention. Therefore, the breadth and scope of the invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should be defined solely by the appended claims and their equivalents.

Claims

1. A multi-card NPU interconnect clock system supporting channel spread spectrum modulation, characterized in that, include: The first clock module, located on the motherboard, is configured to output a first-level reference clock with spread spectrum characteristics, serving as a unified reference for frequency modulation in a multi-card NPU system. A cache module, which is located on the motherboard and is communicatively connected to the first clock module and each of the second clock modules, is configured to synchronously distribute the first-level reference clock to eliminate the frequency phase difference between each NPU daughter card; The second clock module is set on each NPU daughter card and is configured to generate a second-level reference clock locally based on the first-level reference clock, which serves as the reference clock for the RDMA channel in the NPU daughter card.

2. The multi-card NPU interconnect clock system as described in claim 1, characterized in that, The cache module includes multi-level cascaded buffer circuits, which are configured to perform signal shaping and pre-fanout driving on the first-level reference clock.

3. The multi-card NPU interconnect clock system as described in claim 1, characterized in that, The second clock module includes an independent oscillator and a filter network, wherein the filter network is configured to filter out spread spectrum modulation ripple in the primary reference clock.

4. A multi-GPU NPU system, characterized in that, include: Motherboard; The multi-card NPU interconnect clock system as described in any one of claims 1 to 3; The NPU daughter card includes a system-on-a-chip (SoC), the clock input of which is connected to the second clock module of the multi-card NPU interconnect clock system.

5. The multi-card NPU system as described in claim 4, characterized in that, The system-on-a-chip includes: The remote direct memory access physical layer has its clock input pin coupled to the output of the second clock module; A phase-locked loop (PLL) is connected to the output of the second clock module via an I / O port. It is configured to perform high-frequency multiplication on the secondary reference clock output by the second clock module to generate a high-frequency application clock, which is then sent to the remote direct memory access digital link layer and the remote direct memory access physical coding sublayer. A shallow data buffer register is provided between the remote direct memory access physical layer and the remote direct memory access digital link layer and the remote direct memory access physical coding sublayer. The depth setting value of the shallow data buffer register is lower than the buffer threshold of the traditional spread spectrum architecture.

6. A clock control method for a multi-card NPU system as described in claim 5, characterized in that, include: The first clock module oscillates and outputs a first-level reference clock with spread spectrum clock characteristics; The primary reference clock is synchronously transmitted to the physical interface of each NPU daughter card through the caching module; The first-level reference clock is received by the second clock module on each NPU daughter card, and a second-level reference clock is generated based on the first-level reference clock and input to the corresponding NPU SoC chip as the operating reference clock for the RDMA channel.

7. The clock control method as described in claim 6, characterized in that, The first-level reference clock, which oscillates and outputs a spread-spectrum clock characteristic through the first clock module, includes: Configure the spread spectrum modulation center frequency and modulation bandwidth parameters of the first clock module; The noise shaping circuit of the first clock module is activated to output a first-level reference clock with the target amplitude modulation characteristics.

8. The clock control method as described in claim 6, characterized in that, Using the secondary reference clock as the operating reference clock for the RDMA channel includes: The edge signal of the secondary reference clock is captured through the remote direct memory access physical layer; A data sampling window is established at the physical layer via remote direct memory access based on the secondary reference clock; The sampled data is temporarily stored in a shallow cache register by passing the physical layer through remote direct memory access; The secondary reference clock is transmitted to the internal phase-locked loop via the I / O port. At the same time, the phase-locked loop performs frequency synthesis operations to generate a high-frequency application clock for use by the remote direct memory access digital link layer and the physical coding sublayer.

9. The clock control method as described in claim 6, characterized in that, Also includes: The cache module performs fan-out driving and impedance matching processing on the primary reference clock to maintain signal integrity in long-distance motherboard traces.

10. The clock control method as described in claim 6, characterized in that, Also includes: The primary reference clock is internally low-pass filtered and phase-aligned by the second clock module on each NPU daughter card to filter out spread spectrum modulation ripple.

Images