Retimer training with state-state machine synchronization across multiple integrated circuit dies

Abstract

Methods and systems are described herein for exchanging retimer training and state-state machine (RTSSM) state information between a plurality of circuit dies of a multi-chip module using a ring bus for carrying multi-bit channel state signals in a plurality of time slots and interconnecting the plurality of circuit dies into a circuit die ring, where the ring bus is configured to receive a plurality of channel state signals in a plurality of time slots. And each circuit bare chip outputs the stored summarized RTSSM state information to the next circuit bare chip of the ring through the ring bus, and stores the summarized RTSSM state information from the previous circuit bare chip of the ring until each circuit bare chip accumulates the complete multi-bare chip RTSSM state information of all the circuit bare chips. Each circuit die may synchronously perform state switching within the upstream and / or downstream RTSSM of the circuit die according to the complete multi-die RTSSM state information.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 383,192, filed November 10, 2022, by Alexander Koch, entitled “Retimer Training and State Machine Synchronization Across Multiple Integrated Circuit Dies,” which is incorporated herein by reference in its entirety for all purposes.

[0003] References

[0004] PCIe Base Specification (Version 1.0, Revision 6.0.1, September 13, 2022, www.pcisig.com / specifications), hereinafter referred to as the [PCIe Specification]

[0005] PCIe Retimer Test Specification (Version 1.0, Revision 4.0, June 10, 2022, www.pcisig.com / specifications) Background Art

[0006] With the data rate of PCIe 5.0 (32Gbps) increasing from previous generations (for example, PCIe 4.0MAX has a data rate of 16Gbps), the channel reach is becoming shorter than before, and the need for retimers is becoming more apparent. Common channels include system boards, backplanes, cables, riser cards, and add-in cards. The loss of the connection structure across these channels (usually a combination of such channels and their slots) often exceeds the target loss requirement of -36dB at 16GHz. Retimers can extend the channel reach beyond the reach limit when a retimer is not used.

[0007] The retimer splits the link between the host (root complex, abbreviated as RC) and the device (endpoint) into two independent segments. Therefore, the retimer can re-establish a new forward PCIe link, and this re-establishment process includes retraining and appropriate equalization for the physical layer and link layer.

[0008] Redrivers are purely analog amplifiers that boost signals to compensate for attenuation. However, while boosting the signal, redrivers also increase noise, which often exacerbates jitter. In contrast, retimers incorporate both analog and digital logic to equalize the signal, extract its clock timing information, and output a signal with high amplitude and low noise and jitter. Furthermore, retimers maintain power states to keep system power consumption low.

[0009] The retimer specification debuted in PCIe 4.0 and is expected to continue in PCIe 5.0. Figure 1A and Figure 1B Shown are common application scenarios of retimers in some embodiments. Figure 1A A retimer is used and placed on the motherboard so that it is logically located between the PCIe root complex (RC) and the PCIe endpoint devices.

[0010] Figure 1B The scenario shown is using two retimers, where the first retimer is also located on the mainboard, while the second retimer is located on a riser card that serves as a connection between the mainboard and an add-in card, where the PCIe endpoint device is contained.

[0011] In complex PCIe systems, there may be far more PCIe endpoint devices than available PCIe ports. In such cases, a switch can be used to increase the number of PCIe ports. A switch connects multiple endpoint devices to a single root node and routes data packets to specific destinations, rather than mirroring data across all ports. A key feature of switches is bandwidth sharing, allowing all endpoint devices to share the root node's bandwidth. Summary of the Invention

[0012] The methods and systems described herein are used to exchange retimer training and state machine (RTSSM) state information between multiple circuit dies in a multi-chip module using a ring bus. The ring bus is configured to carry multi-bit channel state signals in multiple time slots and interconnect the multiple circuit dies into a circuit die ring. Each circuit die outputs stored aggregated RTSSM state information to the next circuit die in the ring via the ring bus and retains aggregated RTSSM state information from the previous circuit die in the ring until each circuit die has accumulated complete multi-die RTSSM state information for all circuit dies. Each circuit die can synchronously perform state switching within the upstream and / or downstream RTSSMs of the circuit die based on the complete multi-die RTSSM state information.

[0013] This "Summary" section provides a brief summary of the concepts described in detail in the "Detailed Description" section below. This "Summary" section is not intended to provide key or primary features of the claimed technical solution, nor is it intended to assist in determining the scope of the claimed technical solution. Other objectives and / or advantages of the embodiments of the present invention will become readily apparent to those skilled in the art by referring to the "Detailed Description" section and the accompanying figures. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1A and Figure 1BShown are two uses of a retimer in some embodiments.

[0015] Figure 2 In some implementations, a chip structure is provided for performing channel routing on data channels of a PCIe data link through a retimer circuit.

[0016] Figure 3 A chip structure that includes PCIe data links with lanes distributed across multiple circuit dies or "chiplets."

[0017] Figure 4 1 is another chip architecture for a PCIe data link having lanes distributed across multiple circuit dies in some embodiments, and this figure is a block diagram of a CDC buffer.

[0018] Figure 5 FIG. 4 is a block diagram of a retimer pseudo port (PP) in some embodiments.

[0019] Figure 6 Flowchart of a retimer training and status state machine (RTSSM) in some implementations.

[0020] Figure 7 FIG. 1 is a block diagram of a retimer circuit die in some embodiments.

[0021] Figure 8 FIG. 4 is a block diagram of RTSSMs grouped according to link configuration in some implementation modes.

[0022] Figure 9 FIG. 1 is a block diagram of a retimer comprised of two circuit dies in some embodiments, illustrating vertical synchronization between RTSSMs across multiple circuit dies.

[0023] Figure 10 FIG. 5 is a block diagram of a retimer comprised of four circuit dies in some embodiments, illustrating vertical synchronization between RTSSMs across multiple circuit dies.

[0024] Figure 11 FIG. 1 is a block diagram of horizontal synchronization between RTSSMs across multiple circuit dies in some implementations.

[0025] Figure 12 A block diagram of a ring bus that interconnects four circuit dies of a multi-chip module (MCM) in some embodiments.

[0026] Figure 13 FIG. 4 is a block diagram of a state synchronized pipeline (SSP) in some implementations.

[0027] Figure 14 Flowchart of a vertical synchronization method in some implementations.

[0028] Figure 15 Flowchart of a horizontal synchronization method in some implementation modes. DETAILED DESCRIPTION

[0029] While the ability to fully integrate multiple systems into the same integrated circuit is increasing, maintaining multiple chip systems and subsystems separately still has significant advantages. For non-limiting descriptive purposes, at least some aspects of the present invention described herein are illustrated in a system environment comprising at least one point-to-point communication interface connecting two integrated circuit chips, representing a root complex (i.e., a host) and an endpoint device, respectively, wherein the communication interface is supported by a number of data channels, each of which consists of four high-speed transmission line signal conductors.

[0030] A retimer typically consists of a physical physical layer (PHY) and retimer core logic. The PHY consists of a receiver and a transmitter. The PHY receiver is responsible for data reception, data deserialization, and clock recovery, while the PHY transmitter is responsible for data serialization and amplification before output transmission. The retimer core logic is responsible for de-skew (for multi-channel links) and rate adaptation to address frequency differences between ports on both sides.

[0031] Retimers provide additional value because they sit in the path between the root complex (e.g., CPU) and endpoint devices (e.g., cache blocks). Retimers can also incorporate integrated processing units (e.g., accelerators) to handle data processing in the path from the root complex to the endpoint device.

[0032] To achieve a highly flexible solution, the PCIe retimer uses a conventional PHY interface for the PCIe bus and a high-speed die-to-die interconnect for the data processing unit (DPU). The high-speed die-to-die interconnect enables extremely high-speed communication links between cores within the same package structure. The PCIe retimer circuit is a core (die) with a four-channel retimer and can be connected to the DPU core via a high-speed die-to-die interconnect. One, two, or four channels can form a multi-channel link, and data can be transmitted across all links. Alternatively, each channel can be constructed as a single-channel link. Each channel in the PCIe retimer has two PHYs, one at each end (i.e., the upstream port and the downstream port). Given the four channels, one PCIe retimer die uses eight PHYs. In addition, the PCIe retimer die also contains communication lines to enable the exchange of control information between two or more PCIe retimer dies.

[0033] With one (or more) PCIe retimer cores, the following structures can be constructed (these structures are described in further detail below):

[0034] -Quad-channel retimer;

[0035] - Single die with full-flexibility 4×4 static channel routing;

[0036] - Four-channel retimer with accelerator (DPU);

[0037] Two dies housed in the same package: a retimer die and a DPU die.

[0038] - Eight-channel retimer;

[0039] Two dies housed in the same package structure, but with limited static channel routing – Highly flexible 4×4 routing within the same die, but without the ability to cross die boundaries;

[0040] - Eight-channel retimer with full-flexibility channel routing;

[0041] Two dies housed in the same package structure use high-speed inter-die interconnects to route data across different die, but this incurs additional latency.

[0042] - Eight-channel retimer with accelerator (DPU);

[0043] Three dies housed in the same package: two retimer dies and one DPU die.

[0044] - Sixteen-channel retimer;

[0045] Four dies housed in the same package structure, but with limited static channel routing – Highly flexible 4×4 routing within the same die, but without the ability to move data across die boundaries.

[0046] PCIe retimer chip structure

[0047] Figures 2 to 4 Figure 1 shows various PCIe retimer circuit structures from a data flow perspective in some embodiments. In each figure, the package structure contains up to four dies. Figure 2Three optional channel routing schemes for a package structure containing a single die are shown. This type of implementation can be used as a four-lane PCIe retimer. Within the same circuit die, all data is routed from one port to another via channel routing logic. The original data multiplexer routes data between the ports of each data channel separately. Package structure 200 shows a feedthrough path, package structure 205 shows a twisted path, and package structure 210 shows a port mirror. Specifically, although only one direction is shown in package structure 210, another mirroring function exists in the opposite direction. In some embodiments, a serial deserializer (SD) connected to an upstream device such as a root complex can be located above each structure diagram, while a serial deserializer connected to an endpoint device can be located below each structure diagram, and vice versa.

[0048] Figure 3 Two dual-die configurations can be used within the same package. Package 305 can be used to implement an eight-lane PCIe retimer with minimal latency, achieved by utilizing a routing architecture where each lane within the same die is routed between upstream and downstream ports. The two dies exchange deskew information over a communication link to perform lane deskew across all eight lanes.

[0049] Figure 3 The illustrated package structure 310 may correspond to an eight-lane PCIe retimer circuit with full routing flexibility within the entire circuit die, at the expense of additional latency and power consumption due to die-to-die (D2D) interconnects. The raw data multiplexers in each PCIe retimer circuit die either route directly to the opposite port (as shown at 305) or via a high-speed die-to-die interconnect (as shown at 310). Routing via the high-speed die-to-die interconnect allows data to be passed to adjacent dies. In this use case, inter-channel deskew is performed directly on one of the dies, eliminating the need to exchange deskew information between chips.

[0050] Figure 4 A package structure 400 containing four dies is shown. This package structure can be used as a 16-lane PCIe retimer circuit. In this embodiment, the four dies exchange deskew information over a communication link to perform lane deskew across all 16 lanes. In this embodiment, no D2D interconnect is required.

[0051] Retimer Mode

[0052] Figure 5The retimer core logic for the data lanes is shown, and its operation is described below. This core logic consists of the PHY PCS block and the PHY MAC block. In the receive (RX) direction, data is split into an 8b / 10b path (for PCIe Gen 1 and Gen 2) and a 128b / 130b path (for PCIe Gen 3 through Gen 5). Depending on the path, either code group (comma) alignment or block alignment is performed. Logic for performing the 8b / 10b decoder is combined in the "PCS RX" block. The two data streams are combined and forwarded to the PHY MAC block. In 8b / 10b mode, data is aligned on eight-bit symbol boundaries (symbols start at bit 0, 8, 16, or 24). In 128b / 130b mode, data is block aligned (a new block starts at bit 0, and all subsequent thirty-two-bit blocks are aligned on bit 0). In the transmit (TX) direction, data from the PHYMAC block is processed in the "PCS TX" block. Data is divided into two data paths: the first and second generation data uses 8b / 10b encoding; the third to fifth generation data uses 128b / 130b encoding.

[0053] In the RX direction of the PHY MAC module, data is descrambled within the RX lane module and forwarded to rti2pfx, which converts the data in the "Retimer Internal Bus" (rti) format to the "PCS-Flexbus" (pfx) format used between RPCS modules. Simultaneously, the PCS data is forwarded to the training decoder, where the "TX Alignment" module synchronizes the switch between "Forward" and "Execute" modes. In "Forward" mode, data is obtained from the "TS Update" module (see below). In "Execute" mode, data is obtained from the Training Control Unit for link training.

[0054] In the TX direction, some data fields are partially updated to inform subsequent modules of the presence of a retimer on the data path between the root complex and the endpoint device. These updates are performed by the "TS Update" module, which is part of the "TX Alignment" module. In addition, other training decode modules extract data from the TX data stream to enable the Retimer Training Status And State Machine (RTSSM) to monitor control data in both directions. The RTSSM is the central control unit that switches between "forward" and "execute" modes to control link training and monitor the entire retimer core logic.

[0055] The symbol detection module extracts COM symbols, SKP ordered sets, or EIEOS ordered sets (128b / 130b) as part of TS1 / TS2 (8b / 10b) for deskewing. The deskew FIFO (elastic buffer) performs inter-lane alignment (deskewing) and rate adaptation to compensate for small frequency deviations between the receive and transmit clocks. Deskew and rate adaptation control are performed by the link conditioning control module. This module can handle varying numbers of lanes to support lane splitting. In the fully flexible eight-lane mode (feeding data through the D2D interface) and D2D transparent mode, the FIFO writer writes two codewords per clock cycle at a low frequency. After successful alignment, the module first generates an EIEOS block aligned with the ordered set boundary before forwarding the data. The link conditioning module stops data transmission and terminates the data flow by sending an EIOS block. The EIOS block is sent aligned with the ordered set boundary. The link conditioning module is also responsible for extracting data from the elastic buffer. Because the PCS-TX logic adds synchronization header bits to the 128b / 130b data stream, it also inserts idle periods to compensate for the resulting increased bandwidth. Specifically, in 128b / 130b, two bits are inserted for every 128 bits, resulting in an idle / inactive period every 64 clock cycles. Furthermore, the link conditioning module provides electrical idle information as sideband information within each symbol. This electrical idle information is used by the connected PHY to generate electrical idle within the high-speed serial TX channel. The generation of the electrical idle sideband information is synchronized with the outgoing data stream.

[0056] The clock domain crossing (CDC) FIFO is a drift buffer used to transparently forward data from one PHY (PIPE interface) to the opposing PHY. The CDC FIFO is used to perform clock domain crossing and can be four entries deep, but this depth is not considered a limitation. The FIFO depth can be designed to be small enough to minimize latency but large enough to maintain sufficient clearance between the read and write pointers so that they do not conflict.

[0057] like Figure 5As shown, the retimer core logic includes PCS encoders and decoders for 8b / 10b encoding / decoding for PCIe Gen 1 and Gen 2, and 128b / 130b encoding / decoding for PCIe Gen 3 through Gen 5. Note that these encoders and decoders are not required in operating modes employing PCIe Gen 6. This operating mode utilizes a flow control unit (FLIT) scheme, eliminating the need for 8b / 10b or 128b / 130b. In this embodiment, encoding / decoding is not required, but at the same time, PCIe Gen 6-specific functionality, such as FEC decoding (both partial and full decoding), is further included in the datapath in logic. Some functionality within the retimer core logic is shared, such as FIFO inter-lane deskew and rate adaptation.

[0058] Retimer Training and Status State Machine (RTSSM)

[0059] Figure 5 Also included is a retimer training and status state machine (RTSSM) for a given pseudo-port type for a given channel. This RTSSM is the central training and status state machine for the retimer. Both the root complex (RC) and endpoint device (EP) use a link training and status state machine (LTSSM), which is responsible for negotiating the link speed and link width between both link partners based on their nominal capabilities. The RTSSM within the retimer extracts this information and sets the retimer's data rate and link width accordingly. Furthermore, the RTSSM can limit the speed and link width via configuration registers accessible by the CPU subsystem. The data rate fields in TS1 and TS2 need to be updated accordingly. The RTSSM stores the negotiated link parameters in a status register accessible by the CPU subsystem. In the PCIe retimer described in this article, each data lane of the link includes two RTSSMs: one for the upstream direction and one for the downstream direction. Methods and systems are described herein for synchronizing the RTSSMs of a data link (i) when pseudo ports of the same type are distributed across circuit boundaries and RTSSM state information (RTSSM_vsync) for the same pseudo port type is distributed between circuit dies, and (ii) when data lanes across circuit die boundaries and lane-specific state information (RTSSM_hsync) associated with the RTSSM of the same lane with the opposite pseudo port type is distributed between circuit dies. Two operating modes for a multi-die PCIe data link are discussed below, each employing synchronization techniques.

[0060] like Figure 5 As shown, RTSSM is connected to the following two synchronization channels: an hsync channel "RTSSM_hsync" and a vsync channel "RTSSM_vsync" for performing horizontal synchronization and vertical synchronization respectively.

[0061] The retimer described herein provides a highly flexible channel routing configuration scheme between upstream pseudo-ports and downstream pseudo-ports by using two RTSSMs for each channel within any given PCIe data link - one RTSSM located in the upstream PP and the other RTSSM located in the downstream PP. Thus, for any given PCIe data link with N channels, there are 2×N RTSSMs active. The embodiment described herein is used to synchronize all RTSSMs participating in a PCIe data link by exchanging inter-pseudo-port (inter-PP) RTSSM status information between two RTSSMs participating in the same channel and having opposite pseudo-port types using a horizontal synchronization channel (also referred to as "horizontal synchronization" in this article). In addition, RTSSMs of the same pseudo-port type also exchange intra-pseudo-port (intra-PP) RTSSM status information using a vertical synchronization channel (also referred to as "vertical synchronization" in this article).

[0062] RTSSM status information between PPs can be exchanged, for example, during the receiver detection process. For example, after initiating receiver detection, the root complex interacts with the upstream RTSSM, while the endpoint device connects to the downstream RTSSM. Each downstream RTSSM initiates receiver (i.e., endpoint device) detection after receiving notification via the corresponding horizontal synchronization bus. Upon detecting an endpoint device, the downstream RTSSM can provide feedback to the upstream RTSSM via the horizontal synchronization bus, informing it of the detection and enabling it to begin subsequent link training processes.

[0063] Thus, the RTSSMs in the upstream and downstream RPCS modules form a single data channel, but need not always be in the same state and can exchange state information with each other. Table 1 below shows the horizontal synchronization state information exchanged between the RTSSMs in the same channel.

[0064]

[0065] Table I

[0066] As described above, in addition to horizontal synchronization, RTSSMs of the same pseudo-port type also exchange intra-pseudo-port (intra-PP) RTSSM state information to enable other RTSSMs of the same pseudo-port participating in a multi-lane PCIe data link to understand the current state of the state machine, thereby facilitating synchronized state switching for all RTSSMs of the same pseudo-port. The intra-PP RTSSM state information may include: logical "AND" conditions, for example, the RTSSM of a certain type of pseudo-port switches to a new state only when the condition exists on every lane; and logical "OR" conditions, for example, the RTSSM switches to a new state as long as the condition exists on any lane.

[0067] Figure 6The block diagram of the RTSSM is to meet the standard requirements of the PCI base specification and the special requirements listed in the specification. Figure 6 As shown, the RTSSM state diagram includes the Electrical Idle state. During a basic reset, the state machine starts in the Electrical Idle state, with all pseudo ports in high impedance. Shortly after reset is released, the transmitter within the pseudo port performs receiver detection and propagates the results to the corresponding channels of the other pseudo ports (receiver impedance propagation). The LTSSMs within the root complex and endpoint devices connected at both ends of the data link are in the Detect state and perform receiver detection. Upon detecting low receiver impedance, the LTSSM enters the Polling state and begins transmitting TS1 (Training Sequence 1 Ordered Set). Subsequently, the PCIe link starts at 2.5GT / s, and the PHY's analog circuitry detects the state machine exiting the Electrical Idle state at 2.5GT / s. In other words, TS1 causes the state machine to exit the Electrical Idle state.

[0068] If TS1 arrives from only one pseudo-port, the retimer connects to the load board and switches to the Compliant Load Board state. In PCIe applications, the retimer is connected to the PCIe link, and because both LTSSMs at the end of the link are in the Polling state, TS1 is received at both pseudo-ports. Accordingly, the RTSSM switches to the Training Set Forwarding state. The RTSSM's detailed operation is more complex, and additional states are added to meet all requirements.

[0069] In the Training Set Forwarding State, data is sent between pseudo-ports. This data contains the Training Ordered Sets, including TS1 and TS2. A retimer switches to the Non-Training Set Forwarding State upon receiving multiple Logical Idle patterns. The LTSSM in the root complex and endpoint devices sends Logical Idle patterns when transitioning from the Configuration or Recovery states to the L0 state.

[0070] In execution mode, when one of the following three conditions is met, the system enters one of the three corresponding states from the training set forwarding state:

[0071] - If the equalization control field in TS1 is equal to "10b" several times, the state switches to the equalization state;

[0072] -If the pseudo-port receives multiple TS1 or TS2 with the loopback bit set, the RTSSM switches the state to the slave loopback mode;

[0073] - If an Electrical Idle ordered-set is detected or Electrical Idle is inferred, the RTSSM switches to the Electrical Idle state.

[0074] Non-Training Set Forwarding: This state is used for forwarding data. Data is transmitted in the L0 state of the LTSSM in the root complex and on endpoint devices. When the LTSSM is in the L0 state, the RTSSM is in the Non-Training Set Forwarding state. When multiple TS1 and TS2 packets are received, the RTSSM switches to the Training Set Forwarding state.

[0075] Balance: The LTSSM for the root complex and endpoint devices is in the Recovery state. In the Balance state, the optimal transmitter equalization settings are determined for each link segment. Because each link segment for each pseudo-port is involved, the retimer performs equalization training on each link segment connected to the corresponding pseudo-port separately and then enters the Execute mode. After equalization is complete, the RTSSM returns to the Trained Set Forwarding state.

[0076] In the slave loopback state, the transmitter sends receive data, which also tests the receiver. The next state after the test is completed is the electrical idle state.

[0077] Compliance Load Board: The retimer connects to the load board and sends a compliance test pattern to test the transmitter. The next state after the test is completed is the electrical idle state.

[0078] RTSSM synchronization

[0079] Figure 7 FIG. 7 is a block diagram of a retimer circuit die 700 in some embodiments. As shown in the figure, Figure 7 The system includes eight ports, each of which has a PHY and corresponding core logic 702a-702d and 704a-704d. The RTSSM of each port may be included in the corresponding core logic. Figure 7 The illustrated structure is a four-lane PCIe link between a root complex 705 and an endpoint device 710. The retimer circuit die 700 also includes a die-to-die (D2D) interface 715 for communicating with other circuit dies within the multi-chip package structure. Such an interface can be, for example, a Universal Chiplet Interconnect Express (UCIe) interface or other inter-chip interconnect.

[0080] like Figure 7 As shown, the eight ports are subdivided into four ports of upstream pseudo ports (including core logic 702a-702d) and four ports of downstream pseudo ports (including core logic 704a-704d).

[0081] Figure 8 Figure 1 is a block diagram showing horizontal and vertical synchronization between the RTSSMs of the retimer circuit die operating in the PCIe data path. Although a four-lane link is shown, this should not be considered limiting. Figure 8 It includes four upstream RTSSMs 805a to 805d and four downstream RTSSMs 810a to 810d. As shown in the figure, there is a horizontal synchronization channel between RTSSM_1 805a and RTSSM_5 810a, and the horizontal synchronization channel includes hsync input selection logic 815 and 820 for RTSSM_5 and RTSSM_1 respectively. Each hsync input selection logic 815 may have an input terminal connected to each other RTSSM and select one of the input terminals according to the required channel configuration. Although Figure 8 The hsync input selection logic 815 and 820 are shown in the form of multiplexers, but other circuits may also be used. In some embodiments, the multiplexers receive input selection signals from configuration registers of the retimer circuit die. The configuration registers can be configured by the active CPU core within the retimer circuit die. The active CPU core can configure the entire configuration register space of the retimer circuit die according to the desired PCIe link configuration.

[0082] Similarly, vertical synchronization between RTSSMs on the same pseudo port ensures that all RTSSMs on the pseudo port are ready to switch before switching to a new state. Figure 8 In FIG, the vertical synchronization channel is shown as a set of parallel wires between upstream RTSSMs 805a-805d. Each upstream RTSSM provides vertical synchronization RTSSM status information to all other upstream RTSSMs. Figure 8 As shown, each RTSSM also includes a gated input for accepting or rejecting intra-PP RTSSM status information from other RTSSMs. The vertical synchronization input of each RTSSM is gated to ensure that each upstream RTSSM receives vertical synchronization RTSSM status information only from other upstream RTSSMs participating in the same PCIe link. In some embodiments, the CPU core configures the vsync channel in a similar manner by selectively enabling or disabling each gated input of each RTSSM according to the desired channel routing configuration. Figure 8 As shown, the gated inputs of RTSSM_1 are enabled for RTSSM status information received from RTSSM_2 to RTSSM_4 within the PP, and are disabled for RTSSM status information received from RTSSM_5 within the PP. In some embodiments, a full flexibility scheme, described in further detail below, is employed, wherein each RTSSM can receive vertical synchronization inputs not only from all other RTSSMs within the corresponding circuit die, but also from RTSSMs of other circuit dies.

[0083] As described above, the information exchanged via the horizontal and vertical synchronization channels of the RTSSM status information exchange channel can take the form of logical OR and logical AND conditions. A logical OR condition indicates that the RTSSM takes action if the condition occurs in any channel. A logical AND condition indicates that the RTSSM takes action only if the condition occurs in all channels. Furthermore, the logical OR and logical AND conditions exchanged by the RTSSM can have different meanings depending on the current state of the RTSSM.

[0084] Figure 8 To simplify the figure, only a portion of all the connections between the RTSSMs is shown. Therefore, it should be understood that although not explicitly shown in the figure, there are other similar connections between the RTSSMs.

[0085] In some embodiments, a device includes a plurality of pseudo ports (PPs), the plurality of PPs including an upstream PP and a downstream PP, each PP including one or more physical layer transceivers (PHYs). The device also includes a plurality of retimer training and status state machines (RTSSMs), each RTSSM being configured to manage a corresponding PHY. The device also includes channel routing logic configured to route data between each PHY of the upstream PP and a corresponding PHY of the downstream PP. The device also includes a horizontal synchronization (hsync) channel configured to exchange inter-PP RTSSM state information between each RTSSM of the upstream PP and a corresponding RTSSM of the downstream PP. The device also includes a vertical synchronization (vsync) channel configured to exchange corresponding intra-PP RTSSM state information between the RTSSMs of the upstream PP and between the RTSSMs of the downstream PP, respectively.

[0086] In some embodiments, the apparatus further includes a central processing unit (CPU) core for configuring an hsync channel and a vsync channel. In some embodiments, each RTSSM is configured to output inter-PP RTSSM status information and intra-PP RTSSM status information to all other RTSSMs. In some embodiments, each RTSSM includes an hsync input selection circuit, wherein the CPU core configures the horizontal synchronization channel by configuring the input selection circuit of each RTSSM to respectively receive inter-PP RTSSM status information of a corresponding other RTSSM having an opposite pseudo-port type. In some embodiments, the hsync input selection circuit is a multiplexing circuit. In some embodiments, the multiplexing circuit is configured to receive a selection input from a configuration register, wherein the configuration register is configured by the CPU core.

[0087] In some embodiments, each RTSSM includes gating inputs configured to selectively receive intra-PP RTSSM status information from RTSSMs having the same PP type and reject intra-PP RTSSM status information from RTSSMs having different PP types, wherein the CPU core configures the vertical synchronization channel by selectively enabling or disabling each gating input according to a desired channel configuration.

[0088] In some embodiments, the plurality of pseudo ports are located within a first retimer circuit die and a second retimer circuit die interconnected to the first retimer circuit die via a die-to-die (D2D) data interface and a D2D RTSSM synchronization channel. In some embodiments, the PHY of the upstream PP is located within the first retimer circuit die, wherein the PHY of the downstream PP is located within the second retimer circuit die. In some embodiments, the D2D RTSSM synchronization channel interfaces with a horizontal synchronization channel to exchange inter-PP RTSSM status information between the first and second retimer circuit dies.

[0089] In some embodiments, a first PHY of the upstream PP and a corresponding PHY of the downstream PP are located within a first retimer circuit die, wherein a second PHY of the upstream PP and a corresponding PHY of the downstream PP are located within a second circuit die. In some embodiments, a D2D RTSSM synchronization channel interfaces with a vertical synchronization channel to exchange intra-PP RTSSM state information between (i) the RTSSMs associated with the first and second PHYs of the upstream PP and (ii) the RTSSM associated with the corresponding PHY of the downstream PP.

[0090] In some embodiments, a method includes: routing data between physical layer transceivers (PHYs) of a plurality of pseudo ports (PPs), the plurality of PPs including an upstream PP and a downstream PP; managing each PHY with a respective plurality of retimer training and status state machines (RTSSMs); exchanging inter-PP RTSSM state information between each RTSSM of the upstream PP and a corresponding RTSSM of the downstream PP using a horizontal synchronization (hsync) channel; and exchanging respective intra-PP RTSSM state information between the RTSSMs of the upstream PP and between the RTSSMs of the downstream PP using a vertical synchronization (vsync) channel.

[0091] In some embodiments, the method further includes: grouping the PHYs of the plurality of PPs into upstream PPs and downstream PPs by configuring an hsync channel and a vsync channel by a central processing unit (CPU) core. In some embodiments, the method further includes: providing inter-PP RTSSM state information and intra-PP RTSSM state information of a given RTSSM to all other RTSSMs. In some embodiments, the method further includes: selecting, by each RTSSM of the PHY of a first PP among the plurality of PPs, the inter-PP RTSSM state information of a corresponding one of the other RTSSMs of the PHY of a second PP among the plurality of PPs. In some embodiments, the selection is performed by an hsync input selection circuit. In some embodiments, the method further includes: configuring the hsync input selection circuit by the CPU core. In some embodiments, the hsync input selection circuit is a multiplexing circuit, and the method includes: configuring the multiplexing circuit to receive a selection input from a configuration register, the configuration register being configured by the CPU core.

[0092] In some embodiments, the method further includes configuring a gating input of each RTSSM of the PHY of a first PP among the plurality of PPs to selectively receive intra-PP RTSSM status information from the remaining RTSSMs of the PHY of the first PP, while simultaneously rejecting the intra-PP RTSSM status information from the RTSSM of the PHY of a second PP among the plurality of PPs, wherein the CPU core configures the vertical synchronization channel by selectively enabling or disabling each gating input according to a desired channel configuration.

[0093] In some embodiments, the plurality of pseudo ports are located within a first retimer circuit die and a second retimer circuit die interconnected to the first retimer circuit die via a die-to-die (D2D) data interface and a D2D RTSSM synchronization channel. In some embodiments, the PHY of the upstream PP is located within the first retimer circuit die, and the PHY of the downstream PP is located within the second retimer circuit die. In some embodiments, the method further includes exchanging inter-PP RTSSM state information between the first and second retimer circuit dies via the D2D RTSSM synchronization channel.

[0094] In some embodiments, a first PHY of the upstream PP and a corresponding PHY of the downstream PP are located within a first retimer circuit die, wherein a second PHY of the upstream PP and a corresponding PHY of the downstream PP are located within a second circuit die. In such embodiments, the method further comprises exchanging intra-PP RTSSM state information between (i) the RTSSMs associated with the first and second PHYs of the upstream PP and (ii) the RTSSM associated with the corresponding PHY of the downstream PP via a D2D RTSSM synchronization channel interface.

[0095] Multi-chip RTSSM synchronization

[0096] The embodiments described herein include a multi-chip module (MCM) retimer capable of fully flexible channel routing in retimer mode. Accordingly, the RTSSM state information may also need to be exchanged across circuit die boundaries.

[0097] Figures 9 to 11 Shown are various multi-chip configurations of this type of retimer. Figure 9 Figure 1 is a block diagram of a PCIe retimer with 8 lanes across two circuit dies, Die_0 and Die_1. Figure 10 Figure 1 is a block diagram of a PCIe retimer with 16 channels across four circuit dies Die_0 to Die_3. Figure 9 and Figure 10 In the example, the upstream and downstream ports of a given channel are located in the same core. Therefore, the horizontal synchronization RTSSM status information can be directly combined with the above Figure 8 The vertical synchronization RTSSM state information is exchanged through the inter-chiplet connection between the circuit dies. Figure 9 and Figure 10 In , the total number of channels can be split. For example, Figure 9 It may include two 4×PCIe data links, each link communicating via a corresponding circuit die. Figure 10 Supports two 8×PCIe data links, each 8×PCIe data link communicates with two circuit dies. Figure 11 Figure 1 is a block diagram of a PCIe retimer with 8 lanes distributed across two chiplets, which enables full lane routing flexibility between the PHYs of the upstream and downstream PPs. Figure 11 In the example, all RTSSMs of the upstream PP are located in Die_0, while all RTSSMs of the downstream PP are located in Die_1. Accordingly, the vertical synchronization RTSSM state information of each PP's RTSSM can be exchanged directly, while the horizontal synchronization RTSSM state information is exchanged through the inter-chiplet connection. The number of chiplets and the number of data link channels are not limited. Figures 9 to 11As shown, each channel of the PCIe data link includes upstream and downstream core logics, and each core logic includes a corresponding RTSSM. Figure 12 1 is a block diagram of an apparatus in some embodiments. As shown, the apparatus includes a plurality of circuit dies 1205, 1210, 1215, 1220 for retiming a serial data link from a root complex to an endpoint device. A plurality of data lanes are distributed across the plurality of circuit dies and form a PCIe data link, each data lane including a corresponding upstream and downstream retimer training and status state machine (RTSSM). In some embodiments, as Figure 7 and Figure 8 As shown, the RTSSMs for the upstream and downstream pseudo-ports of a given data channel are contained within the same circuit die, while the RTSSMs with the same pseudo-port type are distributed across the multiple circuit dies. Figure 12 The embodiment shown has four circuit dies, but this should not be considered limiting. For example, a similar embodiment may include Figure 11 Two circuit dies for a fully flexible channel-routing octal retimer are shown. Figure 11 In the embodiment, all upstream pseudo ports of the data link are located in the first circuit die, and all downstream pseudo ports of the data link are located in the second die. The "RTSSM_hsync" synchronization information is exchanged across the chiplets.

[0098] The device also includes a die-to-die RTSSM synchronization channel, referred to herein as a "ring bus." Note that the term "ring bus" in the context of this specification may include a set of signal conductors such as wires connected in parallel. Figure 12 As shown, each signal conductor connects the multiple retimer circuit dies to each other in a ring shape. As shown in the figure, the ring bus ring_[8:0] is connected between each retimer circuit die and is used to carry a multi-bit channel status signal, which contains the summary RTSSM status information of each circuit die. These summary RTSSM status information together constitute the complete multi-die RTSSM status information. Although the ring bus ring_[8:0] is a 9-bit wide bus, this width should not be considered as a limitation. Figure 7 and Figure 8 In the illustrated operation mode, the summary RTSSM status information carried by the multi-bit channel status signal corresponds to the vertical synchronization RTSSM status information (RTSSM status information within PP) of the RTSSM with the same pseudo port type. Figure 11In the illustrated operating mode, the aggregated RTSSM status information carried by the multi-bit channel status signal corresponds to the horizontal synchronization RTSSM status information (inter-PP RTSSM status information) of the RTSSM of the same data channel, where the upstream and downstream pseudo ports are located in different circuit dies. The complete multi-die RTSSM status information corresponds to the aggregated RTSSM status information of all circuit dies. In some embodiments, each operating mode (such as Figure 9 and Figure 11 In some embodiments, the ring bus can be expanded to carry both inter-PP and intra-PP RTSSM status information by increasing the parallel interface size and / or adding time slots.

[0099] Multi-chip vertical synchronization

[0100] As mentioned above, the RTSSM typically switches from the previous state to the new state based on a logical AND condition (a condition where all channels are true) and a logical OR condition (a condition where any channel is true).

[0101] like Figure 12 As shown, each circuit die includes a state synchronization pipeline (SSP), such as SSP 1225. Each SSP 1225 may include, for example, an input register 1230 and an output register 1235. The SSP 1225 of a given circuit die is configured to: receive multi-bit channel status signals and accumulate complete RTSSM state information by incrementally storing the aggregated RTSSM state information from the remaining circuit dies in the input register 1230; and output the RTSSM state information associated with the given circuit die to the parallel ring bus via the output register 1235.

[0102] Figure 13 FIG1 is a block diagram of an SSP 1225 in some embodiments. As shown, the aggregated vertical RTSSM status information "RTSSM_vsync_i" is latched by input register 1230 according to the reference clock. After storage, output register 1235 outputs "RTSSM_vsync_o" according to the reference clock. The SSP 1225 also includes other registers 1340 and 1345 that are clocked on the opposite edge of the reference clock, for example, to constrain timing fluctuations. Although Figure 13 The example shows the situation where the vertical synchronization RTSSM status information is exchanged using the SSP logic. However, it should be understood that the SSP can also be used to exchange the horizontal synchronization RTSSM status information depending on the configuration of the retimer.

[0103] The RTSSM 1240 within each die is responsible for analyzing the aggregated RTSSM state information of the multiple die upon synchronous accumulation of complete multi-die RTSSM state information, and for synchronized state transitions between the die. For simplicity, only one RTSSM 1240 is shown for each die. However, it should be noted that each channel involved in a data link includes two RTSSMs, one upstream and one downstream. Furthermore, the upstream and downstream RTSSMs 1240 determine whether state transition conditions are met by analyzing the complete multi-die RTSSM state information for the same pseudo-port type.

[0104] exist Figure 12 In the embodiment, the ring bus is shown as a multi-conductor parallel bus that connects each circuit die to each other, and the number of conductors is N. In some embodiments, N is an integer greater than 1. Although the ring bus can also be a single conductor in some alternative embodiments, the following examples all include a multi-conductor parallel ring bus. The ring bus is used to circulate a multi-bit channel status signal through the multiple circuit dies within a ring cycle, wherein the ring cycle may include multiple time slots. In at least one non-limiting embodiment, the ring cycle includes nine time slots for propagating the summarized RTSSM status information through the ring bus so that each circuit die obtains the summarized RTSSM status information from the other circuit dies to synchronously accumulate the complete multi-die RTSSM status information.

[0105] In some embodiments, the SSP 1225 of each circuit die may include a corresponding time slot counter. The time slot counter of each SSP 1225 of each chip may be synchronized according to a synchronization bit that is propagated through each circuit die at predetermined positions within the multi-bit channel status signal. The remaining positions within the multi-bit channel status signal are used for logical "AND" and logical "OR" conditions of the aggregated RTSSM status information transmitted through each circuit die via the ring bus. In at least one embodiment, synchronizing the time slot counter within each chip includes: upon receiving the synchronization bit, setting the time slot counter of chip M to 2×M, where M={0, 1, 2, 3}. Specifically, the synchronization bit starts at chip 0 (1205) and initializes the count value of the time slot counter of chip 0 to "0". Subsequently, the synchronization bit is transmitted via the ring bus to Chip1 (1210) over two reference clock cycles, initializing the count value of Chip1's slot counter to "2." Simultaneously, the count value of Chip0's slot counter is also incremented to "2." The synchronization bit is then transmitted via the ring bus from Chip1 to Chip2 (1215) over two reference clock cycles, initializing the count value of Chip2's slot counter to "4." Simultaneously, the slot counters of Chip0 and Chip1 are also incremented to 4 during this period. Finally, with the count value of Chip3 (1220) initialized to "6," the count values of the remaining Chips are also "6," thus achieving synchronization of the slot counters. After the count values of the slot counters of the various Chips are synchronized, the summary status information of each Chip is transmitted via the ring bus, such that, for example, the summary status information of Chip0 is transmitted to Chip1, and the summary status information of Chip1 is simultaneously transmitted to Chip2, and so on. At the same time, chiplet 0 obtains the aggregated RTSSM state information from chiplet 3. Each chiplet obtains the aggregated RTSSM state information from the previous chiplet and outputs the currently held aggregated RTSSM state information at the same time. In the next few time slots, chiplet 0 passes the aggregated RTSSM state information of chiplet 3 to chiplet 1, and chiplet 1 passes the aggregated RTSSM state information of chiplet 0 to chiplet 2, and so on. While transmitting on the ring bus, the time slot counter of each SSP can continue to increment. When the time slot counters of all chiplets reach a predetermined value (for example, indicating that the ring cycle has been completed, that is, the accumulated predetermined value of the multi-die RTSSM state information has been completed), the upstream and downstream RTSSMs of each circuit die can analyze the complete multi-die RTSSM state information of the same pseudo-port type to synchronize state switching according to the prompts.

[0106] In some embodiments, the time slot counter of each chiplet can be used to distinguish the RTSSM status information of two different links. Figure 10In one embodiment, there may be two x8 lane links: one link spanning the top two circuit dies, Die_0 and Die_1, and the other link spanning the bottom two circuit dies, Die_3 and Die_4. In such an embodiment, the upstream and downstream RTSSM status information of the first and second links is transmitted within each circuit die via a ring bus, and a time slot counter in each circuit die is used to determine when to accept RTSSM status information within the bus. A specific example of a nine-slot ring bus is as follows:

[0107] Each circuit die can output its own logical AND and logical OR conditions for upstream and downstream RTSSMs in time slot 0 and time slot 1;

[0108] - Each circuit die outputs the logical AND and logical OR conditions for the previous circuit die in time slot 2 and time slot 3 (e.g., circuit die 805 outputs the logical AND and logical OR conditions for logic 820), and so on.

[0109] -Die_1 can save the RTSSM state information of Die_0 in time slots 2 and 3, and propagate (without saving) the RTSSM state information of Die_2 and Die_3 in time slots 4, 5, 6, and 7;

[0110] - Die_0 stores the RTSSM status information of Die_2 and Die_3 in time slots 2, 3, 4, and 5, and stores the RTSSM status information of Die_1 in time slots 6 and 7;

[0111] - Similarly, all other circuit bare chips may filter the RTSSM state information by only storing the RTSSM state information of the circuit bare chips participating in the same data link;

[0112] - Thus, at time slot 8, all circuit dies have saved the RTSSM state information of the circuit dies participating in the same data link, and thus can synchronously perform state switching according to the prompt.

[0113] In some embodiments, the slot counters of each chiplet can be synchronized during the initialization phase using a specific data pattern within the ring bus. In such embodiments, synchronization bits may not be used. After the initialization phase, the specific data pattern can be carried in the RTSSM update slot (i.e., the last slot of the ring cycle) to ensure that all slot counters remain synchronized. It should be noted that other synchronization methods besides the counter-based method described above can also be used, and such methods and systems should not be considered limiting.

[0114] Figure 10 Shown is a sixteen-channel retimer with 16 channels distributed across four circuit dies. Figure 10 In the retimer shown, similarly, horizontal synchronization information is exchanged directly via the on-chip RTSSM to RTSSM (R2R) interface, while vertical synchronization information is exchanged via the ring bus.

[0115] In one non-limiting embodiment, a multi-chip PCIe retimer includes up to four chiplets, each with four lanes. In such an embodiment, data links across multiple circuit dies and using more than four data lanes are synchronized via a chiplet-to-chiplet ring bus. Each lane receives status information for all other lanes. When the RTSSM of each chiplet analyzes logical "AND" and logical "OR" conditions, the current state of the RTSSM is also taken into account. After a state switch, the conditions are updated. The bits in the ring bus can have different meanings depending on the current state of the RTSSM.

[0116] In some embodiments, the number of active channels within a link can be configured. In such embodiments, inactive channels can have a "1" inserted into the multi-bit channel status signal for each "AND" condition and a "0" inserted into the multi-bit channel status signal for each "OR" condition. In some embodiments, the number of active channels is a power of two. In such embodiments, if the number of active channels is greater than four, the number of inactive channels can be only four or eight. In such configurations, one or more complete cores are inactive.

[0117] Figure 14 Flowchart of the method in some embodiments. Figure 14As shown, the method includes: each circuit die in a plurality of circuit dies collects aggregated RTSSM information for upstream and downstream RTSSM ports in the circuit die. The method also includes: outputting the aggregated RTSSM information to a ring bus while saving the aggregated RTSSM information placed in the ring bus by the previous circuit die in the ring, wherein the ring bus is connected to the next circuit die in the ring. If each circuit die has not yet fully accumulated complete RTSSM state information from all remaining circuit dies, each circuit die repeats the process of outputting the currently held aggregated RTSSM state information to the next circuit die in the ring via the ring bus while again saving the aggregated RTSSM information placed in the ring bus by the previous circuit die. After each circuit die has accumulated the complete RTSSM state information of all circuit dies (e.g., based on an indication of a time slot counter), each circuit die analyzes the upstream and downstream RTSSM information. If there is an indication prompting the RTSSM of the upstream and / or downstream pseudo ports to switch states, all upstream and / or downstream RTSSMs can perform state switching synchronously. The plurality of circuit dies may continue to exchange upstream and downstream pseudo-port summary RTSSM state information to ensure that all upstream and downstream RTSSMs maintain the same state.

[0118] Multi-chip horizontal synchronization

[0119] When horizontal synchronization is performed across multiple cores, the type of information exchanged is different from the vertical synchronization information. The horizontal synchronization information is, for example, as described above in conjunction with Table 1. Figure 11 The full-flexibility eight-channel retimer mode is shown, with the two RPCS modules for a given data channel located on different die. In this scenario, while vertical synchronization information is exchanged between the RTSSMs for all links in one direction (i.e., upstream and downstream pseudo-ports) on the same die, horizontal synchronization information is also exchanged across the die boundary because the upstream and downstream RPCS modules for the same channel are located on different dies.

[0120] The RTSSMs of the upstream and downstream pseudo-ports of a given channel need not be in exactly the same state. In at least one embodiment, the states of the upstream and downstream RTSSMs are based on their connectivity to the root complex and endpoint devices, respectively. For example, Figure 11During the boot process, the root complex can initiate a link training sequence. In this way, the RTSSM of the upstream pseudo-port of the top circuit die can enter one state, while the RTSSM of the downstream pseudo-port of the bottom circuit die can enter another state based on the negotiation results with the endpoint device. During the link training sequence, the RTSSM of the upstream pseudo-port switches together, and the RTSSM of the downstream pseudo-port also switches together. In addition, the upstream and downstream RTSSMs of a given channel exchange channel-specific information with each other. This channel-specific status information can, for example, indicate the following notification content: a device is connected to each pseudo-port; valid data is being transmitted; or the pseudo-port at one end of the channel is in an idle state.

[0121] In the case of multi-chip "horizontal synchronization", a time slot type ring bus can be used in a similar manner to the "vertical synchronization" case. Figure 11 In the illustrated eight-channel retimer, the multi-bit channel status signal may include nine time slots—one for each channel, with the ninth time slot being used to update the RTSSM state. In at least one embodiment employing an eight-channel data link, as shown Figure 11 As shown, a slotted ring bus can include nine time slots—one synchronization time slot and eight channel-specific time slots. That is, the RTSSM of each data channel outputs its channel-specific "RTSSM_hsync" information. Each time slot counter can be synchronized in the manner described above, and can be used by the retimer core logic of each channel to determine the time slot in which to retrieve "RTSSM_hsync" information from the corresponding logic of the other circuit die. For example, time slot 0 can correspond to the synchronization period, and time slot 1 corresponds to the information exchange between the upstream and downstream RTSSMs of channel 0. In time slot 1, the downstream RTSSM of port 0 outputs "RTSSM_hsync" information and simultaneously retrieves the "RTSSM_hsync" information placed on the ring bus by the upstream RTSSM of port 0. In time slot 2, the downstream RTSSM of port 1 outputs "RTSSM_hsync" information and simultaneously retrieves the "RTSSM_hsync" information placed on the ring bus by the upstream RTSSM of port 1, and so on.

[0122] Figure 15This is a flow chart of the horizontal synchronization method in some embodiments. As described above, the ring bus used for horizontal synchronization corresponds to the same physical connection as the ring bus used for vertical synchronization, and therefore operates in a similar manner. Specifically, the method includes: outputting channel-specific RTSSM information of one type (upstream / downstream) pseudo-port; and collecting channel-specific RTSSM information of another type (downstream / upstream) pseudo-port of the same channel. Each circuit die continuously exchanges its own pseudo-port channel-specific RTSSM status information with each other until each circuit die of each channel collects the channel-specific RTSSM status information of another circuit die. After completing the exchange of complete channel-specific RTSSM status information of all channels, each circuit die can analyze the channel-specific status information. If a given channel does not meet the expected upstream / downstream conditions, the upstream and / or downstream RTSSM of each circuit die can synchronously perform state switching.

Claims

1. A device, characterized in that include: a plurality of circuit dies for retiming serial data links from the root complex to endpoint devices; a plurality of data lanes distributed across the plurality of circuit dies and forming a PCIe data link, wherein each data lane includes a corresponding upstream retimer training and status state machine and a downstream retimer training and status state machine; a die-to-die retimer training and status state machine synchronization channel connected between the circuit dies and configured to carry a multi-bit channel status signal, wherein the multi-bit channel status signal includes aggregated status information of the retimer training and status state machine of each circuit die, wherein the aggregated status information of each circuit die collectively constitutes complete status information of the multi-die retimer training and status state machine; a state synchronization pipeline within each circuit die, configured to: receive the multi-bit channel state signal; and accumulate the complete state information by incrementally saving the aggregated state information from the remaining circuit dies, wherein the synchronization pipeline is further configured to: output state information of the retimer training and state state machine belonging to a given circuit die to the die-to-die retimer training and state state machine synchronization channel; and The corresponding upstream retimer training and state state machine and downstream retimer training and state state machine located in each circuit die are used to: analyze the summary state information of the multiple circuit dies after synchronously accumulating the complete state information; and synchronously perform state switching within the circuit die.

2. The device according to claim 1, wherein The die-to-die retimer training and state machine synchronization channel is a multi-wire parallel bus.

3. The device according to claim 1, wherein The summary status information for each circuit die includes a logical AND condition and a logical OR condition for the upstream retimer training and status state machine and the downstream retimer training and status state machine for the circuit die.

4. The device according to claim 1, wherein A first circuit die of the plurality of circuit dies includes the upstream retimer training and status state machine, wherein a second circuit die of the plurality of circuit dies includes the downstream retimer training and status state machine.

5. The device according to claim 4, characterized in that The aggregated status information for each circuit die corresponds to channel-specific status information.

6. The device according to claim 1, wherein Each circuit die of the plurality of circuit dies includes the upstream retimer training and status state machine and the downstream retimer training and status state machine.

7. The device according to claim 6, characterized in that The aggregated state information for each circuit die corresponds to pseudo-port specific state information.

8. The device according to claim 1, wherein The multi-bit channel status signal includes a plurality of time slots, wherein each time slot is occupied by aggregated status information from a corresponding circuit die.

9. The device according to claim 8, wherein Each circuit die includes a time slot counter for maintaining a time slot count value, wherein the time slot count value of each ring counter is initialized based on a synchronization bit circulating in a synchronization time slot and a number of chips maintaining a synchronization count value between each counter, wherein accumulation of complete status information for multiple dies occurs at a preset time slot count value of each counter.

10. The device according to claim 9, wherein The multi-bit channel status signal includes control bits within the time slots occupied by the synchronization bits.

11. A method, characterized in that include: sending and receiving serial information over a multi-lane data link distributed across a plurality of circuit dies of a retimer disposed between a root complex and an endpoint device, wherein each circuit die includes a corresponding set of upstream retimer training and status state machines and a set of downstream retimer training and status state machines; generating local summary state information of the retimer training and state machine of each circuit die, wherein the local summary state information includes upstream state information and downstream state information; outputting the local summary status information of each circuit die as a multi-bit channel status signal to a die-to-die retimer training and state machine synchronization channel connected between each circuit die; Incrementally saving aggregated state information of the retimer training and state machines of the remaining circuit dies within the corresponding circuit die to accumulate complete state information of the multi-die retimer training and state machines; In response to accumulating multi-die complete status information, separately analyzing the upstream status information and the downstream status information of the plurality of circuit dies; performing synchronized state switching between all retimer training and state state machines within each set of upstream retimer training and state state machines based on analysis of the upstream state information; and Based on the analysis of the downstream state information, synchronized state switching is performed between all retimer training and state state machines within each group of downstream retimer training and state state machines.

12. The method according to claim 11, wherein The die-to-die retimer training and state machine synchronization channel is a multi-wire parallel bus.

13. The method according to claim 11, wherein The summary status information for each circuit die includes a logical AND condition and a logical OR condition of the upstream retimer training and status state machine and the downstream retimer training and status state machine for the circuit die.

14. The method according to claim 11, wherein The local summary status information output to the die-to-die retimer training and status state machine synchronization channel includes status information of a pseudo inter-port retimer training and status state machine.

15. The method according to claim 11, wherein The local summary status information output to the die-to-die retimer training and status state machine synchronization channel includes status information of the pseudo intra-port retimer training and status state machine.

16. The method according to claim 11, wherein The multi-bit channel status signal includes a plurality of time slots, wherein each time slot is occupied by the aggregated status information from a corresponding circuit die.

17. The method according to claim 16, wherein The plurality of time slots includes a synchronization time slot.

18. The method according to claim 17, wherein Also includes: The time slot count value of each ring counter is initialized according to the synchronization bit circulating in the synchronization time slot and the number of chips that maintain the synchronization count value between the time slot counters of each circuit bare chip.

19. The method according to claim 18, wherein The accumulation of the multi-die complete status information occurs at a predetermined time slot count value of each counter.

20. The method of claim 17, wherein: Also includes: In the synchronization time slot, the control bits in the multi-bit channel status signal are output.

Images