Training reduction for master band chip module interconnect clock lines

Abstract

Aspects relate to training reduction for master band chip module interconnect clock lines. In one example, a method includes transferring an iteration of a first training pattern from a module of a first die to a module partner of a second die on a first master band clock line of a die-to-die connection, the die-to-die connector includes a sideband, a master band including a first master band clock line, and at least one data line supported by at least the first master band clock line. Before iteration of the first training pattern is completed, an automatic result is received from the module partner over the sideband, the automatic result indicating successful reception of the training pattern. Data is communicated with the module partner over the master band using at least the first master band clock line in response to receiving the automatic result.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to pending U.S. Non-Provisional Application No. 18 / 329,462, filed June 5, 2023, which is assigned to the assignee of the present application and hereby expressly incorporated by reference as if fully set forth below and for all applicable purposes. TECHNICAL FIELD

[0003] Aspects of the present disclosure relate generally to data communication between integrated circuit chip modules, and in particular to training of clock lines of a main band connection. BACKGROUND

[0004] As integrated circuit (IC) chips become smaller, it has become possible to package multiple chips into a single package. Each chip can be optimized for cost, materials, manufacturing process, and size that best suit a particular function. In such a package, a central processing unit can be manufactured separately from a graphics processing unit, a specialized processor, volatile memory, non-volatile memory, input / output controller, or other components. Different components can be packaged together to meet different needs without redesigning any of the individual components. By placing these different chips into a single package, the entire system including the package can be made smaller. Similar principles apply to printed circuit board systems and subsystems. Moreover, connections between different chips can be faster at lower cost. In some cases, smaller chips (e.g., memory, specialized processors, or interfaces) are referred to as small chips, however, any chip can be referred to as a small chip.

[0005] Universal Chiplet Interconnect Express (UCIe) Specification version 1.0 (UCIe 1.0) defines physical parameters and protocols for data transfer between a chip and a small chip or between two small chips. The connection can be direct or through a package. The interconnect can be within a single package or across a printed circuit board between two different packages. UCIe 1.0 is intended to support interoperability between small chips from different manufacturers and designers. The UCIe 1.0 interconnect includes a MainBand as the primary data transfer connection and a SideBand as the primary initialization and control connection. A series of state transitions are defined to bring the interconnect from SideBand initialization to MainBand initialization to link initialization to an active state and back to standby and reset states.

[0006] The standard packaged version of UCIe 1.0 uses a MainBand with 16 data lanes, two clock lanes, a track lane, and an active lane. The standard packaged version is targeted for larger bump pitch connectors (e.g., 100-130 microns) and longer distances (e.g., 10-25 mm), which can be found connecting two packages on a circuit board. The premium packaged version of UCIe 1.0 uses a MainBand with 64 data lanes, two clock lanes, a track lane, an active lane, four redundant data lanes, a redundant clock lane, and a redundant active lane. The premium packaged version is targeted for smaller bump pitch connectors (e.g., 25-55 microns) and shorter distances (e.g., less than 2 mm), which can be found within a package. New versions of each and new applications of the new versions can be developed over time. SUMMARY

[0007] The following presents a simplified summary of one or more implementations in order to provide a basic understanding of such implementations. This summary is not an extensive overview of all contemplated implementations, and is intended to neither identify key or critical elements of all implementations nor delineate the scope of any or all implementations. Its sole purpose is to present some concepts of one or more implementations in a simplified form as a prelude to the more detailed description that is presented later.

[0008] In one example, a method includes transmitting an iteration of a first training pattern from a module to a module partner on a first mainband clock line of a die-to-die connection of the module to a module partner of a first die, the die-to-die connection including a sideband, a mainband including the first mainband clock line, and at least one data line supported by the at least first mainband clock line. Receiving an autoboot result from the module partner over the sideband prior to completion of the iteration of the first training pattern, the autoboot result indicating successful receipt of the training pattern, and communicating data with the module partner over the mainband using the at least first mainband clock line in response to receiving the autoboot result.

[0009] In another example, a non-transitory computer readable medium has stored therein instructions for causing a processor of a die-to-die connection to perform operations of the above-described method.

[0010] In another example, an apparatus includes a master-bank transmitter of a module of a first die configured to transmit an iteration of a first training pattern from the module to a module partner of a second die on a first master-bank clock of a die-to-die connection of the module to the module partner, the die-to-die connection including a sideband and a master bank including a first master-bank clock line and at least one data line supported by the first master-bank clock line. A sideband receiver of the module is configured to receive an autoboot result from the module partner over the sideband prior to completion of the iteration of the first training pattern, the autoboot result indicating successful receipt of the training pattern.

[0011] The master-bank transmitter is further configured to communicate data with the module partner over the master bank using at least the first master-bank clock line in response to receiving the autoboot result.

[0012] To the accomplishment of the foregoing and related ends, one or more implementations comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the appended drawings set forth in detail certain illustrative aspects of the one or more implementations. These aspects are indicative, however, of but a few of the various ways in which the principles of various implementations can be employed and the described implementation is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a block diagram of a module and module partner coupled together utilizing a 16 data lane die-to-die connection suitable for use in aspects of the present disclosure.

[0014] Figure 2 is a block diagram of a module and module partner coupled together utilizing a 64 data lane die-to-die connection suitable for use in aspects of the present disclosure.

[0015] Figure 3 is a diagram of a connection operation state machine in accordance with aspects of the present disclosure.

[0016] Figure 4 is a diagram of a master-bank initialization state machine in accordance with aspects of the present disclosure.

[0017] Figure 5 is a signaling diagram of messages between a module and a module partner for sharing initialization of a master bank with an autoboot result in accordance with aspects of the present disclosure.

[0018] Figure 6 is a signaling diagram of messages between a module and a module partner for sharing initialization of a master bank with an autoboot result and training active lines in accordance with aspects of the present disclosure.

[0019] Figure 7 is a block diagram of a die with a module in accordance with aspects of the present disclosure.

[0020] Figure 8 is a flowchart of aspects of operating a die-to-die connector with automatic result sharing in accordance with aspects of the present disclosure. DETAILED DESCRIPTION

[0021] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein can be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring such concepts.

[0022] A die-to-die connector herein can refer to a connector between any two dies, including chips or chiplets. A die can be considered any integrated circuit formed on a wafer and then cut, removed, or otherwise extracted from the wafer. The wafer can be silicon, glass, gallium nitride, or any other suitable material for forming integrated circuits. For a die-to-die connector, each die includes a module. In UCIe 1.0, the module on a die connects data and clock lanes to the die circuitry. The lanes have a transmit line and a receive line for bidirectional connection. The module includes a die-to-die adapter processor, PHY (physical) logic, and a PHY interface. The PHY interface includes a transmitter and receiver for each line. A die can have multiple modules that can be coupled to modules of different dies or to the same die. While examples are presented in the context of UCIe 1.0, this interface specification is not required. The die-to-die connectors described herein can also be used to connect two packages across a printed circuit board.

[0023] A die-to-die connector can provide a bidirectional master lane connection with high data rates by using multiple data lanes and multiple clock lanes. As used herein, each lane consists of a transmit line and a receive line. These lines are reciprocal in that a transmit line from the perspective of one module is a receive line from the perspective of the other module, and vice versa. In other words, a line is coupled to a transmitter of a module at one end and to a receiver of the module at the other end.

[0024] To begin operation, the main band of the die-to-die connection is initialized. A training pattern is transmitted on each line, data line, and clock line, and the corresponding receiver trains on the training pattern and tests the pattern to ensure that the respective line is operational. The clock lines are trained independently of each other to test for shorts and opens that affect the other lines. For the same reason, the transmit lines are tested independently of the receive lines. Longer training patterns provide more time for the receiver to train and allow for more accurate testing. Shorter training patterns allow for faster initialization of the die-to-die connection. As the number of data and clock lines increases, the time to initialize the die-to-die connection also increases.

[0025] As described herein, multiple iterations of the clock training pattern can be used to ensure accurate training and testing. However, a portion of the training can be reduced in length, or truncated, by automatically transmitting the training results before the end of the training pattern and before a request for a training report is received. Automatic result sharing allows the receiving module to automatically share the results of receiving the training pattern upon successful detection of an iteration of the training pattern. This allows clock line training to be completed more quickly when possible. On the other hand, full training can be used when necessary. The described methods can be applied to UCIe MainBand interconnects between IC chip modules with multiple clock lines or to other main band interconnects. In this description, parameters, configuration registers, signaling, and other features can be adapted to accommodate automatic result sharing.

[0026] Figure 1 is a block diagram of a first die 101 and a second die 102 coupled together with a 16 data lane die-to-die connection 100 (e.g., a UCIe standard package interconnect) in accordance with some aspects. Each die can have many other components (not shown) to generate, process, store, or communicate data, or to supply or regulate power, or to perform operational, management, or administrative functions, etc., depending on the nature of the die. The first die 101 has a module 103 and the second die 102 has a module partner 104. The module 103 and the module partner 104 can have the same or different structures and can include additional components (not shown), including transmitters, receivers, interfaces, adapters, logic, buffers, etc.

[0027] The connection has a main band 130 and a side band 136. The main band has a transmit portion 132 from the perspective of the module 103 and a receive portion 134 from the perspective of the module 103. The 16 transmit data lanes each have a data line in the transmit portion 132 and a data line in the receive portion 134, with 16 lines in each direction. The side band 136 also has a transmit portion and a receive portion, each with side band data lines and side band clock lines.

[0028] The connection is symmetric because module 103 has mainband transmitter 110 while module partner 104 has mainband transmitter 122. These two transmitters use the same protocol to perform the same functions and operations in opposite directions. Similarly, module 103 has mainband receiver 112 and module partner 104 has mainband receiver 120. The description refers to the module and its construction and operation, but the description applies equally in the same way to module partner 104. Either side can initiate repair or training, and either side can initiate parameter and configuration changes, etc. Module 103 has sideband transmitter 114 and sideband receiver 116. Module partner 104 also has sideband receiver 124 and transmitter 126.

[0029] UCIe 1.0 has a layered protocol with a physical layer and a die-to-die adapter. The physical layer can consist of all types of current packaging options from different manufacturers and manufacturing processes. Examples include 2D packaging, 2.5D packaging, 3D packaging, and other methods such as silicon bridges, embedded multi-die interconnect bridges (EMIB), Co- Wafer-on-Substrate (CoWoS) and Fan-Out-Substrate-on-Chip (FOCoS) interposer packaging, and any other connection between two dies on the same substrate or between two packages on the same substrate. Optical or electrical connections can be made between the package and other components. UCIe can be extended in future revisions to connect dies or packages across rack-based components.

[0030] The UCIe 1.0 protocol layer runs on top of the physical layer and has many features in common with Peripheral Component Interconnect Express (PCIe), Compute Express Link (CXL), and other pre-existing protocols. The PCIe protocol provides extensive interoperability and flexibility. The CXL protocol provides low latency and high throughput connections. UCIe 1.0 can be extended in future revisions to include other protocols and further modifications away from PCIe and CXL. The structures and methods presented herein are described in the context of UCIe 1.0, but can be adapted to future versions of UCIe under any name and other connection configurations with multiple clock lanes.

[0031] In Figure 1 , the mainband 130 has two clock lanes that operate at the same frequency in different phases. There is one clock P line and one clock N line in the mainband transmit portion 132 and one clock P line and one clock N line in the mainband receive portion 134. In UCIe 1.0, this is the MainBand connection. The two transmit lines of a clock lane are sometimes referred to as TCKP and TCKN, while the two receive lines of a clock lane are sometimes referred to as RCKP and RCKN.

[0032] An active lane can be considered a type of clock lane and will be referred to herein as a clock lane with a transmit active line and a receive active line. In UCIe 1.0, an active signal (called TVLD for transmit and RVLD for receive) is transmitted on the active line to frame the data transmitted on the data lines. The active signal marks the beginning and middle of data packets on the data lines. For each 8-bit data packet with 8 unit intervals (UIs), the active signal is asserted for the first 4 UIs and de-asserted for the second 4 UIs. It is asserted again at the beginning of the next 8-bit packet. The active signal is used to gate the clock distribution for all of the data lines in the lane to enable fast idle exit and entry. The active line also allows data to be passed in raw mode or in various Flit modes.

[0033] A rail lane can be considered another type of clock lane and will be referred to herein as a clock lane. In UCIe 1.0, a rail signal (called TTRK_L for transmit and RTRK_L for receive) can be used to perform runtime recalibration to adjust the receiver clock path according to slowly changing voltage, temperature, and transistor aging conditions. A broken clock lane (whether designated as a clock, rail, or active) can also be remapped to a redundant clock lane or rail lane.

[0034] To train the 2 clock lines and the rail line, the UCIe 1.0 module transmits 128 iterations of a clock repair pattern (called CLKREPAIR) that has 16 clock cycles on each clock line in each direction, followed by 8 low cycles. The lines are tested one at a time and in sequence. To train the active line, the UCIe 1.0 module transmits 128 iterations of a value training (VALTRAIN) pattern that has four 1s on the active line in each direction, followed by four 0s, along with a forward clock.

[0035] The receiving module (whether the module partner in one direction or the module in the other direction) detects each training pattern in each line in turn. In UCIe 1.0, a successful detection is defined as at least 16 consecutive iterations of the corresponding pattern (CLKREPAIR or VALTRAIN) being detected. The receiving module records the result as successful or unsuccessful. After completing the training patterns, the transmitting module transmits a request for a training report over the sideband (referred to as MBINIT REPAIRVAL RESULT REQUEST and MBINIT REPAIRCLK RESULT REQUEST). The receiving module transmits a response including the recorded training pattern results as a sideband response (referred to as MBINIT REPAIRVAL RESULT REQUEST and MBINIT REPAIRCLK RESULT REQUEST). The process is repeated for the other direction, so that the module partner then transmits the iterations of each training pattern as the module detects the pattern.

[0036] As described herein, in some cases, 16 consecutive iterations can be detected before 128 iterations have ended. In some cases, 100 or more iterations can not be required for a successful detection. The number of iterations can then be shortened or truncated, allowing the primary band initialization to be performed more quickly. The receiving module (e.g., the module partner) can automatically transmit the training results before the iteration of the training pattern is complete. When this is before the end of the iteration of the training pattern, then the iteration can be truncated. The iteration can be initiated on the second clock line before the iteration on the first clock line is complete and before the request for a training report has been transmitted. If the training results are shared for all clock lines before the request for a training report, then the request for a training report is not needed.

[0037] This automatic training result can be referred to herein as an automatic result sharing (ARS) feature. The ARS feature allows the receiving module to automatically share the results of receiving the training patterns. The results are automatically shared because the results are shared before a request for the results is received. In contrast, the receiving module (e.g., the module partner) shares the results of the training pattern reception when the pattern is successfully detected (e.g., after 16 consecutive successful iterations). The training results can be transmitted on the sideband that does not interfere with the training of the primary band.

[0038] Figure 2is a block diagram of a first die 201 with a module 203 and a second die 202 with a module partner 204 coupled together with a 64 lane die-to-die link 200 (e.g., a UCIe advanced package interconnect) according to some aspects. The module 203 has a primary bus transmitter 210 connected to a transmit portion 232 of a primary bus 230. The transmit portion 232 includes 64 transmit data lines, a clock P line, a clock N line, a valid line, and a rail line. The advanced package primary bus transmit portion 232 also includes 4 redundant data lines (RD Data), a redundant clock line (labeled RD Clock and referred to as TRDCK), and a redundant valid line (RD Valid). The redundant lines are configured for use in the event of a failure in one or more of the primary data or clock lines. The transmit portion 232 is coupled to a primary bus receiver 220 of the module partner 204. The module 203 has a primary bus receiver 212 connected to a receive portion 234 of the primary bus 230. The receive portion 234 includes 64 receive data lines, a clock P line, a clock N line, a valid line, a rail line, 4 redundant data lines, a redundant clock line (referred to as RRDCK), and a redundant valid line. The receive portion 234 is coupled to a primary bus transmitter 222 of the module partner 204.

[0039] The redundant clock lane is provided as a bidirectional repair mechanism in the event of a failure on one of the clock lanes. A failure on either of the clock lanes P, N will result in the redundant clock lane being configured to replace the failed clock lane. If another one of the clock lanes fails, then the rail lane can also be remapped as a clock lane. If the rail lane fails, then the redundant clock or rail lane can also be remapped as a rail lane. The module also has a sideband transmitter 214 that is coupled to a sideband receiver 224 of the module partner 204 through a sideband 236 of the die-to-die link 200. The sideband 236 can be the same or similar to the sideband example of Figure 1 . The module 203 has a sideband receiver 216 that is coupled to a sideband transmitter 226 of the module partner 204 through the sideband 236 of the die-to-die link 200. The sideband has bidirectional data lanes and bidirectional clock lanes.

[0040] In the context of UCIe 1.0, a specific number of lines is provided as an example, and different numbers of lines can be used to accommodate different die-to-die interconnects. In this advanced package example, 128 iterations of the training pattern are also performed on the redundant clock line and the redundant valid line in each direction, further increasing the time required for primary bus initialization. In other aspects, the processes discussed above with respect to Figure 1 may be applied to this 64 data lane die-to-die link.

[0041] Figure 3is a state machine diagram of a connection operation state machine 300 according to some aspects. The state machine is related to the link layer form of a physical layer connection (e.g., 100, 200 described above). As described above, the same state machine can be used for UCIe advanced mode or standard mode or other configurations with adaptation to accommodate differences in lanes. In UCIe 1.0, as part of MainBand link training, the module and module partner must go through various phases of link initialization. First, the link is in a reset state 302. The link remains in the reset state 302 for a minimum of 4 ms to allow for phase-locked loop (PLL) and other link transmitter and receiver components to stabilize. The link progresses through a sideband initialization 304 (referred to as SBINIT), MainBand initialization 306 (referred to as MBINIT), MainBand training 308 (referred to as MBTRAIN), and link initialization 310 (referred to as LINKINIT) states into an active state 312 (referred to as ACTIVE). In UCIe standard packaging, the MBINIT state initializes the clock P, clock N, lanes, valid, and data lanes. In UCIe advanced packaging, the MBINIT state initializes the clock P, clock N, clock RD, lanes, valid, and data lanes.

[0042] Considerations Figure 3 , the state machine 300 starts in a reset state 302, which can be obtained at startup or upon recovery from a deep sleep or low power state (e.g., L2 state 314), or from a fault. The reset state 302 can also be entered by a command from a higher level layer of the die (e.g., a command to switch to a different main band link width).

[0043] The state machine enters the sideband initialization state 304 from the reset state 302. The sideband is a low speed, high reliability portion of the die-to-die connection. It is configured to be easily initialized even in the presence of other faults or difficult environmental conditions. After the sideband initialization state 304, the state machine can progress to the main band initialization state 306. The main band initialization state includes an automatic result sharing (ARS) handshake substate 320. In some aspects, ARS can be supported and enabled for the transmit and receive clock lines of the main band. These can be initialized in the main band initialization state 306 to prepare for the iterations of training patterns communicated in the main band training state 308. The ARS support and enable parameters can be exchanged in sideband messages during the ARS handshake substate 320, and the parameters can be stored in link configuration registers.

[0044] After completing the primary training state 306 (including the ARS handshake substate 320), the state machine proceeds to the primary training state 308. When training is performed during link startup (i.e., the physical layer transitions out of the RESET state), the hardware is allowed to attempt training multiple times. After the primary training state 308, the state machine 300 proceeds to the link initialization state 310. The link initialization state 310 refers to the connection between the module and the module partner. After the link initialization state 310, the state machine moves to the active state 312 for data communication between the module and the module partner. The state machine continues in the active state 312 until some event requires a transition.

[0045] One type of transition is to the PHY retrain state 316. The PHY retrain state 316 allows the PHY layer of the connection to be retrained in the event of an error or a change in the environment or conditions on the connection. As an example, a failure of a data line or a degradation of a clock line can cause the state machine to transition to the PHY retrain state 316 during the active state 312. A new configuration of the data line or clock line can be trained in the PHY retrain state 316. After the PHY retrain state 316, the state machine returns to the primary training state 308 to train the new configuration of the primary, then returns to the link initialization state 310 and back to the active state 312 that has recovered from the PHY retrain state 316.

[0046] The primary training state 308 can be entered at various other triggers. As an example, a software application can write a start UCIe link training bit in the UCIe link control. This can cause the state machine to transition to the PHY retrain state 316. A die-to-die adapter can trigger primary training based on the state of the physical adapter at the die or at the module partner. A signal failure on the sideband or a new cold start initialization.

[0047] Another type of transition is to the L1 / L2 state 314. The L1 / L2 state 314 includes two different low power or standby conditions to accommodate inactivity on the die-to-die connection. To reduce power consumption, heat generation, and / or wear on the die or connection components, the state machine 300 can transition to the L1 standby, which disables many components of the connection, particularly across the primary portion of the connection. The state machine 300 transitions from the L1 standby of the L1 / L2 state 314 to the primary training state 308. The state machine transitions from the primary training state 308 to the link initialization state 310 and back to the active state 312. The L2 standby is a deeper standby with respect to more components, including clocking off to save more power. From the L2 standby, the state machine 300 transitions from the L1 / L2 state 314 back to the reset state 302. From the reset state 302, the full process of state machine transitions is performed to reach the active state 312. There can be more or fewer standby or low power states than the L1 standby and L2 standby to accommodate different implementations.

[0048] One additional state is a training error state 318 reached as a transition from the reset state 302. This state is a dead end and results in an inoperable connection. If the module is restarted, the module reenters the reset state 302 and can be able to initialize or return to the training error state 318.

[0049] Figure 4 is a diagram of a main band initialization state machine 400 according to some aspects that provides additional details regarding the main band initialization state 306 including the ARS handshake substate 320. The main band initialization state machine 400 enters from the main band initialization state with parameters state 402 during which parameters for operation of the die-to-die connection are shared between the module and the module partner. Next, the ARS handshake state 404 includes transmitting an enable request over the sideband to enable ARS for training the main band clock lines of the die-to-die connection and receiving an enable response over the sideband to enable ARS. The ARS handshake can also include transmitting a support request over the sideband to support automatic result sharing and receiving a support response to support ARS. In some examples, the support request can be made only once at initialization or support can be preconfigured. The enable request can be transmitted in response to receiving the support request in the ARS handshake state 404 or during another state. The ARS handshake state 404 also includes setting parameters in the link configuration registers to support ARS.

[0050] Following the ARS handshake state 404, a calibration state 406 is entered to calibrate the link to use the selected clock and data lanes. A repair clock state 408 follows the calibration state 406 and allows for transmitting and receiving training patterns between the module and the module partner to test the clock connection between the module and the module partner. A repair valid state 410 is used to train the valid lane for framing data on the data lines. The valid signal is a type of clock signal.

[0051] A reverse main band state 412 allows for testing the main band to determine whether reverse should be applied to the data lanes of the main band connection. With reverse, lane 15 or lane 63 becomes lane 0 and the designation of the lanes counts up to lane 0 becoming lane 15 or lane 63.

[0052] The repair mainband state 414 is a training state for mainband data communication in which a known pattern is sent on the data lanes to test proper operation of each data lane. Each data lane is tested in both the transmit data line and the receive data line. When a faulty data line is found in either direction through the repair mainband state 414, then the link width can be modified to exclude the faulty data line. After the operation of the mainband initialization state machine 400 is complete, the link training state machine then transitions from the mainband initialization state 306 to the mainband training state 308, as shown. Figure 3

[0053] In UCIe 1.0, with proper, valid framing on the active lanes and forward clocking, the data lanes are tested in the reverse mainband state when the UCIe module transmits 128 iterations of the per-lane ID pattern (LSB first) on all N MainBand data lanes. N is 68 (64 data + 4 RD) for the premium package interface and 16 for the standard package interface. The UCIe module partner performs per-lane comparisons on its receiver on all N lanes. If at least 16 consecutive iterations of the per-lane ID pattern are detected, the detection on the lane is considered successful. The UCIe module partner records the detection results of its receive lines for lane fault detection. After the 128 iterations of the per-lane ID pattern are transmitted, the UCIe module stops transmitting the pattern and transmits a MBINIT.REVERSALMB RESULT request sideband message to get the recorded results. The UCIe module partner stops the comparisons and responds with a MBINIT.REVERSALMB RESULT response sideband message with N bits (68 for the premium package interface and 16 for the standard package interface) of per-lane results.

[0054] Figure 5 is a signaling diagram of messages between a module 502 of a first die and a module partner 504 of a second die for initializing a mainband using automatic result sharing of a mainband clock line according to some aspects. When the die-to-die adapter processor moves from the reset state 302 of Figure 3 The signaling 500 can be used, for example, at startup or in a transition from the L2 state 314. The signaling 500 begins after the sideband initialization is performed. The module and the module partner transmit a message of sideband initialization complete 512. This corresponds to the end of the sideband initialization state 304 of Figure 3 The module 502 then transmits a mainband initialization parameters request 514 to the module partner 504. The module partner 504 replies with a mainband initialization parameters response 516. These are exchanged over the initialized sideband.

[0055] ​The module 502 and module partner 504 can use the response and request to include an automatic result sharing (ARS) handshake. The ARS handshake can be used to determine whether ARS is supported by including an ARS support parameter in the host strap initialization parameter request 514 and in the host strap initialization parameter response 516. An ARS enable parameter can also be included in the host strap initialization parameter request 514 and the host strap initialization parameter response 516. The parameters can include any other suitable ARS parameters, or the support and enable can be conveyed in another manner. The parameters can then be stored in a link configuration register. In one example, the ARS support is part of the initial configuration, not part of the ARS handshake. In one example, the enable is part of the initial configuration, not part of the ARS handshake.

[0056] To support ARS, the module 502 then transmits a host strap clock training enable request 518 to the module partner 504 over the sideband. The module partner 504 replies with a host strap clock training enable response 520. The enable request requests that clock training begin, and the response indicates that the module partner 504 is ready to receive an iteration of the clock training pattern. An ARS enable parameter can be included in the host strap clock training enable request 518, or it can be transmitted earlier in the initialization process or included as part of the initial configuration.

[0057] In response to receiving the host strap clock training enable response 520 from the module partner 504, the module 502 initiates an iteration of the training pattern (e.g., a first training pattern from the module to the module partner on a first host strap clock line 522). This can include disabling the host strap clock line transmitters other than the first host strap clock line, e.g., tri-stating the other host strap transmit clock lines at the module. The module partner simultaneously detects the iteration of the clock training pattern on all of its receive clock lines. The module partner can also detect the iteration of the clock training pattern on the other receive lines. Detection is successful when the iteration of the clock training pattern is detected only on the expected clock line (e.g., the first host strap clock line). When the training pattern is detected on multiple lines, then there is a short and one or more of the clock lines cannot be used. When the training pattern is not detected on any of the host strap clock lines, then there is an open and the clock line cannot be used.

[0058] UCIe 1.0 provides more specific examples of parameter exchange. There is an exchange called MBINIT.PARAM. Parameters for supporting and implementing ARS can be exchanged during this exchange or during a different operation. In UCIe 1.0, the MainBand initialization request message is called MBINIT.PARAM CONFIG REQUEST for both standard and advanced packaging. This is a sideband request to exchange parameters with a UCIe module partner. The request includes the parameters. The response from the module partner is called MBINIT.PARAM CONFIG RESP and is also a sideband message.

[0059] In UCIe 1.0, the clock repair process on each line includes 128 iterations of a clock repair pattern (called CLKREPAIR or VALTRAIN pattern). The clock repair pattern has 16 clock cycles followed by 8 low cycles. The module partner receiver needs to detect at least 16 iterations of the training pattern to determine that the training pattern was successfully received. All of the transmit clock line iterations of the transmit clock line that transmit the clock repair pattern, then the module requests a report from the module partner in the form of a log of all of the received signals in the received signals. For six clock lines, there will be a total of 768 iterations of the training pattern followed by the request and response period. Then, the module partner transmits iterations of the clock training pattern on each of the receive clock lines separately to the module. The same process is repeated for the receive clock lines as for the transmit clock lines.

[0060] When ARS is supported, additional parameters can be added to the same sideband messages for support and enablement for the clock lines. In some examples, the valid lane can have separate and independent parameters. These parameters can include parameters called CLK_AR_Support, CLK_AR_Eanble, VAL_AR_Support, and VAL_AR_Enable, where a “0” or low value indicates that ARS is not supported or disabled, and a “1” or high value indicates that ARS is supported or enabled. These names are provided as examples, and any other suitable names can be used instead.

[0061] For purposes of this description, clock lines refer to the lines labeled clock, track, and valid, including any redundant lines. Training on all clock lines is referred to herein as a clock repair process 510. In some examples, the training pattern is transmitted on only some of the clock lines, as others of the clock lines have already been trained or will not be used. The iterations of the training pattern follow a known pattern so that the transmitter and receiver at each end are able to detect and train the clock signal. Any other suitable clock training process can be used. The clock repair process 510 can be followed by a mainband data line training 532, then a link initialization 540, then mainband data communication 542.

[0062] As described herein, the receiving module (e.g., Figure 5 The module partner 504 transmits the automatic results 524 of the first clock line, the automatic results 528 of the second clock line, and so on. After the module partner has successfully received the training pattern on the corresponding clock line, it transmits the automatic results for each clock line. If it has not successfully received the training pattern, it will not transmit the results automatically. Instead, the module partner 504 records the results and then transmits a response (not shown) to a request for a report from the module 502. In cases where the results are automatically transmitted after 16 to 20 iterations of the training pattern, and then the module 502 stops transmitting iteration 524 on the first clock line and begins transmitting iteration 526 on the second clock line, then more than 100 iterations of the training pattern are not transmitted on the first clock line. The module 502 receives the automatic results before all 128 iterations of the first training pattern and then moves to the next clock line. For six clock lines, more than 600 iterations of the training pattern are not transmitted. In many cases, avoiding the transmission of clock line iterations of the training pattern typically requires more than 80% of the time. If module partner 504 has already automatically transmitted the results for each clock line, module 502 does not need to request a report from module partner 504. This also avoids the time required for the handshake. Although this description uses the example of 128 iterations transmitted and 16 to 20 iterations received to declare success according to UCIe 1.0, any other number can be used. For example, the number may be varied for newer versions of UCIe and for other die-to-die connections.

[0063] At 522, the module initiates an iteration of the first training mode from the module to its module partner on the first main clock line. The module partner receives the iteration on its corresponding receive clock line. The module partner can record detection results on all clock lines. When other transmit clock lines are disabled and may be tri-state, the module partner records results on the corresponding receive clock line and actively checks whether it has also received the iteration of the first training mode on other clock lines. If the iteration of the first training mode is successfully received on the first clock line, the module partner 504 transmits the automatic result 524 of the first clock line. When the module receives the automatic result, it no longer needs to continue transmitting the iteration of the clock training mode on the first clock line. Therefore, the module stops transmitting the iteration of the first clock line and now initiates the iteration of the clock training mode on the second clock line 528.

[0064] The module partner 504 again detects an iteration of the clock training pattern on one or more of its receive clock lines at its reception, and if the iteration of the clock training pattern is successfully received, it communicates an automatic result 528 of the second clock line. The module 502 can then move to a third clock line and initiate an iteration of the clock training pattern on the third clock line. This continues through the redundant clock lines, track lines, and valid lines in both directions from module to module partner and from module partner to module until the clock line training is complete 530.

[0065] The clock repair process 510 is complete, and the signaling 500 continues to main band data line training 532. The link initialization 540 corresponds to the link initialization state 310 of Figure 3 In the case of main band initialization, the signaling continues to communicate data with the module partner using the at least first main band clock line with main band data 542. This corresponds to the active state 312 of Figure 3

[0066] Table 1 is an example of a portion of a link configuration register (referred to herein as config reg) that can be configured to support automatic result sharing (ARS) of a main band (e.g., MainBand of Advanced Package or MainBand of Standard Package). More registers can be added to accommodate more link variations and other connection configurations. In some aspects, by default and at startup, the values in each location are set to zero. During the main band initialization state 306 described above, and specifically during the ARS handshake substate 320, a sideband support request is communicated to the module partner to support ARS of the main band clock lines. This can be part of the main band initialization parameter request 514, and can include support parameters. Upon receiving a sideband support response (shown as main band initialization parameter response 516, which can include support parameters) from the module partner, the support bits can be added to the configuration register.

[0067] In Table 1, there are support bits CLK AR Support for the clock lines and VAL AR Support for the valid lines. In UCIe 1.0, the clock lines and track lines have one form of clock training pattern, and the valid lines have a second, different form of valid training pattern. However, this is not required, and there is no requirement to establish support or enablement independently. In one example, when the support bits are both “1” or high, both dies support ARS of all clock lines (including valid lines). In one example, when these are both “0” or low, then both dies do not support ARS of the clock lines or valid lines.

[0068] ​Upon receiving a sideband enable response (shown as primary clock training enable response 520 and primary valid training enable response 620) from a module partner, the remaining link configuration parameters of Table 1 can be set for the clock line and the valid line, respectively. Specifically, CLK AR Enable indicates whether ARS is enabled for the primary clock line. VAL AR Enable indicates whether ARS is enabled for the primary valid line. As mentioned above, a single enable bit can be used for all clock lines, including the valid line.

[0069] CLK_config_reg CLK_AR_Support CLK_AR_Enable VAL_config_reg VAL AR_Support VAL_AR_Enable

[0070] Table 1

[0071] When the CLK AR Enable bit is set to "1" or high, then the module partner is configured to transmit an automatic report before the module completes an iteration of the clock line training mode on each primary clock line. Similarly, when the VAL AR Enable bit is set to "1" or high, then the module partner is configured to transmit an automatic report before the module completes an iteration of the valid line training mode on each primary valid line.

[0072] Table 2 is an example of an ARS operation selection table configured to use the configuration register bits of Table 1 to determine the operation of ARS. When CLK AR Support is low as in row 1, then there is no ARS for the clock line. Similarly, when VAL AR Support is low as in row 4, there is no support for ARS for the valid line. In the example of Table 2, ARS for the valid line is independent of the clock line, but there can be a single support bit and a single enable bit, or ARS can be supported for the valid line only if ARS is also supported for the clock line. In other words, there are three states as determined from Table 2: no ARS, ARS for the clock line only, and ARS for the clock line and the valid line. Other configurations are also possible. In one example, if CLK AR Support is high as in rows 2 and 3, then CLK AR Enable determines whether ARS is disabled as in row 2 or enabled as in row 3 and shown in Figure 5 Similarly, if VAL AR Support is high as in rows 5 and 6, then VAL AR Enable determines whether ARS is disabled as in row 5 or enabled as in row 6 and shown in Figure 6 More or fewer bits can be used to represent more or fewer ARS selections for the primary lines.

[0073] CLK_AR_Support CLK_AR_Enable VAL_AR_Support VAL_AR_Enable Notes 0 X Default - no CLK ARS 1 0 CLK ARS supported, disabled 1 1 CLK ARS enabled 0 X Default - no VAL ARS 1 0 VAL ARS supported, disabled 1 1 VAL ARS enabled

[0074] Table 2

[0075] Figure 6 is a signaling diagram of messages between a module 602 of a first die and a module partner 604 of a second die for initializing a primary bus using automatic result sharing of a primary bus active line according to some aspects. When the die-to-die adapter processor moves from a reset state 302 of Figure 3 Signaling 600 can be used, for example, at startup or in a transition from L2 state 314. Signaling 600 begins after execution of sideband initialization. The module and module partner communicate a message of sideband initialization complete 612. This corresponds to the end of sideband initialization state 304 of Figure 3 The module 602 then communicates a primary bus initialization parameters request 614 to the module partner 604. The module partner 604 replies with a primary bus initialization parameters response 616. These are exchanged over the initialized sideband.

[0076] The module 602 and module partner 604 can use this response and request to include an automatic result sharing (ARS) handshake. The ARS handshake can be used to determine whether ARS is supported by including an ARS support parameter in the primary bus initialization parameters request 614 and in the primary bus initialization parameters response 616. An ARS enable parameter can also be included in the primary bus initialization parameters request 614 and primary bus initialization parameters response 616. The parameters can include any other suitable ARS parameters, or the support and enable can be conveyed in another manner. The parameters can then be stored in a link configuration register. In one example, the ARS support is part of the initial configuration, not part of the ARS handshake. In one example, the enable is part of the initial configuration, not part of the ARS handshake.

[0077] To support ARS, the module 602 then communicates a primary bus active training enable request 618 to the module partner 604 over the sideband. The module partner 604 replies with a primary bus active training enable response 620. The enable request requests that active line training begin, and the response indicates that the module partner 604 is ready to receive iterations of the active training pattern. An ARS enable parameter can be included in the primary bus active training enable request 618, or it can be communicated earlier in the initialization process or as part of the initial configuration.

[0078] As shown, clock line training 617 is completed before the primary bus active training enable request 618. This clock line training can be performed as shown in Figure 5 . Figure 5 The clock line training of 617 can include active line training, although another clock line or active line can be trained independently, as shown in Figure 6 .

[0079] In response to receiving the primary valid training enable response 620 from the module partner 604, the module 602 initiates an iteration of the valid training mode (e.g., a valid training mode from module to module partner on the primary valid line 622). The valid line training will occur with the forward clock training. The module partner simultaneously detects the iteration of the valid training mode on all of its receive clock lines. The module partner can also detect the iteration of the valid training mode on other receive lines. Detection is successful when the iteration of the valid training mode is detected only on the intended clock line (e.g., the primary valid line). When the training mode is detected on multiple lines, then there is a short and one or more of the valid lines cannot be used. When the training mode is not detected on any of the primary clock lines, then there is an open and the valid line cannot be used.

[0080] UCIe 1.0 also provides a more specific example of parameter exchange with respect to the valid line. In UCIe 1.0, the transmitter communicates 128 iterations of a valid training (VALTRAIN) mode with four ones on the valid line in each direction followed by four zeros, along with a forward clock. Detection is considered successful if at least 16 consecutive iterations of the VALTRAIN mode are detected. As with CLKREPAIR, the module partner logs 4 or more iterations and then can respond with the log results upon request from the module.

[0081] The valid training mode can include an initialization request and response followed by the valid training mode communicated in sequence in each direction and on each clock line along with training a forward clock. The training of the valid line and the redundant valid line is referred to herein as a valid line repair procedure 610. In a standard package, there is only one valid line and no redundant valid line. The iterations of the valid training mode follow a known pattern so that the transmitter and receiver at each end can detect and train the valid signal. Any other suitable valid training procedure can be used. The valid line repair procedure 610 can be followed by primary data line training 632, then link initialization 640, then primary data communication 642.

[0082] As described herein, the receiving module (e.g., the module partner 604 in Figure 6 The module partner 604 transmits the auto results 624 for the valid line, the auto results 628 for the redundant valid line, etc. After the module partner has successfully received the training mode on the respective valid line, it will transmit the auto results for each of the clock lines. If it does not successfully receive the training mode, it will not automatically transmit the results. Instead, the module partner 604 will log the results and then transmit a response to a request from the module 602 for a report (not shown). The report can include a log of all of the clock lines after the clock line training is complete 617.

[0083] If the iteration of the first training pattern is successfully received on the active line, the module partner 604 communicates an automatic result 624 of the active line. When the module receives the automatic result, it is no longer necessary to continue to communicate iterations of the active training pattern on the active line. Thus, the module stops communicating iterations of the active line and now initiates iterations of the active training pattern on the redundant active line 626.

[0084] The module partner 604 again detects iterations of the active training pattern on one or more of its receive clock lines and, if the iteration of the active training pattern is successfully received, it communicates an automatic result 628 of the redundant active line. The active line repair process 610 is complete and the signaling 600 continues to the main band data line training 632. The link initialization 640 corresponds to the link initialization state 310 in Figure 3 In the case of main band initialization, the signaling continues to communicate data with the module partner using the main band clock line through the main band data communication 642. This corresponds to the active state 312 in Figure 3

[0085] Figure 7 is a block diagram of an example of a hardware implementation for a die 700 (e.g., a central processor, a graphics processor, a specialized processor, a volatile memory, a non-volatile memory, an input / output controller, or any other suitable component having a die-to-die connection 710 with a sideband and a mainband). In this example, the die has a processor 720 for performing the primary operations of the die and a memory 705. The die has a module 714 for supporting the die-to-die connection 710 with a die-to-die adapter processor 704, a computer-readable medium 706, a PHY logic 712, a PHY receive block 716, and a PHY transmit block 718. The die-to-die adapter processor 704 performs the operations described above to service the die-to-die connection 710 between the die 700 and one or more other dies (not shown).

[0086] According to various aspects of the disclosure, the module 714 can implement elements or any portion of elements or any combination of elements. Examples of the module 714 include a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a state machine, gated logic, discrete hardware circuits, and other suitable hardware configured to control the die-to-die connection 710 and the clock mode as described throughout this disclosure. In various examples, the die 700 can be configured to perform any one or more of the functions described herein. The die includes other components (not shown) configured to perform other functions of the die as appropriate for the type of die.

[0087] ​In this example, the module has a module with a processor 720 for performing primary operations of the module, a memory 705, and a computer-readable medium 706. The die-to-die adapter processor 704 performs the operations described above to service the interconnection between the module 714 and a module partner or another module. In accordance with various aspects of the present disclosure, elements or any portion of elements or any combination of elements can be implemented with the module 714. Examples of processors include central processing units, graphics processing units, specialized processors, memory controllers, and input / output controllers. Examples of die-to-die adapter processors 704 include microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to control die-to-die connections and clock modes as described throughout this disclosure. In various examples, the module 714 can be configured to perform any one or more of the functions described herein. The module contains other components (not shown) configured to perform other functions of the module, as is appropriate for the type of die.

[0088] In this example, the die 700 can be implemented with a bus architecture, generally represented by the bus 702. The bus 702 can include any number of interconnecting buses and bridges, depending on the specific application of the module 714, the die 700, and overall design constraints. The bus 702 communicatively couples various circuitry including the processor 720, the die-to-die adapter processor 704, the memory 705, and the computer-readable medium having instructions stored thereon, which is generally represented by the computer-readable medium 706. The bus 702 can also link various other circuitry, such as timing sources, peripherals, data buffers, modules, power management circuitry, and other processing cores, which are not further described. The bus interface 708 provides an interface between the bus 702 and other optional external interfaces (e.g., control interface 730 and data interface 732, etc.). The processor 720 is a higher layer with respect to the die-to-die adapter processor 704 and is coupled to the die-to-die adapter processor through the bus 702. The processor 720 can communicate operational, management, or administrative control with the die-to-die adapter processor 704, or the die-to-die adapter processor 704 can operate autonomously. In some examples, the die-to-die adapter receives a request from a higher layer (e.g., the processor 720) to reduce a data rate of a primary band.

[0089] Control interface 730 can be used to provide a communication interface or component that communicates with various other devices and equipment (e.g., other equipment housed within the same package or system) over internal buses or external transmission media, such as a command and control interface for power regulation, power-on testing, and other purposes. Data interface 732 can be used to provide data connections to other types of components within the package or system in addition to die-to-die connections 710. Control interface 730 and data interface 732 can be connected to higher layers to receive reset and configuration commands that can cause the die-to-die processor to switch to a single clock mode.

[0090] Module 714 includes PHY receive blocks 716 corresponding to the sideband and mainband receivers described above and PHY transmit blocks 718 corresponding to the sideband and mainband transmitters described above. PHY transmit blocks 716 and PHY receive blocks 718 are coupled to die-to-die connections 710, which correspond to the physical portions of the sideband 136 and mainband 130 lanes described above that couple first die 101 and second die 102 over pins on the respective die connectors. The module also includes PHY logic 712, which can include link logic to control data applied to each line and the state machines described above under control of the die-to-die adapter processor. PHY logic 712 can also include a clock generator coupled to a clock source to generate the sideband and mainband clock signals as described above.

[0091] Die-to-die adapter processor 704 is responsible for managing the PHY logic 712 and interface processing, including executing software stored on computer-readable media 706. The software, when executed by the die-to-die adapter processor 704, causes module 714 to perform the various functions described below for any particular apparatus. Computer-readable media 706 and memory 705 can also be used for storing data used in execution of the software by die-to-die adapter processor 704.

[0092] The die-to-die adapter processor 704 can be part of the processor 720 or one or more other processor cores (not shown) of the die 700 and perform operations by way of a processor core executing software stored in the computer-readable medium 706 or the die-to-die adapter processor 704 can be independent of any other processing resources of the die 700 to perform software stored on the computer-readable medium 706 using its own processing resources. Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. The software can reside on the computer-readable medium 706. The die-to-die adapter processor 704 controls operations performed by the state machine, such as those of Figure 3 and Figure 4 and causes signaling 500, 600 of the signaling diagram to be transmitted and received, and causes the clock repair training mode to be transmitted and received.

[0093] The computer-readable medium 706 can be a non-transitory computer- readable medium. By way of example, non-transitory computer-readable media includes magnetic storage devices, flash memory devices, random access memories (RAMs), read only memories (ROMs), programmable ROMs (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), registers, and any other suitable medium for storing software and / or instructions that can be accessed and read by a controller. The computer-readable medium 706 can reside in the module 714 or another portion of the die 700. The computer-readable medium 706 can be embodied in firmware for the operations of the state machine or parameters of the ASIC. Those skilled in the art will recognize how to best im plement the described functionality presented throughout this disclosure depending on the particular application and general design constraints imposed on the overall system.

[0094] The die 700 can be configured to perform any one or more of the operations described herein. In some aspects of the disclosure, the die-to-die adapter processor 704 as utilized in the die 700 can include circuitry configured for various functions. The die-to-die adapter processor 704 is coupled to the memory 705 by way of the bus 722. The memory 705 includes parameter and configuration registers 715, which can include parameters for different link widths of data lanes as shown in Tables 1 and 2. Any other parameter and configuration values can also be stored, including those for operation of a host lane in PCIe or CXL operation.

[0095] The die-to-die adapter processor 704 can include ARS support circuitry 741 to utilize module partner transfer of ARS for support of the clock lane of the primary band and to set support parameters in the link configuration registers of the parameter and configuration registers 715 and receive responses. The ARS support circuitry 741 can include one or more hardware components that provide the physical structure that performs various processes related to utilizing module partner transfer of ARS and setting support parameters and receiving responses. The ARS support circuitry 741 can include functionality of components to transfer support requests to a module partner and to receive support responses from a module partner over a sideband for support of the clock lane of the primary band and to set support parameters in the link configuration registers of the parameter and configuration registers 715. The ARS support circuitry 741 can be further configured to execute ARS support instructions 751 included on the computer-readable media 706 to implement the ARS support described herein.

[0096] The die-to-die adapter processor 704 can include ARS enable circuitry 742 configured to transfer and receive enable requests over a sideband to a module partner and receive an enable response from the module partner. As discussed herein, the enable request is an automatic result sharing from a receiver (e.g., module partner) to a transmitter (e.g., module). The ARS enable circuitry 742 can include functionality of components to enable ARS in response to a command or table inference from, for example, Table 2, and to transfer and receive requests and responses to enable ARS. The ARS enable circuitry can further set parameters in the link configuration registers for operating the ARS. The ARS enable circuitry 742 can be further configured to execute ARS enable instructions 752 included on the computer-readable media 706 to implement one or more functions described herein.

[0097] The module 714, die-to-die adapter processor 704 can include link initialization circuitry 743 configured to perform operations to specify a transmit link width and to specify a receive link width to initialize the primary band as discussed herein. The link initialization circuitry 743 can include functionality to initialize a link and can also include functionality for primary band clock training and primary band data line training. The link initialization circuitry system 743 can include functionality of components to receive automatic results in response to transferring iterations of a training pattern on a clock line. The link initialization circuitry 743 can be further configured to execute link initialization instructions 753 included on the computer-readable media 706 to implement one or more functions described herein.

[0098] The die-to-die adapter processor 704 can include a data communication circuit 744 configured to communicate data with another module over a primary band connection of the die-to-die connection 710 using initialization of the primary band with a clock training procedure, as discussed herein. The data communication circuit 744 can include functionality of the components for communicating data with another module over the primary band connection. The data communication circuit 744 can further set parameters for operating using the clock lane according to the link configuration register. The data communication circuit 744 can be further configured to execute data communication instructions 754 included on the computer-readable medium 706 to implement one or more functions described herein.

[0099] The circuit architectures described herein can be implemented on one or more ICs, chips, chiplets, modules, interposers, packages, system printed circuit boards (PCBs), etc. The circuit architectures described herein can also be fabricated with various process technologies, such as complementary metal-oxide-semiconductor (CMOS), NMOS, PMOS, bipolar junction transistor (BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), heterojunction bipolar transistor (HBT), high electron mobility transistor (HEMT), silicon-on-insulator (SOI), etc.

[0100] Figure 8 is an example of a flow diagram illustrating a method for automatic result sharing of a clock lane of a primary band using a die-to-die connection, such as, Figure 1 and Figure 2 The method can be performed in the die-to-die adapter processor 704 or other circuitry of Figure 7 and software as described in the context of Figure 7 Optionally, the method 800 begins at block 802 with transmitting a support request to a module partner over a sideband supporting automatic result sharing (ARS) of a primary band. Transmitting the support request can be performed during initialization of the primary band. Alternatively, the die can be configured to support ARS without any handshaking of support parameters.

[0101] In response to the request message, the module partner can optionally transmit a response. The method 800 continues at block 804 with optionally receiving a support response from the module partner over the sideband supporting ARS of the primary band.

[0102] In block 806, optionally performing transmitting, by a sideband, an enable request to a module partner to enable ARS for training a primary band clock line. Then, at block 808, the module partner can optionally perform receiving, by a sideband, an enable response from the module partner to enable ARS for training a primary band clock line. The enable request can include ARS of the valid line and clock and rail lines, or a separate ARS handshake can be performed on the valid line. The enable request can involve a specified clock line or involve all lines in a single request. These operations can also be performed at another time, or the module and module partner can be preconfigured to enable ARS.

[0103] In block 810, performing transmitting, from the module to the module partner, an iteration of a first training pattern on a first primary band clock line. In block 812, performing receiving, from the module partner, an autoboot result prior to completion of the iteration of the first training pattern, the autoboot result indicating successful receipt of the training pattern. In some aspects, the autoboot result is absent for some clock lines, and a log request report is made for all received training patterns.

[0104] After receiving the autoboot result, at 814, optionally performing transmitting, from the module to the module partner, an iteration of a second training pattern on a second primary band clock line prior to completion of the iteration of the first training pattern for any other clock line. In this way, the autoboot result from the module partner reduces the number of iterations of the first training pattern. In block 816, performing communicating data with the module partner over the primary band in response to receiving the autoboot result.

[0105] As used herein, "or" is intended to mean an inclusive or and not an exclusive or; that is, unless specified otherwise, "or" is intended to mean any of the items listed. As an example, "a or b" is intended to mean "at least one of a or b" and, for example, "a or b" is intended to cover a, b, or c. As used herein, a "set" or "group" of items includes any one or more of the items in the set or group, including single items. As an example, a "set of items" includes any one or more of the items in the set, including single items. As used herein, a "set of items" includes any one or more of the items in the set, including single items.

[0106] The various illustrative components, logic, logical blocks, modules, circuits, operations and algorithm processes described in connection with the implementations disclosed herein can be implemented as electronic hardware, firmware, software, or combinations of hardware, firmware or software, including the structures disclosed in this specification and the structural equivalents thereof. The interchangeability of hardware, firmware and software has been described generally, in terms of their functionality, and illustrative components have been described above in various illustrative components, blocks, modules, circuits and processes. Whether such functionality is implemented in hardware, firmware or software depends on the specific application and design constraints imposed on the overall system.

[0107] The various illustrative logical blocks, modules, and circuits described in connection with the exemplary aspects disclosed herein can be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the alternative, the processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0108] In one or more exemplary aspects, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Combinations of the above should also be included within the scope of computer-readable media.

[0109] An overview of embodiments of the disclosure is provided below.

[0110] Example 1 : A method comprising: transmitting, from a module of a first die to a module partner of a second die, iterations of a first training pattern on a first primary strap clock line of a die-to-die connection of the module to the module partner, the die-to-die connection comprising a sideband, a primary strap comprising the first primary strap clock line, and at least one data line supported by at least the first primary strap clock line; receiving, from the module partner, an autobaud result over the sideband prior to completion of the iterations of the first training pattern, the autobaud result indicating successful receipt of the training pattern; and in response to receiving the autobaud result, communicating data with the module partner over the primary strap using at least the first primary strap clock line.

[0111] Example 2: The method of example 1, further comprising: in response to receiving the autobaud result, ceasing to transmit iterations of the first training pattern.

[0112] Example 3: The method of example 1 or 2, further comprising, prior to completion of the iterations of the first training pattern and in response to receiving the automatic result, transmitting iterations of a second training pattern from the module to the module partner on a second primary clock line.

[0113] Example 4: The method of any one or more of the examples above, wherein the first training pattern is a clock repair pattern.

[0114] Example 5: The method of any one or more of the examples above, wherein the first training pattern is an active line training pattern.

[0115] Example 6: The method of any one or more of the examples above, wherein transmitting iterations of a first training pattern comprises disabling primary clock line transmitters other than the first primary clock line.

[0116] Example 7: The method of any one or more of the examples above, wherein transmitting iterations of the first training pattern comprises transmitting 128 iterations of the first training pattern, and wherein receiving the automatic result comprises receiving the automatic result prior to all of the 128 iterations of the first training pattern and after 16 iterations of the first training pattern.

[0117] Example 8: The method of any one or more of the examples above, further comprising transmitting, from the module, an enable request to enable automatic result sharing for training primary clock lines over the sideband; and receiving, from the module partner, an enable response to enable automatic result sharing over the sideband.

[0118] Example 9: The method of any one or more of the examples above, wherein transmitting iterations of the first training pattern is in response to receiving the enable response.

[0119] Example 10: The method of example 8, wherein transmitting the enable request comprises transmitting the enable request during primary initialization.

[0120] Example 11 : The method of example 8 or 9, further comprising storing an enable bit in a link configuration register, the enable bit indicating that automatic result sharing is enabled for training primary clock lines.

[0121] Example 12: The method of example 8, 9, or 10, further comprising transmitting, to the module partner over the sideband, a support request to support the automatic result sharing, and receiving, from the module partner, a support response to support the automatic result sharing, wherein transmitting the enable request is in response to receiving the support response.

[0122] Example 13: The method of example 10, wherein transmitting the support request comprises transmitting an automatic result sharing support parameter, the method further comprising storing the automatic result sharing support parameter in a configuration register in response to receiving the support response.

[0123] Example 14: The method of example 13, wherein the automatic result sharing parameter identifies a clock line of the primary band over which the iterations of the first training pattern are transmitted.

[0124] Example 15: The method of example 13 or 14, wherein transmitting the enable request comprises transmitting the enable request in response to an automatic result sharing support parameter being present in a configuration register.

[0125] Example 16: A non-transitory computer readable medium having stored therein instructions for causing a processor of a die to perform operations comprising: transmitting iterations of a first training pattern from a module of a first die to a module partner of a second die on a first primary band clock line of a die-to-die connection of the module to the module partner, the die-to-die connection comprising a sideband, a primary band including the first primary band clock line, and at least one data line supported by at least the first primary band clock line; receiving an automatic result from the module partner over the sideband prior to completion of the iterations of the first training pattern, the automatic result indicating successful receipt of the training pattern; and in response to receiving the automatic result, communicating data with the module partner over the primary band using at least the first primary band clock line.

[0126] Example 17: The computer readable medium of example 16, the operations further comprising: in response to receiving the automatic result, ceasing transmission of iterations of the first training pattern.

[0127] Example 18: The computer readable medium of example 16 or 17, the operations further comprising: prior to completion of the iterations of the first training pattern and in response to receiving the automatic result, transmitting iterations of a second training pattern from the module to the module partner on a second primary band clock line.

[0128] Example 19: The computer readable medium of any one or more of examples 16 to 18, the operations further comprising: transmitting an enable request from the module over the sideband to enable automatic result sharing for training primary band clock lines; and receiving an enable response from the module partner over the sideband to enable automatic result sharing.

[0129] Example 20: An apparatus comprising: a primary band transmitter of a module of a first die, the primary band transmitter configured to transmit an iteration of a first training pattern from the module to a module partner of a second die on a first primary band clock of a die-to-die connection that connects the module of the first die to the module partner, the die-to-die connection comprising a sideband, a primary band including the first primary band clock line, and at least one data line supported by at least the first primary band clock line; and a sideband receiver of the module, the sideband receiver configured to receive an autoboot result from the module partner over the sideband prior to completion of the iteration of the first training pattern, the autoboot result indicating successful receipt of the training pattern; the primary band transmitter further configured to communicate data with the module partner over the primary band using at least the first primary band clock line in response to receiving the autoboot result.

[0130] Example 21 : The apparatus of example 20, wherein the iteration of the first training pattern comprises 128 iterations of the first training pattern, and wherein the sideband receiver is configured to receive the autoboot result prior to all of the 128 iterations of the first training pattern and after 16 iterations of the first training pattern.

[0131] Example 22: The apparatus of example 20, further comprising: a link configuration register and a processor configured to store an enable bit in the configuration register, the enable bit indicating that autoboot result sharing is enabled for training primary band clock lines.

[0132] Example 23: The apparatus of example 22, wherein the autoboot result sharing parameter identifies clock lines of the primary band through which the iteration of the first training pattern is transmitted.

Claims

1. A method comprising: transmitting an iteration of a first training pattern from a module of a first die to a module partner of a second die on a first primary band clock line of a die-to-die connection of the module to the module partner, the die-to-die connection comprising a sideband, a primary band comprising the first primary band clock line, and at least one data line supported by at least the first primary band clock line; receiving an autoboot result from the module partner over the sideband prior to completion of the iteration of the first training pattern, the autoboot result indicating successful receipt of the training pattern; and communicating data with the module partner over the primary band using at least the first primary band clock line in response to receiving the autoboot result.

2. The method of claim 1, further comprising: stopping transmitting iterations of the first training pattern in response to receiving the autoboot result.

3. The method of claim 1, further comprising: transmitting an iteration of a second training pattern from the module to the module partner on a second primary band clock line prior to completion of the iteration of the first training pattern and in response to receiving the autoboot result.

4. The method of claim 1, wherein the first training pattern is a clock repair pattern.

5. The method of claim 1, wherein the first training pattern is a valid line training pattern.

6. The method of claim 1, wherein transmitting an iteration of the first training pattern comprises: disabling primary band clock line transmitters other than the first primary band clock line.

7. The method of claim 1, wherein transmitting the iteration of the first training pattern comprises: transmitting 128 iterations of the first training pattern, and wherein receiving the autoboot result comprises receiving the autoboot result prior to all of the 128 iterations of the first training pattern and after 16 iterations of the first training pattern.

8. The method of claim 1, further comprising: transmitting an enable request to enable autoboot result sharing for training primary band clock lines from the module over the sideband; and receiving an enable response to enable autoboot result sharing from the module partner over the sideband.

9. The method of claim 8, wherein transmitting iterations of the first training pattern is in response to receiving the enable response.

10. The method of claim 8, wherein transmitting the enablement request comprises: transmitting the enable request during primary band initialization.

11. The method of claim 8, further comprising: storing an enable bit in a link configuration register, the enable bit indicating that autoboot result sharing is enabled for training primary band clock lines.

12. The method of claim 8, further comprising: transmitting a support request to support the autoboot result sharing to the module partner over the sideband; and receiving a support response supporting the autoboot result sharing from the module partner, wherein transmitting the enable request is in response to receiving the support response.

13. The method of claim 10, wherein transmitting the support request comprises: transmitting an autoboot result sharing support parameter, the method further comprising storing the autoboot result sharing support parameter in a configuration register in response to receiving the support response.

14. The method of claim 13, wherein the autoboot result sharing parameter identifies a clock line of the primary band through which the iteration of the first training pattern is transmitted.

15. The method of claim 13, wherein transmitting the enablement request comprises: transmitting the enable request in response to an autoboot result sharing support parameter existing in a configuration register.

16. A non-transitory computer-readable medium having stored therein instructions for causing a processor of a die to perform operations comprising: transmitting, from a module of a first die to a module partner of a second die on a first master lane clock line of a die-to-die connection that connects the module of the first die to the module partner, iterations of a first training pattern, the die-to-die connection comprising a sideband, a master lane comprising the first master lane clock line, and at least one data line supported by at least the first master lane clock line; receiving, from the module partner over the sideband prior to completion of the iterations of the first training pattern, an autoboot result indicating successful receipt of the training pattern; and communicating data with the module partner over the master lane using at least the first master lane clock line in response to receiving the autoboot result.

17. The computer-readable medium of claim 16, the operations further comprising: stopping transmitting iterations of the first training pattern in response to receiving the autoboot result.

18. The computer-readable medium of claim 16, the operations further comprising: transmitting, from the module to the module partner on a second master lane clock line, iterations of a second training pattern prior to completion of the iterations of the first training pattern and in response to receiving the autoboot result.

19. The computer-readable medium of claim 16, the operations further comprising: transmitting, from the module to the module partner over the sideband, an enable request to enable autoboot result sharing for training master lane clock lines; and receiving, from the module partner over the sideband, an enable response to enable autoboot result sharing.

20. An apparatus comprising: a master lane transmitter of a module of a first die, the master lane transmitter configured to transmit, from the module to a module partner of a second die on a first master lane clock line of a die-to-die connection that connects the module of the first die to the module partner, iterations of a first training pattern, the die-to-die connection comprising a sideband, a master lane comprising the first master lane clock line, and at least one data line supported by at least the first master lane clock line; and a sideband receiver of the module, the sideband receiver configured to receive, from the module partner over the sideband prior to completion of the iterations of the first training pattern, an autoboot result indicating successful receipt of the training pattern, the master lane transmitter further configured to communicate data with the module partner over the master lane using at least the first master lane clock line in response to receiving the autoboot result.

21. The apparatus of claim 20, wherein the iterations of the first training pattern comprise 128 iterations of the first training pattern, and wherein the sideband receiver is configured to receive the autoboot result prior to all of the 128 iterations of the first training pattern and after 16 iterations of the first training pattern. a link configuration register and a processor configured to store an enable bit in the configuration register, the enable bit indicating that autoboot result sharing is enabled for training master lane clock lines.

22. The apparatus of claim 20, further comprising: ​ 23. The apparatus of claim 22, wherein the automatic result sharing parameter identifies a clock line of the host bus through which the first training mode iterates.

Images