Apparatus and method for configuring interconnect links between chiplets

Abstract

The disclosed techniques store certain information of functional modules and channels to optimize die-to-die interconnect links. Based on the information, the apparatus may optimize a link width of the interconnect link and a multi-module link configuration. An integrated circuit device includes a first die, a second die, and a die-to-die (D2D) interconnect link connected between the first die and the second die. The D2D interconnect link includes a plurality of channels grouped into a plurality of modules. The apparatus maintains a training result of the D2D interconnect link based on the training of the D2D interconnect link, the training result comprising one or more link configurations of the plurality of modules. The device selects a link configuration of the one or more link configurations to configure a D2D interconnect link comprising one or more modules of the plurality of modules.

Description

Cross Reference to Related Applications

[0001] This patent application claims priority to pending U.S. Non-Provisional Application No. 18 / 463,852, filed September 8, 2023, which is assigned to the assignee of the present application and is hereby expressly incorporated by reference as if fully set forth below and for all applicable purposes. TECHNICAL FIELD

[0002] The present disclosure relates generally to integrated circuit technology, and more particularly to techniques for configuring interconnect links between small chips or dies. BACKGROUND

[0003] A computer system can include a single or multiple integrated circuits, where multiple cores or processors exist on one or more separate integrated circuit devices or dies. An integrated circuit device can include one or more dies, which can include various components such as processing cores, graphics processors, interfaces, memory, etc. Multiple integrated circuit dies can be included in the same package, and on-die interconnects can be used to connect the semiconductor dies together. In some examples, an integrated circuit die can be referred to as a small chip. A small chip is a functional unit that performs certain specific tasks or provides certain functionality within an integrated circuit device that includes multiple small chips or dies.

[0004] An example of a die-to-die (D2D) interconnect link is a Universal Chiplet Interconnect Express (UCIe) link. UCIe is an open, multi-protocol supported on-die interconnect standard for connecting multiple dies (e.g., small chips) on the same package. A UCIe link can support multiple protocols (Peripheral Component Interconnect Express (PCIe), Compute Express Link (CXL), etc.) on top of a common physical and link layer. A UCIe link can provide interoperability across a variety of devices with different performance characteristics and requirements. SUMMARY

[0005] The following presents a simplified summary of one or more implementations in order to provide a basic understanding of such implementations. This summary is not an extensive overview of all contemplated implementations, and is intended to neither identify key or critical elements of all implementations nor delineate the scope of any or all implementations. Its sole purpose is to present some concepts of one or more implementations in a simplified form as a prelude to the more detailed description that is presented later.

[0006] Certain aspects of the present disclosure relate to an integrated circuit (IC) device including multiple small chips or dies connected using a die-to-die (D2D) interconnect link.

[0007] In various aspects of the disclosure, a method of configuring a die-to-die (D2D) interconnect link includes training a D2D interconnect link between a plurality of dies, the D2D interconnect link including a plurality of channels grouped into a plurality of modules; maintaining, based on the training of the D2D interconnect link, a training result of the D2D interconnect link, the training result including one or more link configurations of the plurality of modules; and selecting a link configuration of the one or more link configurations to configure the D2D interconnect link including one or more modules of the plurality of modules.

[0008] In various aspects of the disclosure, an integrated circuit device includes a first die; a second die; and a die-to-die (D2D) interconnect link connected between the first die and the second die, the first die including a communication interface connected to the D2D interconnect link and one or more processors connected to the communication interface. The one or more processors are configured to train the D2D interconnect link between the first die and the second die, the D2D interconnect link including a plurality of channels grouped into a plurality of modules; maintain, based on the training of the D2D interconnect link, a training result of the D2D interconnect link, the training result including one or more link configurations of the plurality of modules; and select a link configuration of the one or more link configurations to configure the D2D interconnect link including one or more modules of the plurality of modules.

[0009] In various aspects of the disclosure, an integrated circuit device includes a die-to-die (D2D) interconnect link connected between a first die and a second die; and means for training a D2D interconnect link between a plurality of dies, the D2D interconnect link including a plurality of channels grouped into a plurality of modules; means for maintaining, based on the training of the D2D interconnect link, a training result of the D2D interconnect link, the training result including one or more link configurations of the plurality of modules; and means for selecting a link configuration of the one or more link configurations to configure the D2D interconnect link including one or more modules of the plurality of modules. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 An apparatus that can be implemented as an integrated circuit device including a plurality of components connected by an interconnect is illustrated in accordance with some aspects of the disclosure.

[0011] Figure 2 Certain aspects of an apparatus that can be constructed using a chiplet are illustrated in accordance with some aspects of the disclosure.

[0012] Figure 3 An exemplary apparatus in which a plurality of chiplets are stacked vertically on a substrate is illustrated in accordance with some aspects of the disclosure.

[0013] Figure 4 An exemplary Universal Chiplet Interconnect Express (UCIe) protocol layer is illustrated in accordance with some aspects of the present disclosure.

[0014] Figure 5 A UCIe link physical layer is illustrated in more detail in accordance with some aspects of the present disclosure.

[0015] Figure 6 Exemplary states of a link training and status state machine (LTSSM) are illustrated in accordance with some aspects of the present disclosure.

[0016] Figure 7 An exemplary table containing information for configuring a contiguous multi-module link is illustrated in accordance with some aspects of the present disclosure.

[0017] Figure 8 is a flowchart illustrating a method of configuring a multi-module link in accordance with some aspects of the present disclosure.

[0018] Figure 9 An example of a logical module mapping of a multi-module link is illustrated in accordance with some aspects of the present disclosure.

[0019] Figure 10 An exemplary UCIe link is illustrated in accordance with some aspects of the present disclosure.

[0020] Figure 11 An exemplary table for configuring a link with contiguous lanes is illustrated in accordance with some aspects of the present disclosure.

[0021] Figure 12 is a flowchart illustrating a method for configuring a D2D interconnect link based on contiguous link information in accordance with some aspects of the present disclosure. DETAILED DESCRIPTION

[0022] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein can be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring such concepts.

[0023] Several aspects of the application will now be presented with reference to various apparatus and methods. These apparatus and methods will be described in the following detailed description and illustrated in the accompanying drawings by various blocks, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, firmware, or any combination thereof. Whether such elements are implemented as hardware or software depends on the particular application and design constraints imposed on the overall system.

[0024] Aspects of the disclosure provide various systems, apparatuses, and techniques for optimizing die-to-die interconnect links. In some aspects, techniques can store certain information of work modules and lanes to optimize interconnect links. Based on the information, an apparatus can optimize link width and multi-module link configuration of the interconnect links.

[0025] Figure 1 An example of an apparatus 100 that can be implemented as an integrated circuit device including multiple components or dies (e.g., chiplets) connected by one or more interconnects is illustrated. In some examples, the apparatus can be enclosed within a portable or wearable device, such as a smart watch or a mobile device (e.g., a smartphone, a mobile phone, etc.). In some aspects, the apparatus 100 includes various circuits. In various examples, the circuits can be implemented using one or more dies (e.g., chiplets) arranged in a configuration that can be suitable for mobile computing, embedded computing, edge computing, etc. In one example, the apparatus 100 can be configured to support multiple communication technologies, modes, or protocols. In some aspects, the apparatus 100 can include a system on chip (SoC) 104, one or more peripheral devices 106, and a transceiver 108 that cooperate to enable the apparatus to communicate with a radio access network, a core access network, the Internet, and / or another network through an antenna 122. In some examples, the apparatus 100 can communicate with another device using a device-to-device connection (e.g., Bluetooth, vehicle-to-vehicle (V2V) connection, or vehicle-to-everything (V2X) connection) or a mesh network.

[0026] In some aspects, the SoC 104 can include various circuitry, such as one or more processors 112, one or more modems 110, on-board memory 114 (one or more memories), communication interface circuitry 116 (e.g., bus interfaces), and / or other logic circuitry or functionality. The SoC can be controlled by an operating system that provides an application programming interface (API) layer that enables the one or more processors 112 to execute software modules or instructions that reside in on-board memory 114 and / or other processor-readable storage 118 provided on the SoC. The software modules can include instructions and data stored in the on-board memory 114 and / or the processor-readable storage 118. The SoC 104 can access its on-board memory 114, processor-readable storage 118, and / or storage external to the device 100. The on-board memory 114 and the processor-readable storage 118 can include read-only memory (ROM), random-access memory (RAM), electrically erasable programmable ROM (EEPROM), flash memory, or any memory device useful for processing systems and computing platforms. The device can include, implement, or access a local database or other parameter storage (e.g., tables, databases, etc.) that can maintain operational parameters and other information used to configure and operate the device 100 and / or the SoC 104. The local database can be implemented using registers, database modules, flash memory, magnetic media, EEPROM, floppy or hard disks, etc. The SoC 104 can also be coupled to external devices, such as antennas 122, displays, operator controls, switches or buttons, and other components.

[0027] The device 100 can provide an interconnect link (e.g., interconnect link 120) that enables communication between different components (e.g., SoC 104, peripherals 106 and / or 108 RF transceivers) that can be implemented in one or more dies (e.g., small chips). In one example, the link 120 can be a Universal Chiplet Interconnect Express (UCIe) link. In one example, the SoC 104 can include communication interface circuitry 116 coupled to the interconnect. Each of the interface circuitry 116 can include a combination of circuitry, counters, timers, control logic, and other configurable circuitry or modules. In one example, certain interconnect interface circuitry 116 can be configured to operate, e.g., a UCIe link, according to a communication specification or protocol defined by a standard. The device 100 can include or control power management functionality that configures and manages the operation of the device 100.

[0028] In some aspects, the device 100 can be included in a mobile phone, tablet computer, palmtop computer, portable digital assistant (PDA), portable gaming console, tablet, and other portable electronic devices. Various components and dies (e.g., chiplets) of the device 100 can communicate with each other via one or more in-chip communication interconnects. The device 100 can be packaged in an integrated circuit (IC) package (which can be referred to as a “semiconductor package” or a “chip package”). An IC package typically includes a package substrate and one or more IC chips, dies, chiplets, or other electronic modules mounted to the package substrate to provide electrical connectivity to the IC chips, dies, chiplets. For example, an IC chip in an IC package can be configured as a SoC. The IC chips are electrically coupled to other IC chips and / or to other components in the IC package by electrical coupling to metal lines in the package substrate. The IC chips can also be electrically coupled to other circuitry external to the IC package by electrical connections of external metal interconnects (e.g., solder bumps) of the IC package.

[0029] Process technology for fabricating semiconductor devices, including IC devices, is continually improving. Process technology includes fabrication methods for manufacturing IC devices and defining transistor size, operating voltage, and switching speed. Features that are constituent elements of circuits in IC devices can be referred to as technology nodes and / or process nodes. The terms “technology node,” “process node,” and “process technology” can be used to characterize a particular semiconductor fabrication process and corresponding design rules. Faster and more power-efficient technology nodes are continually being developed by using smaller feature sizes to produce smaller transistors that enable higher-density ICs. Design rules for newer process technology using low-voltage transistors can exclude the use of higher-voltage transistors supported by previous generations of process technology. The unavailability of certain higher-voltage transistors can pose a hurdle to circuit designers of IC devices that include multiple voltage domains.

[0030] In some aspects, chiplet technology can be used to address some of the performance, power, and size design requirements of complex systems used in certain mobile or wearable devices. Figure 2 The block diagram in FIG. 1 illustrates certain aspects of a system-on-a-chip (SoC) 200 that can be constructed using chiplets in accordance with some aspects. In some examples, the SoC 200 can be a mobile phone, a tablet computer, a palmtop computer, a portable digital assistant (PDA), a portable gaming console, a tablet, or other portable electronic device. Figure 1SoC 104. SoC 200 can be configured by selecting a combination of small chips that implement certain subsystems or different functional elements. In the illustrated example, SoC 200 can include multiple small chips, e.g., a first small chip 202, a second small chip 204, and a third small chip 206 connected to each other via an interconnect link 208. Each of the small chips 202, 204, 206 can include a communication interface (e.g., communication interface circuit 116) connected to the interconnect link 208. In other examples, SoC 200 can include fewer or more small chips than shown. The small chips can provide various subsystems or functions of the SoC. In some aspects, the interconnect link 208 can be a UCIe link. Two small chips connected by a UCIe link can be referred to as a UCIe module and a UCIe module partner, respectively. Figure 2

[0031] In some aspects, SoC 200 can include various processing engines such as central processing units (CPUs) with multiple cores, graphics processing units (GPUs), digital signal processors (DSPs), neural processing units (NPUs), wireless transceiver units (also referred to as modems), peripherals, displays, and imaging interfaces, etc. Each of these subsystems and other functional elements can be implemented as a separate small chip, or as a combination of small chips (e.g., small chips 202, 204, and / or 206). The small chips included in SoC 200 can be proprietary or available from various sources. SoCs can be constructed from small chips manufactured at different process nodes, operating at different voltages, and / or operating at different frequencies.

[0032] Figure 3 An example of SoC 300 is illustrated in which three example small chips 304, 306, 308 are stacked on a substrate 310. In other examples, some small chips can be included in a stack deployed across a surface of substrate 310, while other small chips can be individually mounted on a surface of substrate 310. In some aspects, small chips can be mounted on a surface of a substrate using solder balls 302 (e.g., flip-chip bumps) that provide electrical and / or thermal coupling between substrate 310 and mounted small chips 304, 306, and 308. An interconnect structure (e.g., a UCIe link) can be formed that enables small chips 304, 306, 308 (e.g., in a stack of small chips) to communicate with each other, with other small chips mounted on substrate 310, and with input / output structures that connect SoC 300 with other circuitry, displays, imaging sensors, and other peripherals, and devices. In some examples, SoC 300 can be any of the SoCs described above in Figure 1 and Figure 2

[0033] ​​The use of chiplets can reduce the area size of substrate 310 and increase 3D packaging density. Chipslets can provide complex features and high performance within a smaller form factor, operating at lower power specifications. Furthermore, each chiplet can define multiple power domains, operate at different frequencies, and different chipslets can independently manage power / frequency modes. In some instances, two or more chipslets can operate in mutually exclusive power states. Additionally, besides application-defined operating modes, the operating conditions of the SoC can also depend on the type, number, and arrangement of the chips included on the substrate. It is necessary to consider the power usage of all chips in the SoC to ensure compliance with the power budget assigned to the application or device.

[0034] Interconnection or communication links between various chips or dies can use layered protocols. An example of such a protocol is the UCIe protocol. In the following, UCIe links and protocols will be used to illustrate aspects of this disclosure. However, this disclosure is not limited to UCIe links and protocols. In other examples, any suitable interconnects and protocols can be used to implement aspects of this disclosure.

[0035] Figure 4 An exemplary protocol stack 400 for a UCIe link according to some aspects of this disclosure is illustrated. Protocol stack 400 includes a protocol layer 402, a die-to-die (D2D) adapter layer 404, and a physical (PHY) layer 406. In some aspects, the protocol layer may be proprietary. In some examples, protocol layer 402 may support Peripheral Component Interconnect High Speed ​​(PCIe), Compute High Speed ​​Link (CXL), etc. D2D adapter layer 404 coordinates with protocol layer 402 and physical layer 406 to ensure successful data transmission across the UCIe link. D2D adapter layer 404 is responsible for coordinating higher-level link state machines and initial startup, exchanging protocol option-related parameters with remote link partners, and coordinating power management with remote link partners (where supported).

[0036] Figure 5 The UCIe link physical layer 500 according to some aspects of this disclosure is illustrated in more detail. In some aspects, the physical layer 500 may use one or more components including a processor 112 and a communication interface 116 in a SoC 104 ( Figure 1 Implemented at ) . In one example, physical layer 500 could be Figure 4 The physical layer is 460. A UCIe link can be formed by one or more modules, and each module can provide multiple channels (e.g., 16 or 64 channels in a single UCIe module). Figure 5Four exemplary modules 502, 504, 506, and 508 (module 0, module 1, module 2, and module 3) are shown. Each module has corresponding PHY logic, sidebands, and a mainband. Module 0 may be referred to as the first module or LSB module that provides the least significant byte (LSB) of the UCIe link. A given instance of protocol layer 402 or D2D adapter 404 can transmit data through one or more modules. Sidebands can be used for parameter exchange, register access for debugging / compliance, and coordination with remote partners for link training and management. Sidebands include forward clock pins (CLK) and data pins (DATA) in each direction. Each module has its own set of sideband pins (CLK and DATA). In some respects, redundant clock and data pin pairs are provided in each direction for sideband repair. The mainband connection constitutes the main data path of the UCIe link or module. The mainband of each module includes a forward clock (FW-CLK), a data valid pin (Valid), and N data channels (N DATA) for each module.

[0037] During link initialization and training, UCIe devices (e.g., UCIe modules and UCIe module partners) establish a reliable communication link between devices connected via the UCIe link. UCIe devices negotiate and configure various parameters to ensure correct communication and data transmission. In some respects, the Link Training and State Machine (LTSSM) governs the initialization and training of the UCIe link. The LTSSM transitions through various states until it reaches an active state, indicating a successfully initialized and trained UCIe link between the devices. During link initialization and training, UCIe devices can negotiate and agree on link width and speed based on their capabilities. Once the link initialization and training process is complete, UCIe devices can begin exchanging data using the established and configured communication link.

[0038] Figure 6 This is a diagram illustrating exemplary states of the LTSSM 600 according to some aspects of this disclosure. RESET 602 is the state following a master reset or exit from TRAINERROR 604. The TRAINERROR state is used as a transitional state arising from any fatal or non-fatal event (e.g., a link error or training error) that requires the state machine to return to the RESET state. A fatal event is a severe error that may result in the termination of the current transaction or the entire UCIe link. Fatal events cannot be recovered or corrected by the UCIe device. When a fatal event occurs, the affected device or link can be disabled or reset to recover from the erroneous state. Non-fatal events are less severe than fatal events and can generally be recovered or corrected without terminating the entire UCIe link or transaction. For example, non-fatal errors can be resolved by retrying the operation or taking corrective actions.

[0039] A link error refers to a problem or error occurring at the physical layer of a UCIe interconnect. It indicates a failure or interruption in the communication link between two components (e.g., a chiplet or die) connected via a UCIe link. This can be caused by various factors, such as electrical noise, signal integrity issues, timing mismatch, or physical defects in the interconnect itself. A link error indicates that the communication channel is not functioning correctly and may require troubleshooting or repair. On the other hand, training errors are typically related to the process of training or configuring interconnects to achieve optimal performance. Training errors indicate that the training process was unsuccessful or encountered errors. This can be caused by problems with signal integrity, channel conditions, or compatibility between the transmitting and receiving components. In summary, link errors generally refer to physical layer problems in interconnects, indicating a problem with the communication link, while training errors involve the training process and suggest difficulties in configuring interconnects to achieve optimal performance.

[0040] In some respects, the TRAINERROR state can be entered from SBINIT 606 when the sideband interface needs repair, and from MBINIT when the mainband interface needs degradation or repair. If the sideband is active, a sideband handshake is performed for both devices (e.g., a UCIe module and a UCIe module partner) to enter the TRAINERROR state from any state other than SBINIT. From TRAINERROR, the link can again go through the initialization and training states (SBINIT and MBINIT) to repair and / or retrain the link if possible or necessary.

[0041] SBINIT 606 is the sideband initialization state, where the sideband can be detected, repaired (where applicable), initialized, and a non-reset message is sent. MBINIT 608 is the main band initialization state, where main band (MB) initialization occurs after the sideband initialization in SBINIT. MB can be initialized at the lowest possible speed. In MBINIT, die (e.g., chiplet) calibration can be performed, followed by interconnect repair (where applicable). In MBTRAIN 610, the main band (e.g., data, clock, and valid signals) operating speed can be set to the highest negotiated data rate between link partners. In MBTRAIN, die-to-die training of the main band can be performed to center the clock relative to the data. LINKINIT 612 is the state for exchanging adapter and link management messages. ACTIVE 614 is the state for transmitting and receiving transactions between link partners. L1 / L2 are power management states. PHYRETRAIN 616 is the state for initiating a link retraining process during runtime.

[0042] In the current UCIe specification, each module in a multi-module link is initialized and trained independently using its sidebands. For example, when using two or four modules (e.g., any of modules 502, 504, 506, 508), a separate multi-module PHY logic (MMPL) 510 coordinates operation across modules. For instance, the MMPL is responsible for coordinating data transfer across multiple modules. In some respects, modules in a multi-module link operate with the same link width and speed. During initialization or training, if any module fails to train, the MMPL can degrade the multi-module configuration to the next allowed (degraded) configuration, including module 0. Degrading the link reduces the number of modules and / or channels included in the link. Any differences in speed and link width between different modules can then be resolved. In some respects, a UCIe link can include 16 channels (logical channels 0 to 15) in a standard packet interface. In cases where channel repair is not supported, resilience for failed channels can be provided by configuring the link to an x8 width (e.g., channels 0-7 or channels 8-15 for excluding failed channels). For example, if one or more faulty channels are in channels 0-7, channels 8-15 can be used to configure the link to an x8 width. Configuration can be done during link initialization or training.

[0043] In the current UCIe implementation of a multi-module link, a single sideband is used to transmit and receive sideband messages for all modules or messages related to Raw Die-to-Die Interface (RDI) state transitions from upper layers (e.g., protocol layer 402 and D2D adapter layer 404). In this case, the device transmits sideband messages on the sideband interface of its logical LSB module, which is always mapped to module 0. However, the current method for degrading the multi-module link in the event of a module failure is inefficient because it can only form a UCIe link starting from module 0 to map logical sidebands to the sidebands of module 0, regardless of the maximum number of consecutive modules included in the configuration. Furthermore, if a sideband failure is observed on module 0 (i.e., the LSB module), the multi-module link becomes unavailable even when the mainband of module 0 is still functioning, and the entire link needs to be retrained. For example, if the module 0 link training fails, the multi-module link bandwidth becomes zero, and communication between dies is lost because, in the current method, a functional link cannot be formed without module 0. In another example, when training of modules 0, 2, and 3 succeeds but training of module 1 fails, the resulting multi-module link will only include module 0 (e.g., 16 channels in the case of a standard UCIe package), instead of a dual-module link that includes consecutive modules (modules 2 and 3) (e.g., providing 32 mainband channels in the case of a standard package). The UCIe link configuration described above can result in significant bandwidth loss and throughput degradation.

[0044] This disclosure provides various techniques for configuring multi-module UCIe links with optimized bandwidth. In one aspect, a UCIe device can maintain information about the configuration of one or more consecutive multi-module links. Based on this information, the device can configure a multi-module link with the maximum available link width. In some aspects, the device can remap any initiating module of the selected configuration (other than an LSB module or module 0) to logical module 0 (logical first module) of the link, and remap the next consecutive module in the sequence. Logical modules handle higher-level functions of data transmission, including data packetization, flow control, and protocol management. For example, logical modules of a UCIe link can maintain reliable and efficient communication between devices, abstracting the underlying physical layer details (e.g., ...). Figure 5 The PHY uses physical modules 0, 1, 2, and 3 to provide a high-level interface for data exchange. The physical module number is a physical identifier used by the PHY logic to identify each module, while the adapter / upper layer can assign logical module numbers that may be the same as or different from the physical module number of the underlying module. This is similar to the situation where logical and physical channel numbers are assigned to different channels within a module. By default, two logical channel numbers can be the same as the physical channel numbers until a channel reversal is required or a channel failure occurs. In some aspects, the device can use any sideband of the module's sideband to act as a single logical sideband for a multi-module link. These techniques will be described in more detail below with examples.

[0045] Figure 7 Exemplary tables containing information for configuring consecutive multi-module links are illustrated according to some aspects of this disclosure. Each table 700, 702, 704 includes initiation module information and multi-module link width information for the UCIe link. For example, a device may store training result information in a table for a multi-module link that may include up to four modules (e.g., Figure 5 Modules 0, 1, 2, and 3). In some respects, the table can be stored in the device's memory (e.g., ...). Figure 1 In the memory 114 and / or storage device 118). Referring to Table 700, when no failure is observed during training of the module, the table indicates a single multi-link configuration, where the startup module (LSB module) is module 0 and the multi-module link width is x4 (i.e., 4 modules).

[0046] In another example, when only module 0 is observed to fail, but modules 1, 2, and 3 are successfully trained, Table 702 can indicate a first multi-link configuration where the initiating module is module 1 and the multi-module link width is x2. The table can also include a second multi-link configuration where the initiating module is module 2 and the multi-module link width is x2. In some respects, UCIe supports three configurations of multi-module links to reduce complexity, while using x1, x2, and x4 configurations to place data on channels.

[0047] In another example, when a failure of module 1 is observed during training, Table 704 may indicate two multi-link configurations. The first configuration indicates that the starting module is module 0, and the multi-link width is x1. The second configuration indicates that the starting module is module 2, and the multi-link width is x2. In this case, the second configuration uses two consecutive modules to provide the maximum available link width. Figure 7 Some examples are shown, and it should be understood that, depending on the training results, the table may indicate other consecutive multi-module configurations (e.g., x2, x3, ... etc.).

[0048] Figure 8 This is a flowchart illustrating a method for configuring a multi-module UCIe link according to some aspects of this disclosure. At 802, the device can train the UCIe link in MBTRAIN state 610. For example, the link may include four modules (e.g., Figure 5 (Modules 0, 1, 2, and 3). During MB training, UCIe modules and their UCIe module partners can exchange sideband messages to facilitate and coordinate MB training, for example, to determine if a module is operating correctly at a certain data rate. Each module can report its training results to the multi-module PHY logic 510, which coordinates across modules.

[0049] At position 804, the multi-module PHY logic 510 can store training results in a database or table. For example, depending on the training results, the table could be similar to the one described above. Figure 7 Any table described. A table may indicate one or more multi-module link configurations. The table or the information contained in the table may be stored in the device's memory (e.g., memory 114 and / or storage device 118).

[0050] At 806, the multi-module PHY logic can configure modules based on training results received from the modules. In some aspects, the multi-module PHY logic can configure multi-module links with the maximum available link width based on training results (e.g., stored in a table). In some aspects, the device (e.g., the multi-module PHY logic) can remap the selected configured initiating module (LSB module or module 0) to logical module 0, and remap consecutive modules sequentially. In some aspects, the device can use any sideband of the module's sideband as a logical single sideband of the multi-module link.

[0051] Figure 9 Two examples of logical module mapping for multi-module links according to some aspects of this disclosure are illustrated. In a first example 900, training of module 0 fails, but modules 1, 2, and 3 succeed. In this case, the table will have two multi-module configurations, as shown in Table 702. Either configuration can provide a maximum of 2 consecutive multi-module links (x2 consecutive modules). For example, the device can be configured with a multi-module link having module 1 (the startup module) and module 2 (e.g., providing x32 main band channels). In this case, logical module 0 can be mapped to module 1, and logical module 1 can be mapped to module 2. In another example, the device can be configured with a multi-module link having module 2 (the startup module) and module 3.

[0052] In another example 902, modules 0, 2, and 3 train successfully, but module 1 fails. In this case, the table will have two multi-module configurations, as shown in Table 704. Therefore, the device can be configured with a multi-module link (providing x32 main band channels) with module 2 (the startup module) and module 3. In this case, logical module 0 can be mapped to module 2, and logical module 1 can be mapped to module 3.

[0053] In some respects, the techniques described above can be implemented in the LINKSPEED substate of MBTRAIN state 610 (see [link to relevant documentation]). Figure 6 In the MBTRAIN.LINKSPEED substate, if the module has been successfully trained, the UCIe module can exit to the LINKINIT state 612; otherwise, the UCIe module can enter the Repair substate or the Speed ​​Idle substate. In the Speed ​​Idle substate, the module can change its data rate or link speed. In the Repair substate, the module can apply repair and / or link width degradation procedures. If channel repair is possible, the UCIe module can apply repair on its channel and transmit a repair sideband message to its UCIe module partner. In response, the module partner applies the repair and responds with a repair sideband message once the required repair has been applied.

[0054] Figure 10An exemplary UCIe link 1000 according to some aspects of this disclosure is illustrated. Link 1000 has multiple channels 1002 connecting two devices. In one example, the link may have a link width of x16, including lines 0 to 15 connecting a first device 1004 (e.g., device 0) to a second device 1006 (e.g., device 1). In some aspects, devices 1004 and 1006 may be UCIe modules (e.g., a UCIe module and a UCIe module partner). In other examples, link 1000 may have fewer or more than 16 channels. In one example, multiple channels 1002 belong to a single physical module (e.g., ...). Figure 5 (Modules 0, 1, 2, or 3). In some respects, Link 1000 can be as described above regarding... Figure 1 to Figure 9 Any UCIe link described in the UCIe link description.

[0055] In some respects, when one or more channels of link 1000 encounter errors, devices 1004 and 1006 can degrade the link to include fewer than all channels (e.g., sixteen channels), such as eight channels (e.g., channels 0-7 or channels 8-15). Modules can exchange sideband messages to coordinate the process used to degrade the link. In one example, if one or more faulty channels exist in channels 0-7, the link can be degraded to an x8 width using only channels 8-15. Similarly, if one or more faulty channels exist in channels 8-15, the link can be degraded to an x8 width using only channels 0-7. Reconfiguration can be performed during link initialization or retraining, and the transmitters of the disabled channels can be put into high impedance (hi-Z), and the receivers are disabled.

[0056] In some respects, the degraded links can include any consecutive functional channels (e.g., eight channels (x8 width)), and are not limited to any particular group of channels. For example, channel 1002 can be grouped into a first group including channels 0 through 7 and a second group including channels 8 through 15. In other examples, channels can be grouped into more than two groups. In one example, a faulty channel can exist in both groups of channels (e.g., Figure 10 (Channels 3 and 12 in the example). In this case, the link can be downgraded to a reduced link width (e.g., x8 link width) including channels 1010 from both groups (e.g., channels 4 to 11). Therefore, the link can remain operational as long as a predetermined number (e.g., x8) of consecutive functional channels are available.

[0057] In some respects, according to some aspects of this disclosure, the device can maintain information about the continuous function channel. Figure 11An exemplary continuous channel table 1100 is illustrated, storing information about continuous function channels. For example, table 1100 stores one or more sets 1102 of continuous function channel information. For instance, the table stores a start channel indicator 1104 and an end channel indicator 1106 for each set of continuous function channels (configurable). Each set includes continuous channels (including endpoint values) between the channel identified by the start channel indicator 1104 and the channel identified by the end channel indicator 1106. In this example, table 1100 stores information such as... Figure 10 The scenario shown corresponds to three consecutive functional channel sets where channels 3 and 12 are faulty. Set 0 includes channels 0 to 2 as indicated by the start and end channel indicators. Set 1 includes channels 4 to 11 as indicated by the start and end channel indicators. Set 2 includes channels 13 to 15 as indicated by the start and end channel indicators. In this example, set 1 provides the maximum link width. In some aspects, table 1100 may be stored in one or more memories (e.g., Figure 1 In memory 114 and / or storage device 118). In some examples, table 1100 may optionally include link width information 1108 for each consecutive channel set.

[0058] Figure 12 This is a flowchart illustrating a method 1200 for configuring a D2D interconnect link based on contiguous link information, according to some aspects of this disclosure. For example, method 1200 can be used by a UCIe device to configure a UCIe link using contiguous link information stored in table 1100 described above.

[0059] At point 1202, the device can train D2D interconnect links between multiple dies. For example, the device could be as described above. Figure 1 to Figure 11 The UCIe device described. A D2D interconnect link can be a UCIe link between multiple chiplets (e.g., chiplets 202, 204, and / or 206). The UCIe link can include multiple modules (e.g., module 0, module 1, module 2, and module 3) that can be used to establish the UCIe link. In one example, processor 112 can provide [facilities for] [application / processing]. Figure 6 The Link Training and State Machine (LTSSM) 600 is a component for training D2D interconnect links.

[0060] At point 1204, the device can maintain the training results of the D2D interconnect links based on the training of the D2D interconnect links. The training results include one or more link configurations for multiple modules. In one example, the device can maintain the training results in accordance with the above description... Figure 7Tables 700, 702, and 704, similar to those described, describe this type of table. The table may include startup module information for the UCIe link and multi-module link width information. Each configuration in the table provides a link configuration with a certain link width including one or more modules. In one example, processor 112 may provide, for example, information in memory (e.g., ... Figure 1 The component that maintains the training results in the memory (114 and / or 118).

[0061] At 1206, the device can select one or more link configurations to configure a D2D interconnect link comprising one or more modules from a plurality of modules. In one example, processor 112 may provide components for selecting a link configuration from memory 114 and / or storage device 118.

[0062] In one configuration, device 100 includes components for performing the above-described... Figure 12 Various units of the processes and technologies described herein. The aforementioned components may be... Figure 1 The processor 112 of the present invention, shown herein, is configured to perform the functions described by the foregoing components. Alternatively, the foregoing components may be circuitry or any device configured to perform the functions described by the foregoing components.

[0063] Of course, in the example above, processor 112 is provided merely as an example, and other components for performing the described functions may include, but are not limited to, those stored in a computer-readable storage medium (e.g., Figure 1 Instructions in the storage device 118), or in Figure 1 to Figure 11 The description in any of them and the use of, for example, this article about Figure 1 to Figure 11 Any other suitable device or component of the described process and / or algorithm.

[0064] Using the techniques described above, D2D interconnect links (e.g., UCIe links) can be configured to provide the highest available bandwidth achievable from the available hardware resources within each module and the link. The techniques can significantly increase throughput by increasing the number of active channels in the event of module training failure. The techniques disclosed herein avoid the need for link retraining in the event of a sideband failure in module 0 (LSB module), and thus prevent the link bandwidth from becoming zero. Furthermore, the disclosed techniques significantly reduce the power consumed during link retraining. Even when the failed channels are in different channel groups, the techniques can enhance link robustness by providing optimal available bandwidth in situations where a reduction in link width is required due to one or more failed channels. Therefore, bandwidth loss of functional channels due to any channel corruption or failure can be significantly reduced.

[0065] Some specific implementation examples are described in the following numbered clauses: A method for configuring a die-to-die (D2D) interconnect link, the method comprising: training the D2D interconnect link between a plurality of dies, the D2D interconnect link including a plurality of channels grouped into a plurality of modules; maintaining the training result of the D2D interconnect link based on the training, the training result including one or more link configurations of the plurality of modules; and selecting a link configuration from the one or more link configurations to configure the D2D interconnect link including one or more modules of the plurality of modules.

[0066] 2. The method according to Clause 1, wherein the training results indicate one or more available configurations of the plurality of modules.

[0067] 3. The method according to Clause 2, wherein each of the one or more available configurations includes startup module information and link width information for the D2D interconnect link using one or more of the plurality of modules.

[0068] 4. The method according to Clause 2 or 3, wherein the selected link configuration comprises a plurality of consecutive modules having the largest width among the one or more available configurations.

[0069] 5. The method according to Clause 4, wherein in response to the least significant byte module among the plurality of modules not being included in the selected link configuration, the plurality of consecutive modules exclude the least significant byte module.

[0070] 6. The method according to Clause 5, the method further comprising: mapping a first logical module to a first module among the plurality of consecutive modules, the first logical module being configured to abstract one or more modules among the plurality of modules.

[0071] 7. The method according to Clause 1, 2 or 3, wherein the D2D interconnect link includes a single logical sideband that is mapped to a sideband of any module excluding the least significant byte module among the plurality of modules.

[0072] 8. The method according to Clause 1, 2 or 3, wherein the training result indicates multiple fault channels corresponding to a single module among the plurality of modules, the method further comprising: configuring multiple consecutive function channels.

[0073] 9. The method according to Clause 8, wherein each of the one or more available configurations of the plurality of modules includes starting channel information and link width information of the D2D interconnect link using one or more channels of the plurality of channels corresponding to a single module of the plurality of modules.

[0074] 10. The method according to Clause 9, wherein the plurality of consecutive functional channels correspond to a single module among the plurality of modules having the maximum link width in the one or more available configurations.

[0075] 11. An integrated circuit device comprising: a first die; a second die; and a die-to-die (D2D) interconnect link connected between the first die and the second die, the first die including: a communication interface connected to the D2D interconnect link; and one or more processors connected to the communication interface, the one or more processors being configured to: train the D2D interconnect link between the first die and the second die, the D2D interconnect link including a plurality of channels grouped into a plurality of modules; maintain training results of the D2D interconnect link based on the training of the D2D interconnect link, the training results including one or more link configurations of the plurality of modules; and select a link configuration from the one or more link configurations to configure the D2D interconnect link including one or more of the plurality of modules.

[0076] 12. The integrated circuit device according to Clause 11, wherein the training result indicates one or more available configurations of the plurality of modules.

[0077] 13. The integrated circuit device according to Clause 12, wherein each of the one or more available configurations includes startup module information and link width information for the D2D interconnect link using one or more of the plurality of modules.

[0078] 14. An integrated circuit device according to clause 12 or 13, wherein the selected link configuration comprises a plurality of consecutive modules having the largest width among the one or more available configurations.

[0079] 15. The integrated circuit device according to Clause 14, wherein the plurality of consecutive modules exclude the least significant byte module in response to the least significant byte module being not included in the selected link configuration.

[0080] 16. The integrated circuit device according to Clause 15, wherein the one or more processors are configured to: map a first logic module to a first module among the plurality of consecutive modules, the first logic module being configured to abstract one or more modules among the plurality of modules.

[0081] 17. An integrated circuit device according to clauses 11, 12 or 13, wherein the D2D interconnect link includes a single logical sideband that is mapped to a sideband of any module excluding the least significant byte module among the plurality of modules.

[0082] 18. An integrated circuit device according to clauses 11, 12 or 13, wherein the training result indicates multiple fault channels corresponding to a single module among the plurality of modules, wherein the one or more processors are further configured to: configure multiple consecutive functional channels.

[0083] 19. The integrated circuit device according to Clause 18, wherein each of one or more available configurations of the plurality of modules includes starting channel information and link width information of the D2D interconnect link using one or more channels of the plurality of channels corresponding to a single module of the plurality of modules.

[0084] 20. The integrated circuit device according to Clause 19, wherein the plurality of consecutive functional channels correspond to a single module among the plurality of modules having the maximum link width in the one or more available configurations.

[0085] 21. An integrated circuit device, the integrated circuit device comprising: a die-to-die (D2D) interconnect link connected between a first die and a second die; and means for training the D2D interconnect link between a plurality of dies, the D2D interconnect link including a plurality of channels grouped into a plurality of modules; means for maintaining a training result of the D2D interconnect link based on the training of the D2D interconnect link, the training result including one or more link configurations of the plurality of modules; and means for selecting a link configuration among the one or more link configurations to configure the D2D interconnect link including one or more of the plurality of modules.

[0086] 22. The integrated circuit device according to Clause 21, wherein the training result indicates one or more available configurations of the plurality of modules.

[0087] 23. The integrated circuit device according to Clause 22, wherein each of the one or more available configurations includes startup module information and link width information for the D2D interconnect link using one or more of the plurality of modules.

[0088] 24. An integrated circuit device according to clause 22 or 23, wherein the selected link configuration comprises a plurality of consecutive modules having the largest width among the one or more available configurations.

[0089] 25. The integrated circuit device according to Clause 24, wherein the plurality of consecutive modules exclude the least significant byte module in response to the least significant byte module being not included in the selected link configuration.

[0090] 26. The integrated circuit device according to Clause 25, further comprising: A component for mapping a first logic module to a first module among the plurality of consecutive modules, the first logic module being configured to abstract one or more modules among the plurality of modules.

[0091] 27. An integrated circuit device according to clauses 21, 22 or 23, wherein the D2D interconnect link includes a single logical sideband that is mapped to a sideband of any module excluding the least significant byte module among the plurality of modules.

[0092] 28. An integrated circuit device according to clauses 21, 22 or 23, wherein the training result indicates multiple fault channels corresponding to a single module among the plurality of modules, the integrated circuit device further comprising: a component for configuring multiple consecutive functional channels.

[0093] 29. The integrated circuit device according to Clause 28, wherein each of one or more available configurations of the plurality of modules includes starting channel information and link width information of the D2D interconnect link using one or more channels of the plurality of channels corresponding to a single module of the plurality of modules.

[0094] 30. The integrated circuit device according to Clause 29, wherein the plurality of consecutive functional channels correspond to a single module among the plurality of modules having the maximum link width in the one or more available configurations.

[0095] It should be understood that the specific order or hierarchy of the steps in the disclosed process is an example of the exemplary method. It should be understood that the specific order or hierarchy of the steps in these processes can be rearranged according to design preferences. Furthermore, some steps can be combined or omitted. The appended method claims present elements of multiple steps in a sample order, but are not intended to limit one to the specific order or hierarchy presented.

[0096] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but are to be consistent with the full scope of the language of the claims, wherein an element referred to in the singular is not intended to mean “one and only one,” but rather “one or more,” unless specifically stated otherwise. Unless otherwise specifically stated, the term “some” refers to one or more. All structural and functional equivalents of the elements throughout the various aspects described herein that are known to or will later be known to a person skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly recited in the claims. No claim element should be construed as a component plus a function unless the element is explicitly stated using the phrase “component for…”.

[0097] It should be understood that the specific order or hierarchy of the steps in the disclosed process is an example of the exemplary method. It should be understood that the specific order or hierarchy of the steps in these processes can be rearranged according to design preferences. Furthermore, some steps can be combined or omitted. The appended method claims present elements of multiple steps in a sample order, but are not intended to limit one to the specific order or hierarchy presented.

Claims

1. A method of configuring a die-to-die (D2D) interconnect link, the method comprising: training the D2D interconnect link between a plurality of dies, the D2D interconnect link comprising a plurality of lanes grouped into a plurality of modules; maintaining, based on the training of the D2D interconnect link, training results of the D2D interconnect link, the training results comprising one or more link configurations of the plurality of modules; and selecting a link configuration of the one or more link configurations to configure the D2D interconnect link comprising one or more modules of the plurality of modules.

2. The method of claim 1, wherein the training results indicate one or more available configurations of the plurality of modules.

3. The method of claim 2, wherein each available configuration of the one or more available configurations comprises start module information and link width information of the D2D interconnect link using one or more modules of the plurality of modules.

4. The method of claim 2, wherein the selected link configuration comprises a plurality of consecutive modules having a maximum width of the one or more available configurations.

5. The method of claim 4, wherein the plurality of consecutive modules excludes a least significant byte module of the plurality of modules in response to the least significant byte module not being included in the selected link configuration.

6. The method of claim 5, the method further comprising: mapping a first logical module to a first module of the plurality of consecutive modules, the first logical module configured to abstract one or more modules of the plurality of modules.

7. The method of claim 1, wherein the D2D interconnect link comprises a single logical lane mapped to a lane of any module of the plurality of modules excluding a least significant byte module.

8. The method of claim 1, wherein the training results indicate a plurality of faulty lanes corresponding to a single module of the plurality of modules, the method further comprising: configuring a plurality of consecutive functional lanes.

9. The method of claim 8, wherein each available configuration of the one or more available configurations of the plurality of modules comprises start lane information and link width information of the D2D interconnect link using one or more lanes of the plurality of lanes corresponding to a single module of the plurality of modules.

10. The method of claim 9, wherein the plurality of consecutive functional lanes correspond to a single module of the plurality of modules having a maximum link width of the one or more available configurations.

11. An integrated circuit device, the integrated circuit device comprising: a first die; a second die; and a die-to-die (D2D) interconnect link connected between the first die and the second die, the first die comprising: a communication interface connected to the D2D interconnect link; and one or more processors connected to the communication interface, the one or more processors configured to: ​ training the D2D interconnect link between the first die and the second die, the D2D interconnect link comprising a plurality of lanes grouped into a plurality of modules; maintaining a training result of the D2D interconnect link based on the training of the D2D interconnect link, the training result comprising one or more link configurations of the plurality of modules; and selecting a link configuration of the one or more link configurations to configure the D2D interconnect link comprising one or more modules of the plurality of modules.

12. The integrated circuit device of claim 11, wherein the training result indicates one or more available configurations of the plurality of modules.

13. The integrated circuit device of claim 12, wherein each available configuration of the one or more available configurations comprises start module information and link width information of the D2D interconnect link using one or more modules of the plurality of modules.

14. The integrated circuit device of claim 12, wherein the selected link configuration comprises a plurality of contiguous modules having a maximum width of the one or more available configurations.

15. The integrated circuit device of claim 14, wherein responsive to a least significant byte module of the plurality of modules not being included in the selected link configuration, the plurality of contiguous modules excludes the least significant byte module.

16. The integrated circuit device of claim 15, wherein the one or more processors are configured to: map a first logical module to a first module of the plurality of contiguous modules, the first logical module configured to abstract one or more modules of the plurality of modules.

17. The integrated circuit device of claim 11, wherein the D2D interconnect link comprises a single logical lane mapped to a lane of any module of the plurality of modules excluding a least significant byte module.

18. The integrated circuit device of claim 11, wherein the training result indicates a plurality of faulty lanes corresponding to a single module of the plurality of modules, wherein the one or more processors are further configured to: configure a plurality of contiguous functional lanes.

19. The integrated circuit device of claim 18, wherein each available configuration of the one or more available configurations of the plurality of modules comprises start lane information and link width information of the D2D interconnect link using one or more lanes of the plurality of lanes corresponding to a single module of the plurality of modules.

20. The integrated circuit device of claim 19, wherein the plurality of contiguous functional lanes correspond to a single module of the plurality of modules having a maximum link width of the one or more available configurations.

21. An integrated circuit device, the integrated circuit device comprising: a die-to-die (D2D) interconnect link connected between a first die and a second die; and means for training the D2D interconnect link between a plurality of dies, the D2D interconnect link comprising a plurality of lanes grouped into a plurality of modules; a component to maintain a training result for the D2D interconnect link based on the training of the D2D interconnect link, the training result comprising one or more link configurations of the plurality of modules; and a component to select a link configuration of the one or more link configurations to configure the D2D interconnect link comprising one or more modules of the plurality of modules.

22. The integrated circuit device of claim 21, wherein the training result indicates one or more available configurations of the plurality of modules.

23. The integrated circuit device of claim 22, wherein each available configuration of the one or more available configurations comprises start module information and link width information for the D2D interconnect link using one or more modules of the plurality of modules.

24. The integrated circuit device of claim 22, wherein the selected link configuration comprises a plurality of contiguous modules having a maximum width of the one or more available configurations.

25. The integrated circuit device of claim 24, wherein responsive to a least significant byte module of the plurality of modules not being included in the selected link configuration, the plurality of contiguous modules excludes the least significant byte module.

26. The integrated circuit device of claim 25, further comprising: a component to map a first logical module to a first module of the plurality of contiguous modules, the first logical module configured to abstract one or more modules of the plurality of modules.

27. The integrated circuit device of claim 21, wherein the D2D interconnect link comprises a single logical lane mapped to a lane of any module of the plurality of modules excluding a least significant byte module.

28. The integrated circuit device of claim 21, wherein the training result indicates a plurality of faulty lanes corresponding to a single module of the plurality of modules, the integrated circuit device further comprising: a component to configure a plurality of contiguous functional lanes.

29. The integrated circuit device of claim 28, wherein each available configuration of the one or more available configurations of the plurality of modules comprises start lane information and link width information for the D2D interconnect link using one or more lanes of the plurality of lanes corresponding to a single module of the plurality of modules.

30. The integrated circuit device of claim 29, wherein the plurality of contiguous functional lanes correspond to the single module of the plurality of modules having a maximum link width of the one or more available configurations.

Images