Techniques for unified test and debug architecture

Abstract

Techniques for a unified debug and test architecture in a core particle are disclosed. In an illustrative embodiment, a number of cores are integrated on an integrated circuit package. The cores are connected by a package interconnect (e.g., a universal core interconnect express (UCIe) interconnect). Each core grain includes a number of commissioning nodes connected over a network on the core grain. One of the cores (referred to as a package debug endpoint) is used as a link endpoint for a package outer link (e.g., a Peripheral Component Interconnect Express (PCIe) link). In use, a debug message may be sent over a PCIe link to a package debug endpoint. Debug messages may be routed within and between cores, allowing a common protocol to be used to probe debug functionality at each debug node. In this manner, cores from different vendors may be integrated into the same package and tested using general purpose software.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of the filing date under 35 U.S.C. § 119(a) of Indian Patent Application No. 202341023191, filed March 29, 2023, entitled “TEST AND DEBUG INFRASTRUCTURE FOR UNIVERSAL CHIPLET INTERCONNECT EXPRESS (UCIe)-BASED CHIPLETS AND PACKAGES.” BACKGROUND

[0003] In recent years, there has been a significant shift in the field of integrated circuits (ICs) from using traditional monolithic chips to using chiplets. A chiplet is a smaller, independent semiconductor component that performs a specific function (e.g., processing, storage, or communication). This shift is driven by the increasing complexity and diversification of modern ICs, as well as limitations imposed by the manufacturing processes of larger chips. By adopting chiplets, designers can achieve greater flexibility, modularity, and cost efficiency in their IC designs. These chiplets can be manufactured independently, enabling different technology nodes, customization, and improved yields. Additionally, these chiplets can be combined and interconnected in various configurations, forming package-level systems that offer superior performance and scalability.

[0004] To connect chiplets in a package, a common protocol such as UCIe TM (Universal Chiplet Interconnect Express) can enable seamless integration and interoperability of chiplets within an integrated circuit package. Such a protocol can address challenges associated with different chiplet architectures, interfaces, and power requirements, thereby facilitating compatibility and interoperability between chiplets from different manufacturers. BRIEF DESCRIPTION OF DRAWINGS

[0005] The concepts described herein are illustrated by way of example in the accompanying drawings, in which like references indicate similar elements, and in which:

[0006] Figure 1 is a simplified diagram of at least one embodiment of a computing device.

[0007] Figure 2 is a simplified diagram of at least one embodiment of a chiplet of the computing device of Figure 1 is a simplified diagram of at least one embodiment of a chiplet of the computing device of

[0008] Figure 3 is a simplified diagram of an integrated circuit package of a computing device having Figure 2 Figure 1 is a simplified diagram of at least one embodiment of an integrated circuit package of a computing device.

[0009] Figure 4 is a simplified diagram of an integrated circuit package of a computing device having Figure 3

[0010] Figure 5 is a diagram of one embodiment of a register configuration for a corelet of Figure 2

[0011] Figure 6 is a diagram of one embodiment of a debug message that can be sent or received by a corelet of Figure 2

[0012] Figure 7 is a diagram of one embodiment of a flit (flow control unit) that can be sent or received by a corelet of Figure 2

[0013] Figure 8 is a simplified flow diagram of one embodiment of a method that can be performed to test a corelet of Figure 2 DETAILED DESCRIPTION

[0014] In various embodiments disclosed herein, a computing device can include several integrated circuit packages, each having several corelets. The corelets in a package implement a unified test and debug architecture that allows debug messages to be sent between and within various corelets, allowing testing of the functionality of different portions of different corelets. Corelets can be tested at several stages of manufacture, from a sortation stage of individual corelets to field debugging at a system level of an integrated circuit package having several corelets from several vendors.

[0015] The unified test and debug architecture implemented in corelets and integrated circuit packages includes an interface-independent external hardware debug interface. Debug messages or packets can be sent to debug nodes in corelets. In illustrative embodiments, the debug messages are Universal Corelet Interconnect Express (UCIe TM ​​​​​​) packets. These packets can be sent locally on the debug interface or tunneled, allowing a variety of different test and debug interface solutions, e.g., Joint Test Action Group (JTAG), inter-integrated circuit (I2C), improved inter-integrated circuit (I3C), universal serial bus (USB), system management bus (SMBus), general purpose input / output (GPIO), or UCIe TM Pin bumps. This approach would allow for reusable upper layer test / debug software. As described in more detail below, the architecture also specifies how debug messages are sourced and routed through a package containing multiple dies, thus allowing for interoperable debug / test solutions. In some cases, the debug messages themselves can be tunneled via management component transport protocol (MCTP) vendor-defined packets, allowing the debug functionality to be discovered / controlled as a subnetwork and / or meshed as part of the MCTP fabric.

[0016] As used herein, the phrase “communicatively coupled” refers to the ability of a component to send a signal to or receive a signal from another component. The signal can be any type of signal, such as an input signal, an output signal, or a power signal. A component can send or receive signals to or from another component with which it is communicatively coupled via a wired or wireless communication medium, such as a conductive trace, a conductive contact, electromagnetic radiation. Examples of components that are communicatively coupled include integrated circuit dies located in the same package that communicate via embedded bridges in the package substrate, and integrated circuit components attached to a printed circuit board that send signals to or receive signals from other integrated circuit components or electronic devices attached to the printed circuit board.

[0017] In the following description, specific details are set forth to provide an understanding of the embodiments described herein. However, embodiments of the technology described herein can be practiced without some or all of these specific details. Well-known circuits, structures and techniques have not been shown to avoid obscuring the understanding of this description. Phrases such as “one embodiment,” “various embodiments,” “some embodiments,” etc., can include features, structures or characteristics but not every embodiment can include every feature, structure or characteristic.

[0018] Some embodiments can have some, all or none of the features described for other embodiments. “First,” “second,” “third,” etc. descriptions refer to common objects and indicate different instances of the same object being referenced. Such adjectives do not imply that the objects so described must be in a given sequence in time or space, ranked, or in any other manner. “Connected” can indicate direct physical or electrical contact, and “coupled” can indicate elements that cooperate or interact, but the elements can or can not be in direct physical or electrical contact. Optical components (e.g., optical fibers or waveguides) can be “connected” if the gap between them is small enough that light can be transferred from one optical fiber or waveguide to the other without any intervening optical elements (e.g., lenses or mirrors). Also, the terms “comprising,” “including,” “having” and the like, as used with respect to embodiments of the present disclosure, are synonymous. The term “substantially” as used with respect to embodiments of the present disclosure includes arrangements, orientations, distances or positions that are slightly different from the listed meaning. For example, a central axis of a magnetic plug that is in substantial coaxial alignment with a through-hole can have a few degrees of deviation from the central axis of the through-hole. In another example, a substrate assembly feature (e.g., a through-width) that is described as having a listed dimension substantially can vary within a few percent of the listed dimension.

[0019] References in the specification to “one embodiment,” “an embodiment,” “illustrative embodiment,” etc. indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can or can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of those skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. Additionally, it should be understood that an item included in a list of items, with a conjunction such as “and,” or “or,” can mean one, some, or all of the items in the list. Similarly, an item included in a list of items with a conjunction such as “at least one of’ can mean one, some, or all of the items in the list.

[0020] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried by or stored on a transitory or non-transitory machine-readable (e.g., computer-readable) storage medium, which can be read and executed by one or more processors. The machine-readable storage medium can be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a form readable by a machine, e.g., a volatile or non-volatile memory, media disc, or other media device.

[0021] It will be appreciated that in the examples shown and described further below, the drawings can not be drawn to scale, and can not include all possible layers and / or circuit components. Further, it will be appreciated that while certain drawings show transistor designs including source / drain regions, electrodes, etc. having orthogonal (e.g., vertical) boundaries, embodiments herein can implement such boundaries in a substantially orthogonal manner (e.g., within + / - 5 degrees or + / - 10 degrees of orthogonal) due to manufacturing methods used to create such devices or for other reasons.

[0022] Reference is now made to the drawings, which illustrate embodiments of the application by example, and not by limitation, as follows: Like reference numerals can be used to describe like elements throughout the following disclosure. Like reference numerals with different letter suffixes can be used to denote different instances of like elements. The drawings illustrate the various embodiments discussed in this document, by way of example and not by limitation.

[0023] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present application. It will be apparent, however, that the novel embodiments can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing these embodiments. The intention is to cover all modifications, equivalents, and alternatives falling within the scope of the claims.

[0024] As used herein, the phrase “on” in the context of a first layer or component being on a second layer or component refers to the first layer or component being directly physically attached to the second part or component (with no layers or components between the first layer or component and the second layer or component) or being physically attached to the second layer or component with one or more intervening layers or components.

[0025] As used herein, the term “adjacent” refers to layers or components being in physical contact with each other. That is, there are no layers or components between the stated adjacent layers or components. For example, layer X being adjacent to layer Y refers to layer X being in physical contact with layer Y.

[0026] Reference is now made toFigure 1 In one embodiment, a computing device 100 is shown. The illustrative computing device 100 can include one or more integrated circuit packages 302 (see Figure 3 ), each of which can include one or more dies 202 (see Figure 2 ). The integrated circuit packages 302 and dies 202 can implement a unified test and debug architecture, allowing for testing of components of the computing device 100 at the die level.

[0027] The computing device 100 can be embodied as any type of computing device. For example, the computing device 100 can be embodied as or otherwise included in, without limitation, a server computer, an embedded computing system, a System-on-a-Chip (SoC), a multi-processor system, a processor-based system, a consumer electronic device, a smart phone, a cellular phone, a desktop computer, a tablet computer, a notebook computer, a laptop computer, a networked computer, a wearable computer, a handset, a messaging device, a camera device, and / or any other computing device. In some embodiments, the computing device 100 can be located in a data center, such as, for example, an enterprise data center (e.g., a data center owned and operated by a company and typically located at a company’s premises), a managed services data center (e.g., a data center managed by a third party on behalf of a company), a co-location data center (e.g., a data center in which data center infrastructure is provided by a data center host and a company provides and manages its own data center components (servers, etc.)), a cloud data center (e.g., a data center operated by a cloud service provider that hosts applications and data for companies), and an edge data center (e.g., a data center that typically has a smaller footprint than other data center types, located near the geographic region it serves).

[0028] The illustrative computing device 100 includes a processor 102, a memory 104, an input / output (I / O) subsystem 106, a data storage 108, a communication circuit 110, a display 112, and one or more peripheral devices 114. In some embodiments, one or more of the illustrative components of the computing device 100 can be incorporated into another component, or otherwise form a portion thereof. For example, in some embodiments, the memory 104, or portions thereof, can be incorporated into the processor 102. In some embodiments, one or more of the illustrative components can be physically separated from another component.

[0029] The processor 102 may be embodied as any type of processor capable of performing the functions described herein. For example, the processor 102 may be embodied as (one or more) single-core or multi-core processors, single-socket or multi-socket processors, digital signal processors, graphics processors, neural network computing engines, image processors, microcontrollers, or other processors or processing / control circuits. Similarly, the memory 104 may be embodied as any type of volatile or non-volatile memory or data storage device capable of performing the functions described herein. In operation, the memory 104 may store various data and software used during operation of the computing device 100, such as an operating system, applications, programs, libraries, and drivers. The memory 104 is communicatively coupled to the processor 102 via the I / O subsystem 106, which may be embodied as circuit systems and / or components for facilitating input / output operations with the processor 102, the memory 104, and other components of the computing device 100. For example, the I / O subsystem 106 may be embodied as or otherwise include a memory controller hub, an input / output control hub, a firmware device, communication links (i.e., point-to-point links, bus links, wires, cables, optical guides, printed circuit board traces, etc.), and / or other components and subsystems that facilitate input / output operations. The I / O subsystem 106 may utilize any suitable connectors, interconnects, buses, protocols, etc. (e.g., SoC architecture, USB2, USB3, USB4, The I / O subsystem 106 may connect the various internal and external components of the computing device 100 to each other. In some embodiments, the I / O subsystem 106 may form part of a system on a chip (SoC) and be incorporated onto a single integrated circuit chip along with the processor 102, memory 104, and other components of the computing device 100.

[0030] The data storage device 108 may be embodied as any type of one or more devices configured for short-term or long-term storage of data. For example, the data storage device 108 may include any one or more memory devices and circuits, memory cards, hard drives, solid-state drives, or other data storage devices.

[0031] The communication circuitry 110 may be embodied as any type of interface capable of interfacing the computing device 100 with other computing devices, for example, via one or more wired or wireless connections. In some embodiments, the communication circuitry 110 may be capable of interfacing with any suitable cable type (e.g., electrical or optical cable). The communication circuitry 110 may be configured to communicate using any one or more communication technologies and associated protocols (e.g., Ethernet, WiMAX, near field communication (NFC), etc.). The communication circuitry 110 can be located on a separate silicon from the processor 102, or the communication circuitry 110 can be included in a multi-chip package with the processor 102, or even on the same die as the processor 102. The communication circuitry 110 can be embodied as one or more add-in cards, daughter cards, network interface cards, controller chips, chipsets, specialized components (e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)), or other devices that can be used by the computing device 100 to interface with another computing device. In some embodiments, the communication circuitry 110 can be embodied as part of a system on a chip (SoC) that includes one or more processors, or be included in a multi-chip package that also contains one or more processors. In some embodiments, the communication circuitry 110 can include a local processor (not shown) and / or a local memory (not shown) that are both local to the communication circuitry 110. In such embodiments, the local processor of the communication circuitry 110 can be capable of performing one or more of the functions of the processor 102 described herein. Additionally or alternatively, in such embodiments, the local memory of the communication circuitry 110 can be integrated at the board, socket, chip, and / or other level into one or more components of the computing device 100.

[0032] In some embodiments, the computing device 100 can include other or additional components, such as those commonly found in a computing device. For example, the computing device 100 can also have peripheral devices 114 such as a keyboard, a mouse, a speaker, an external storage device, etc. In some embodiments, the computing device 100 can be connected to a dock that can interface with a variety of devices including the peripheral devices 114. The computing device 100 can include several components, such as a battery, one or more antennas, one or more connectors (e.g., one or more USB2 connectors, one or more USB3 connectors, an SD card slot, a headphone and / or microphone jack, a power connector, etc.), etc.

[0033] Referring now to Figure 2 and Figure 3 in illustrative embodiments, the die 202 of the computing device 100 is shown in Figure 2 and the integrated circuit package 302 is shown in Figure 3 Figure 2 The die 202 shown can be similar to Figure 3 ​Other die 202 integrations in the illustrated integrated circuit package 302. The die 202 can correspond to any suitable portion of any suitable integrated circuit package 302, e.g., the processor 102, the memory 104, the data storage 108, the communication circuitry 110, any other suitable portion of the computing device 100.

[0034] The illustrative die 202 includes a debug endpoint 204 and one or more debug nodes 206. In the illustrative embodiment, the debug endpoint 204 and the debug nodes 206 implement a version of UCIe TM (UCIe TM v 1.0 specification, published by the UCIe Consortium on March 2, 2022) or are otherwise UCIe TM compatible. In the illustrative embodiment, the debug endpoint 204 functions as a hub and the debug nodes 206 function as spokes. The illustrative debug endpoint 204 is a central block for coordinating all debug actions within the die 202. Each debug node 206 is responsible for testing and debug functions of various interfaces or functions within the die 202. The debug endpoint 204 can also be referred to as a UCIe TM test and debug hub (UCIe TM test and debug hub, UDH). The debug nodes 206 can also be referred to as UCIe TM test and debug spokes (UCIe TM Test and Debug Spoke, UDS) or debug agents. The die 202 can include any suitable number of debug nodes 206, e.g., 1-1024. In some embodiments, the debug endpoint 204 can include debug functions. To this end, the debug endpoint 204 can also be referred to as a debug node 206.

[0035] The illustrative integrated circuit package 302 includes one or more dies 304 and / or memory dies 306. The dies 304, 306 can be embodied as the die 202. The dies 304, 306 can be connected by package interconnect circuitry 308. The package interconnect circuitry 308 can be configured to implement any suitable protocol, e.g., UCIe TM . The off-package interconnect 310 can be connected to a test / debugger using another interface such as PCIe. The off-package interconnect 312 can be connected to other packages or other components using the same protocol as the package interconnect 308.

[0036] In illustrative embodiments, one or more of the integrated circuit package 302, the one or more of the dielets 304, 306 will bring out appropriate debug hardware interfaces, e.g., PCIe, SMBus, JTAG, GPIO, USB, I2C / I3C, etc. The (or those) dielets 304, 306 (or debug endpoints 204 within the dielets) will be package-level debug endpoints or package debug endpoints for the integrated circuit package 302. The package debug endpoints can also be referred to as package-level UCIe test and debug hubs or pUDHs. The package debug endpoints are responsible for receiving test / debug commands from an external tester / debugger and forwarding the commands to the various debug endpoints 204 and debug nodes 206 in the integrated circuit package 302 as needed.

[0037] In illustrative embodiments, communication between the debug nodes 206 and the debug endpoints 204 is over the interconnects 208. The interconnects 208, the debug endpoints 204, and the debug nodes 206 form a network on dielet, allowing communication between the debug endpoints 204 and the debug nodes 206 over debug messages. In illustrative embodiments, the interconnects 208 are UCIe TM interconnects, and communication between the debug nodes 206 and the debug endpoints 204 is over UCIe TM packets. One of the debug nodes 206 can be connected to another dielet through a dielet-dielet interconnect 212, which can be a UCIe TM interconnect. One of the debug nodes 206 can also be connected to an external tester / debugger through an external interconnect 210. The external interconnect 210 can be, for example, a peripheral component interconnect express (PCIe). The debug messages can be sent using a UCIe TM vendor ID as normal PCIe vendor-defined message (VDM) transportation layer packets (TLPs) over a PCIe link, can be embodied or encapsulated within a no operation (NOP) PCIe flit, can be tunneled or encapsulated through another protocol such as I2C, I3C, SMBus, GPIO, USB, or can be set via registers in a JTAG-based test / debug setup. The debug messages can also be referred to as UCIe TM debug messages or UCIe TM debug vendor-defined messages (UCIe TMdebug vendor-defined message, UDV) packets. In illustrative embodiments, the external interconnects 210, 212 are connected to the debug node 206. Additionally or alternatively, in some embodiments, the external interconnects 210, 212 can be connected to the debug endpoint 204.

[0038] In illustrative embodiments, the debug endpoint 204 is addressed by an 8-bit endpoint ID in the debug message header 600 (see Figure 6 ). After reset, the debug endpoint ID (DEID) in the first debug message received by the die 202 is used by the illustrative debug endpoint 204 as its endpoint ID until the next reset of the die 202. The package debug endpoint 204 is always assigned the debug endpoint ID 0. The die 202 also stores the port at which it received the first debug message for the purpose of completing debug message routing. In illustrative embodiments, the endpoint ID 0 is reserved for the package debug endpoint 204, and the endpoint ID 0xFF is reserved for broadcast. Debug messages for read completion always indicate a debug endpoint ID of 0, forwarding the read completion to the target port identified when initializing the debug system after reset.

[0039] The illustrative debug endpoint 204 provides a set of registers (see Figure 5 ), including standardized registers and optional proprietary registers. In the set of standardized registers, the debug endpoint 204 presents the vendor ID (VID) of the vendor of the die 202 and the device ID (DID) assigned to the die 202 by the vendor. The illustrative registers can also include the package vendor ID (PVID) and package device ID (PDID) of the package-level integrator corresponding to the package 302.

[0040] Each of the debug nodes 206 within the corelet 202 is assigned a unique debug node ID within the corelet 202. The assignment of debug node IDs is design specific. The debug endpoints 204 use the debug node IDs to route debug messages to the correct debug node 206. In an illustrative embodiment, the debug node ID value 0 is assigned for registers within the debug endpoint 204 itself and potentially for any integrated debug node functionality present within the debug endpoint 204. The illustrative debug nodes 206 can provide a similar or identical set of registers as the debug endpoints 204. The debug nodes 206 provide a standardized mailbox register set that can be read and written using debug messages. The address offset of the register being read or written is provided in the debug message. For example, the mailbox registers can be used to trigger various debug actions, report the status of various debug actions, simple monitor debug signals, etc. In some embodiments, if the debug node 206 type field corresponds to a standard specification such as PCIe, CXL, or UCIe, then the standard registers defined in these specifications for testing or conformance can also be accessed via debug messages. To facilitate implementing this approach, the mailbox index register can provide the encoding for accessing the standardized capabilities of, for example, PCIe, CXL, or UCIe via the capability ID and offset.

[0041] In one embodiment, the mailbox registers are defined as shown in Tables 1-6.

[0042] Table 1, mailbox index low (offset 0x20):

[0043] Table 2, mailbox index high (offset 0x24):

[0044] Table 3, mailbox data low (offset 0x28):

[0045] Bit Attribute Description 31:0 RW Data associated with the request opcode

[0046] Table 4, mailbox data high (offset 0x2C):

[0047]

[0048] Table 5, mailbox control:

[0049]

[0050] Table 6, mailbox status (offset 0x31):

[0051]

[0052] Debug nodes 206 can perform any suitable debug action, e.g., self-test, debug, repair, telemetry monitoring, etc. For example, a debug node 206 for a memory array internal to a die 202 can perform and report a memory built-in self test (MEMBIST). The same debug node 206 can expose its error related telemetry / repair during runtime. Similarly, external links such as PCIe / compute express link (CXL) / UCIe can expose their test, debug, repair, and telemetry capabilities through the corresponding debug node 206.

[0053] In use, debug messages sent to the die 202 are received by the debug endpoints 204 directly or routed through the debug nodes 206. The debug endpoints 204 forward the debug messages to the debug nodes 206 specified in the debug messages using internal buses / structures within the die 202. The debug messages can facilitate a simple register read / write mechanism at the debug nodes 206 to trigger various test and / or debug actions specific to the interface or functionality corresponding to the particular debug node 206. What functionality, parameters, information, etc. are controlled or observed via the debug nodes 206 is design specific. In some cases, the functionality, parameters, information, etc. can be based on or partially or fully correspond to those already defined in a specification such as PCIe, CXL, UCIe, etc. Some examples are test or debug functionality related to external interfaces such as PCIe, CXL, DRAM, etc., test or debug functionality related to internal modules or blocks, or test or debug functionality related to die-to-die interfaces such as UCIe TM ) related to external interfaces such as PCIe, CXL, DRAM, etc., test or debug functionality related to internal modules or blocks, or test or debug functionality related to die-to-die interfaces such as UCIe

[0054] In some embodiments, the debug messages can instruct the debug nodes 206 to enter a streaming mode in which the debug nodes 206 continuously stream debug messages to the debug endpoints 204 or a package debug endpoint 204 on another die 202 in the same package 302. Additionally or alternatively, the debug messages can instruct the debug nodes 206 to dump internal state for debugging upon occurrence of a particular internal event. In these cases, the debug messages can be autonomously generated by the debug nodes 206 that are sent to the debug endpoints 204 on the die 202 and / or a package debug endpoint 204 on another die 202 in the same package 302. The driver interfacing with the debug nodes 206 is responsible for querying the debug system capability to absorb such streaming or burst packets and then enabling such features in the debug nodes 206. In illustrative embodiments, the maximum size of such packets cannot exceed 128 bytes. In some embodiments, the debug nodes 206 can provide a mechanism to snapshot debug signals onto vendor defined registers that can then be read out.

[0055] In operation, the test / debug software is responsible for assigning an endpoint ID to each die 202 and programming the routing registers in each die 202. The package endpoints are responsible for receiving debug messages from the external test / debug software and forwarding the debug commands to the destination debug endpoints 204 or debug nodes 206. The test / debug software configures the debug endpoints 204 for each die 202 in the integrated circuit package 302, for example, by using a depth first search to identify all of the debug endpoints 204. For example, in the illustrated example, the die 304 with DEID of 0 is configured first. The die 304 then configures the adjacent dies 304 with DEID of 1 and DEID of 2. The test / debug software then configures the die 304 with DEID of 3 and the die 306 with DEID of 4 via the interconnect 308 between the die 304 with DEID of 1 and the die 304 with DEID of 3. By configuring in a depth first manner, the debug endpoint values for all of the debug endpoints 204 below the interconnect link 308 from the package debug endpoint 204 can be consecutive, simplifying the routing of debug messages. Figure 4

[0056] Referring now to Figure 4 In one embodiment, example routing paths 402, 404, 406, 408 for sending messages to various dies 304, 306 are shown. In the illustrative embodiment, the routing paths 402, 404, 406, 408 are determined based on the scope registers associated with the interconnects 308 between the dies 304, 306. For example, the scope register associated with the interconnect 308 between the die 304 with DEID of 0 and the die 304 with DEID of 1 is set for debug endpoints 204 1-204 4. The scope register associated with the interconnect 308 between the die 304 with DEID of 0 and the die 304 with DEID of 5 is set for debug endpoints 204 5-204 6. The scope register associated with the interconnect 308 between the die 304 with DEID of 1 and the die 304 with DEID of 2 is set for debug endpoint 204 2. The scope register associated with the interconnect 308 between the die 304 with DEID of 0 and the die 304 with DEID of 7 is set for debug endpoints 204 7-204 8. The scope registers for the debug endpoints 204 have a starting debug endpoint ID value and a length that needs to be forwarded down the corresponding port. Since the debug / test software will know the die topology in the particular package that is being tested, the software can configure the debug endpoint IDs in a depth first manner and program the routing scope registers accordingly.

[0057] Referring now to Figure 5 ​In one embodiment, the layout of the registers 500 in the debug node 206 or the debug endpoint 204 is shown. The fields with vertical lines in the background apply to the debug node 206 and optionally to the debug endpoint 204. The fields with diagonal lines in the background apply to the debug endpoint 204 and are reserved for the debug node 206.

[0058] UDx_CAP[7:0] in the debug endpoint 204 indicates the number of debug nodes 206 that the debug endpoint 204 supports. UDx_CAP[8] indicates whether there is a debug function present. UDx_CAP[8] is always set for the debug node 206 and can be set for the debug endpoint 204 if there is a debug function present. UDx_CAP[l l:9] indicates the type of debug node 206, where 0 indicates UCIe.D2D, 1 indicates UCIe.PHY, 2 indicates UCIe.D2D_p_PHY, 3 indicates PCIe, 4 indicates CXL, 5 indicates a streaming protocol, 6-14 are reserved, and 15 indicates a custom type. UDx_CAP

[13] indicates whether the debug node 206 has timestamp capability. UDx_CAP

[14] indicates the size of the routing register, where 0 indicates a 4-bit range register and 1 indicates an 8-bit range register. Rt_Base and Rt_Len indicate the debug endpoint 204 range register used for UCIe, PCIe, or CXL spoke routing debug messages.

[0059] It should be appreciated that Figure 5 The layout of the registers 500 shown in FIG. 5 is merely one possible embodiment, and other embodiments can have more, fewer, or different registers arranged in different ways.

[0060] Reference is now made to Figure 6 In one embodiment, an illustrative header 600 for an illustrative debug message is shown. An optional payload can be used to read or write content. In one illustrative embodiment, four-byte reads and writes of the registers shown are supported. Support for larger read / write sizes up to a maximum size of 128 bytes can be implementation specific. In an illustrative embodiment, there is a 1:1 correspondence between a request count and a completion count. The illustrative read and write commands have a returned completion packet and have a success or error indicator in the header. Figure 5

[0061] ​Debug messages with header 600 can be sent over the PCIe link as normal PCIe VDM TLPs, can be embedded within NOP flits on the PCIe link, can be transmitted via USB or I2C / SMBus tunneling, or can be set via registers in the JTAG-based test / debug setup. The Destination ID field in the debug message header 600 can be irrelevant and set to 0x0. The Route Type can be set to "Terminate at Receiver".

[0062] When tunneling debug messages over UCIe TM When routing debug messages between chiplets, either the primary lane channel can be used or the sideband channel can be used, as indicated by the UDH ID[0] in the debug message.

[0063] Debug node 206 can optionally support timestamping functionality as part of debug data dump. If debug node 206 supports timestamping, the timestamp always occupies the first doubleword of the payload and is indicated with the "Timestamp Present (TP)" bit set in the debug message.

[0064] Referring now to Figure 7 In some embodiments, if the PCIe link is trained through the PHY layer and supports PCIe flit mode, debug messages can be tunnelled inside PCIe NOP flits in order to be provided in case of PCIe link physical layer up only. In order to support this feature, during PHY layer training, the "PHY Debug NOP Flit Mode" should be negotiated in the modified TS1 / TS2 training sequences of 8b / 10b encoding using the N_FTS field that would otherwise be reserved in flit mode. Once negotiated, debug NOP flits can be sent with debug messages tunnelled inside, as Figure 7The Flit_Marker field 702 is a 2-byte field that indicates the type and subtype of the Flit_Marker. The Flit_Status field 704 is a 2-byte field that indicates the status of the flit. The Flit_Marker field 706 indicates the PTM message contained in the flit. The type and subtype fields in the Flit_Marker tag in the DLP are used to indicate what format bytes 0-235 of the NOP flit have. The type field indicates the format of the debug NOP flit (e.g., PCIe, CXL, or vendor defined). In illustrative embodiments, type 0x0 indicates a PCIe Foundation defined debug NOP flit, type 0x1 indicates a CXL Foundation defined debug NOP flit, type 0x2 indicates a vendor defined debug NOP flit, and other types are reserved. For type 0x0, subtype 0x0 indicates N tunneling TLPs in bytes 0-235 of the associated flit, subtype 0x1 indicates FC information tracked by Tx in bytes 0-235 of the associated flit, and subtype 0x2 indicates FC information tracked by Rx in bytes 0-235 of the associated flit.

[0065] In illustrative embodiments, for debug messages, the type is 0x0 and the subtype is 0x0 to indicate that the debug message is being tunnel in a NOP TLP. Many debug messages can be included internally as long as they do not roll over into the next flit. Subtype opcodes are also provided to debug various new features of PCIe 6.0, e.g., shared FC. Since this mechanism lacks proper flow control, if a receiver is unable to handle the debug messages it receives, it can simply discard the packet and set a status bit in the UDx_Status register to indicate as much.

[0066] To connect to the debug system on the die 202 through the PCIe flits, there are two operational scenarios. In a first scenario, a standalone logic analyzer can be connected to the die’s PCIe port, e.g., through the interconnect 210. In this case, the logic analyzer simply receives the debug flits described above and captures the debug information, which it then visualizes to a user. In a second scenario, the die 202 is connected to an active PCIe link, e.g., through the interconnect 210. In this scenario, the debug information is periodically or opportunistically sent as debug flits that do not go through the replay buffer or transaction layer queues. Additionally or alternatively, a PCIe device on the link can be used as a bridge to a different interface, which can then receive the debug information.

[0067] In an illustrative embodiment, a debug capability structure is defined that declares that the die 202 is capable of sending and receiving debug flits. The frequency of debug flits can be defined in a configuration register that can indicate that x consecutive debug flits should be sent every y flits, where (x=l, y=0) indicates opportunistic debug flit insertion, and (x=0, y=0) indicates all debug flits, e.g., for the case where a standalone logic analyzer is connected to the die 202. A die 202 that expects to support this capability structure bypasses its link layer and transaction layer and connects from its logical PHY to debug flits through its chip-specific internal interface. The die 202 can then route to / from debug endpoints 204 logic.

[0068] If the die 202 supports standalone mode, it must advertise that capability in the capability registers. There must be registers under this capability structure for: (i) target link speed, (ii) Tx presets or coefficients (1b), (iii) per-lane Tx preset values (4b), (iv) per-lane Tx equalization values: 2nd precursor (C-2) - 3 bits, 1st precursor (C-1) - 4 bits, post-cursor (C+1) - 5 bits, (vi) bypass to L0 (1b), (vii) training time (in milliseconds) for the logic analyzer to utilize TS1 ordered sets - 8 bits. It is expected that the test setup has been checked for the coefficients and optimal presets for the data rate at which the logic analyzer is working, and that value has been programmed. When this mode is enabled, the PCIe endpoint on the die 202 ignores what it receives, sets up each of its transmitter lanes with the programmed preset or coefficient at the target data rate, waits for bypass to L0 to be set to 1, sends the required number of TS1 ordered sets, followed by 32 TS2 ordered sets (with EIEOS and SKPOS as required by the specification), sends a SDS ordered set, then enters L0, and sends debug flits while sending periodic SKP ordered sets.

[0069] In a normal link with two full-function ports, debug mode needs to be negotiated in TS2 ordered sets (and modified TS2 ordered sets for alternative protocols such as CXL) along with flit mode during the configuration state on symbol 3, which is currently reserved for flit mode (NFTS in non-flit mode), as follows: bit 0: support debug flits, bit 7: 1 continue to be reserved. When both sides support, debug flits are enabled (and reflected in the status register in debug capability) regardless of whether the re-timer (if any) supports it.

[0070] It should be appreciated that the embodiments described in detail above are merely some possible embodiments of the concepts disclosed herein relating to a unified testing and debug architecture, and other embodiments can also be contemplated. For example, the network-on-chip can be implemented in a manner different than the hub-spoke model (e.g., a mesh network). The protocols used for communication and the debug messages can be implemented differently than described above. In general, other embodiments can allow debug messages to be received over a PCIe link in one die 202 of, for example, the integrated circuit package 302. These debug messages can then be routed within the die 202 and between the dies 202 of the integrated circuit package 302 to reach a target die 202 or debug node 206 within the die 202, which can implement debug functions and / or return information to the requesting device using the debug messages. The methods described herein allow for a general method of debugging individual dies 202 from different vendors and debugging dies 202 from different vendors that have been integrated into the same integrated circuit package 302.

[0071] Reference is now made to Figure 8 In use, the method 800 for creating and testing the dies 202 can be performed. As described in more detail below, hardware such as a computing device with test / debug software (which can be similar to the computing device 100) can be used to test and / or debug the dies 202, the integrated circuit package 302, and the system. The test / debug software can be implemented as instructions carried or stored on a transitory or non-transitory machine-readable (e.g., computer-readable) storage medium that can be read and executed by one or more processors. Additionally or alternatively, hardware such as a logic analyzer can be connected to the dies 202, the package 302, or the system to test / debug the dies 202, the package 302, or the system.

[0072] The method 800 begins at block 802, where one or more dies 202 are created with debug circuitry (e.g., debug endpoints 204, debug nodes 206, interconnects 208, and any other network-on-chip components). In block 804, the dies 202 are tested using the debug circuitry. For example, in one embodiment, a logic analyzer is connected to an external interconnect of the dies 202, e.g., the interconnect 210. Software on the logic analyzer or software controlling the logic analyzer sends debug messages to the dies 202, thereby configuring the debug circuitry. The logic analyzer sends and receives debug messages to, for example, read and write registers, trigger debug functions, receive debug information, etc. If a fault or error is detected, the die 202 can be discarded, thereby preventing a bad die 202 from being integrated into the integrated circuit package 302. Additionally or alternatively, a full link such as a PCIe link can be established with the die 202, where debug messages are sent to test the chip in a similar manner.

[0073] In block 806, one or more dies 202 are integrated into an integrated circuit package 302. The dies 202 can be connected to each other through an on-package network (e.g., through interconnects 308).

[0074] In block 808, the dies 202 in the integrated circuit package 302 are tested using debug circuitry on the dies 202. A link can be established with one die 202, which can route debug messages to other dies 202, as described above. Software can enumerate the various dies 202 and test the functionality of each, as appropriate. A package 302 with a faulty die 202 can have the die 202 replaced or can be discarded entirely.

[0075] In block 810, the package 302 is integrated into a system, e.g., by connecting the package 302 with other packages 302 or other components of a computing device 100.

[0076] In block 812, the dies 202 in the package 302 are tested at a system level. For example, debug software can be run by an operating system or as an application on a computing device 100 that includes the package 302. In another example, remote debug software can interface with a computing device 100 that includes the package 302 to test and / or debug the dies 202, e.g., by using the communication circuitry 110.

[0077] Embodiments

[0078] Illustrative examples of the technologies disclosed herein are provided below. An embodiment of the technology can include any one or more, and any combination of, the examples described below.

[0079] Example 1 includes a die, comprising: a plurality of debug nodes, wherein individual debug nodes of the plurality of debug nodes are to implement a debug function; and an on-die network connecting the plurality of debug nodes, wherein the on-die network comprises circuitry to directly connect at least one debug node of the plurality of debug nodes to an on-die network of another die.

[0080] Example 2 includes the subject matter of Example 1, and wherein the die and the other die are integrated on an integrated circuit package, wherein one or more debug nodes of the plurality of debug nodes are to: receive a debug message from a device external to the integrated circuit package; determine that the debug message is targeted to the other die; and route the debug message to the other die.

[0081] Example 3 includes the subject matter of any of Examples 1 and 2, and wherein one or more debug nodes of the plurality of debug nodes are to: receive a debug message encapsulated in a packet conforming to a first protocol; decapsulate the debug message from the packet conforming to the first protocol; and send the debug message to the other core particle using a second protocol different from the first protocol.

[0082] Example 4 includes the subject matter of any of Examples 1-3, and wherein the first protocol is a Peripheral Component Interconnect Express (PCIe) protocol, and wherein the second protocol is a Universal Chip Interconnect Express (UCIe) protocol.

[0083] Example 5 includes the subject matter of any of Examples 1-4, and further includes a debug endpoint, wherein the debug endpoint is to receive debug messages and route the debug messages to the plurality of debug nodes.

[0084] Example 6 includes the subject matter of any of Examples 1-5, and wherein each of the plurality of debug nodes is configurable to: enter a streaming mode in response to receiving a debug message, in which the debug node continuously sends debug messages to a debug endpoint on the core particle.

[0085] Example 7 includes the subject matter of any of Examples 1-6, and wherein each of the plurality of debug nodes is configurable to enter a dump mode in response to receiving a debug message, wherein the debug node sends information of an internal state to a debug endpoint on the core grain in response to an internal event.

[0086] Example 8 includes the subject matter of any of Examples 1-7, and wherein the debug function of a first debug node of the plurality of debug nodes is a memory built-in self test (MEMBIST).

[0087] Example 9 includes the subject matter of any of Examples 1-8, and wherein the debug function of a second debug node in the plurality of debug nodes is an interconnect test function.

[0088] Example 10 includes an integrated circuit package comprising the die of claim 1 and one or more additional die.

[0089] Example 11 includes a computing device comprising the integrated circuit package of claim 10, a processor, and a memory.

[0090] Example 12 includes an integrated circuit package comprising: a plurality of corelets, wherein each corelet of the plurality of corelets comprises one or more debug nodes and an on-corelet network, wherein each debug node of the one or more debug nodes of each corelet of the plurality of corelets implements a debug function; and package interconnect circuitry to connect the on-corelet networks of the plurality of corelets, wherein the one or more debug nodes of each corelet of the plurality of corelets are able to communicate with the one or more debug nodes of other corelets of the plurality of corelets using the package interconnect circuitry.

[0091] Example 13 includes the subject matter of Example 12, and further including: a package debug endpoint on a first corelet of the plurality of corelets, wherein the package debug endpoint comprises link endpoint circuitry to form a package-external link with a device.

[0092] Example 14 includes the subject matter of any one of Examples 12 and 13, and wherein the link endpoint circuitry is to form a Peripheral Component Interconnect Express (PCIe) link with the device, wherein the link endpoint circuitry is to form a Universal Corelet Interconnect Express (UCIe) with another corelet of the plurality of corelets.

[0093] Example 15 includes the subject matter of any one of Examples 12-14, and wherein the package debug endpoint is to: receive a debug message from the device over the package-external link; determine that the debug message is targeted to another corelet of the plurality of corelets; and route the debug message to the other corelet.

[0094] Example 16 includes the subject matter of any one of Examples 12-15, and wherein the package debug endpoint is to: receive a debug message encapsulated in a packet conforming to a first protocol; decapsulate the debug message from the packet of the first protocol; and send the debug message to another corelet using a second protocol different from the first protocol.

[0095] Example 17 includes the subject matter of any one of Examples 12-16, and wherein the first protocol is a Peripheral Component Interconnect Express (PCIe) protocol, wherein the second protocol is a Universal Corelet Interconnect Express (UCIe) protocol.

[0096] Example 18 includes a computing device comprising the integrated circuit package of claim 12, a processor, and a memory.

[0097] Example 19 includes a method comprising: testing a core die using a plurality of debug nodes of the core die prior to integrating the core die into an integrated circuit package; integrating the core die into the integrated circuit package with a plurality of other core dies, wherein integrating the core die into the integrated circuit package comprises connecting the core die to the plurality of other core dies with package interconnect circuitry; and testing the core die using the plurality of debug nodes and the package interconnect circuitry while integrating the core die into the integrated circuit package.

[0098] Example 20 includes the subject matter of Example 19, and further including: integrating the integrated circuit package into a computing device; and testing the core die using the plurality of debug nodes and the package interconnect circuitry while integrating the integrated circuit package into the computing device.

Claims

1. A core particle comprising: a plurality of debugging nodes, wherein each debugging node in the plurality of debugging nodes is used to implement a debugging function; and A network-on-chip connecting the plurality of debug nodes, wherein the network-on-chip comprises circuitry for directly connecting at least one of the plurality of debug nodes to the network-on-chip of another chip.

2. The core particle according to claim 1, wherein The core particle and the another core particle are integrated on an integrated circuit package, wherein one or more debug nodes of the plurality of debug nodes are used to: receiving a debug message from a device external to the integrated circuit package; Determining that the target of the debug message is the other core particle; and The debug message is routed to the other core particle.

3. The core particle according to claim 1, wherein One or more debugging nodes among the plurality of debugging nodes are configured to: receiving a debug message encapsulated in a packet conforming to a first protocol; decapsulating the debug message from the packet conforming to the first protocol; as well as The debug message is sent to the other coren using a second protocol different from the first protocol.

4. The core particle according to claim 3, wherein The first protocol is a Peripheral Component Interconnect Express (PCIe) protocol, and wherein the second protocol is a Universal Chip Interconnect Express (UCIe) protocol.

5. The core particle according to claim 1 further comprising a debug endpoint, wherein The debugging endpoint is used to receive debugging messages and route the debugging messages to the multiple debugging nodes. The core particle according to claim 1 , wherein: Each debug node of the plurality of debug nodes can be configured to enter a streaming mode in response to receiving a debug message, in which the debug node continuously sends debug messages to a debug endpoint on the chiplet.

7. The core particle according to claim 1, wherein Each debug node of the plurality of debug nodes can be configured to enter a dump mode in response to receiving a debug message, wherein the debug node sends information of an internal state to a debug endpoint on the core chip in response to an internal event.

8. The core particle according to claim 1, wherein The debug function of a first debug node among the plurality of debug nodes is a memory built-in self test (MEMBIST).

9. The core particle according to claim 8, wherein The debug function of a second debug node among the plurality of debug nodes is an interconnect test function.

10. An integrated circuit package comprising the die according to claim 1 and one or more additional die.

11. A computing device comprising the integrated circuit package according to claim 10, a processor and a memory.

12. An integrated circuit package, comprising: a plurality of core particles, wherein each core particle of the plurality of core particles comprises one or more debug nodes and an on-core particle network, wherein each debug node of the one or more debug nodes of each core particle of the plurality of core particles implements a debug function; and a package interconnect circuit system for connecting the network on chiplets of the plurality of chiplets, The one or more debug nodes of each of the plurality of coredies are capable of communicating with the one or more debug nodes of other coredies of the plurality of coredies using the package interconnect circuitry.

13. The integrated circuit package of claim 12, further comprising: a package debug endpoint on a first chiplet of the plurality of chiplets, The package debug endpoint includes a link endpoint circuit system, and the link endpoint circuit system is used to form an out-of-package link with a device.

14. The integrated circuit package of claim 13, wherein: The link endpoint circuitry is configured to form a Peripheral Component Interconnect Express (PCIe) link with the device, wherein the link endpoint circuitry is configured to form a Universal Chip Interconnect Express (UCIe) link with another chiplet of the plurality of chiplets.

15. The integrated circuit package of claim 13, wherein: The encapsulated debug endpoint is used to: receiving a debug message from the device via the outer encapsulation link; determining that the target of the debug message is another corelet among the plurality of corelets; and The debug message is routed to the other core particle.

16. The integrated circuit package of claim 13, wherein: The encapsulated debug endpoint is used to: receiving a debug message encapsulated in a packet conforming to a first protocol; decapsulating the debug message from the packet of the first protocol; as well as The debug message is sent to another chiplet using a second protocol different from the first protocol.

17. The integrated circuit package of claim 16, wherein: The first protocol is a Peripheral Component Interconnect Express (PCIe) protocol, wherein the second protocol is a Universal Chip Interconnect Express (UCIe) protocol.

18. A computing device comprising the integrated circuit package of claim 12, a processor, and a memory.

19. A method comprising: testing the chiplet using a plurality of debug nodes of the chiplet before integrating the chiplet into an integrated circuit package; Integrating the die into the integrated circuit package along with a plurality of other die, wherein integrating the die into the integrated circuit package comprises: connecting the die to the plurality of other die using package interconnect circuitry; and When the die is integrated into the integrated circuit package, the die is tested using the plurality of debug nodes and the package interconnect circuitry.

20. The method according to claim 19, further comprising: integrating the integrated circuit package into a computing device; as well as The die is tested using the plurality of debug nodes and the package interconnect circuitry when the integrated circuit package is integrated into the computing device.

Images