Chip-on-chip (C2C) link status monitor

Abstract

Techniques are disclosed for chip-to-chip (C2C) serial communications, such as communications between core grains on a multi-chip package. In some aspects, a method of on-die monitoring of a C2C link includes detecting a change of the C2C link from a first link state to a second link state; and storing the link state change information in an on-die first-in first-out (FIFO) buffer. The link state change information indicates the first link state, the duration of the C2C link in the first link state and the speed of the C2C link in the first link state. Upon detection of a request for link state change information, the link state change information is retrieved from the FIFO buffer and sent in a serial fashion to an output pin of the die, such as a general purpose input / output (GPIO) pin.

Description

Background Art 1. Technical Field

[0001] Aspects of the present disclosure relate generally to computer chip design, and particularly to chip-to-chip (C2C) communications, such as communications between die on a multi-chip package.

[0002] 2. Description of related technologies

[0003] A chiplet is a part of a processing module that makes up a larger integrated circuit, such as a computer processor. Instead of manufacturing a processor on a single silicon wafer with the desired number of cores, chiplets allow manufacturers to use multiple smaller chips to make up a larger integrated circuit. Multiple chiplets working together in a single integrated circuit are called a multi-chip module (MCM). Chips are becoming increasingly popular in different market segments due to the advantages they offer over larger monolithic chips, such as improved performance and higher wafer processing yields.

[0004] Chip-to-chip (C2C) communications, such as between die on a multi-chip package, typically occur on conductive traces on or within a package substrate to which the die or die is attached. Die-to-die (D2D) communications are similar, and the term C2C ​​will be used hereinafter to refer to either or both C2C and D2D. C2C link architecture implementations are influenced by the Peripheral Component Interconnect Express (PCIE) physical link layer and data link layer.

[0005] However, a C2C or D2D link may not have external visibility for electrical analysis or protocol analysis, which means that a protocol analyzer cannot be used because the link traces are not visible on the chip package. Summary of the invention

[0006] The following presents a simplified summary of the invention related to one or more aspects disclosed herein. Thus, the following summary of the invention should neither be considered as an exhaustive overview related to all conceived aspects, nor should it be considered to identify key or decisive elements related to all conceived aspects or to delineate the scope associated with any particular aspect. Therefore, the sole purpose of the following summary of the invention is to present certain concepts related to one or more aspects of the mechanisms disclosed herein in a brief form before the detailed embodiments presented below.

[0007] In one aspect, a method for on-die monitoring of a chip-to-chip (C2C) serial link state includes: detecting a change in a link state of the C2C serial link from a first link state to a second link state; storing link state change information in an on-die first-in-first-out (FIFO) buffer, the link state change information including information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; detecting a request for link state change information; retrieving link state change information from the FIFO buffer; and sending the link state change information to an output pin in a serial manner.

[0008] In one aspect, an apparatus for on-die monitoring of C2C serial link training status and link status comprises: a die comprising: a C2C serial link circuit comprising: a physical layer configured to send and receive data via a C2C serial link according to a C2C communication protocol; a serial link state machine (SLSM) configured to perform link training of the C2C serial link; and an SLSM monitor coupled to the SLSM and configured to: detect a change in the link state of the C2C serial link from a first link state to a second link state; store link state change information in an on-die FIFO buffer, the link state change information comprising information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; detect a request for link state change information; retrieve link state change information from the FIFO buffer; and send the link state change information serially to an output pin of the die.

[0009] In one aspect, an apparatus includes: means for detecting a change in a link state of a C2C serial link from a first link state to a second link state; means for storing link state change information in an on-die FIFO buffer, the link state change information including information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; means for detecting a request for link state change information; means for retrieving link state change information from the FIFO buffer; and means for sending the link state change information in a serial manner to an output pin.

[0010] In one aspect, a non-transitory computer-readable medium storing computer-executable instructions that, when executed by an apparatus, cause the apparatus to: detect a change in a link state of a C2C serial link from a first link state to a second link state; store link state change information in an on-die FIFO buffer, the link state change information comprising information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; detect a request for link state change information; retrieve the link state change information from the FIFO buffer; and send the link state change information in a serial manner to an output pin.

[0011] In one aspect, information about the C2C link status (including the actual state of the state machine, the validity of the state, and the speed of the link) is output on a general purpose input / output (GPIO) pin that can be monitored using a logic analyzer. In this way, the C2C link status of both link partners can be monitored simultaneously.

[0012] Other objects and advantages associated with the various aspects disclosed herein will be apparent to those skilled in the art based on the drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are presented to aid in describing various aspects of the present disclosure and are provided solely for illustration and not limitation of the various aspects.

[0014] Figure 1 The top-level states of the Serial Link State Machine (SLSM) are illustrated, and specifically the PCIE Link Training Status and State Machine (LTSSM) is illustrated.

[0015] Figure 2 Chip-to-chip (C2C) communication using an on-die C2C SLSM monitor (SLSMM) in accordance with aspects of the present disclosure is illustrated.

[0016] Figure 3 An example C2C SLSMM according to aspects of the present disclosure is illustrated in more detail.

[0017] Figure 4A , Figure 4B and Figure 4C are diagrams each illustrating an example C2CSLSMM serial output according to aspects of the present disclosure.

[0018] Figure 5 is a flow chart illustrating a portion of an example process associated with an on-die C2C SLSMM according to aspects of the present disclosure. DETAILED DESCRIPTION

[0019] Techniques for chip-to-chip (C2C) serial communications, such as communications between die on a multi-chip package, are disclosed. In some aspects, a method for on-die monitoring of a C2C link includes: detecting, at an on-die C2C link circuit for communicating via the C2C link according to a C2C protocol, a change of the C2C link from a first link state to a second link state; and storing link state change information in an on-die first-in-first-out (FIFO) buffer. The link state change information indicates the first link state, the duration that the C2C link is in the first link state, and the speed of the C2C link in the first link state. Upon detecting a request for link state change information, the link state change information is retrieved from the FIFO buffer and sent serially to an output pin of the die, such as a general purpose input / output (GPIO) pin.

[0020] Various aspects of the present disclosure are provided in the following description and related drawings for various examples provided for illustrative purposes. Alternative aspects may be designed without departing from the scope of the present disclosure. Additionally, well-known elements of the present disclosure will not be described in detail or will be omitted to avoid making the relevant details of the present disclosure difficult to understand.

[0021] The words "exemplary" and / or "example" are used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" and / or "example" is not necessarily to be construed as preferred or superior to other aspects. Likewise, the term "aspects of the disclosure" does not require that all aspects of the disclosure include the discussed feature, advantage, or mode of operation.

[0022] Those skilled in the art will appreciate that any of a variety of different techniques and methods may be used to represent the information and signals described below. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the following description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof, which depends in part on the specific application, in part on the desired design, in part on the corresponding technology, and so on.

[0023] In addition, many aspects are described in terms of a sequence of actions to be performed by, for example, an element of a computing device. It will be appreciated that the various actions described herein may be performed by a specific circuit (e.g., an application specific integrated circuit (ASIC)), by program instructions executed by one or more processors, or by a combination of the two. In addition, the sequence of actions described herein may be considered to be fully embodied in any form of non-transitory computer-readable storage medium, in which a corresponding computer instruction set is stored, which, when executed, will cause or command the associated processor of the device to perform the functionality described herein. Therefore, various aspects of the present disclosure may be embodied in a variety of different forms, all of which have been contemplated to be within the scope of the claimed subject matter. In addition, for each of the various aspects described herein, the corresponding form of any such aspect may be described herein as, for example, "logic configured to perform the described actions."

[0024] A chiplet is a part of a processing module that makes up a larger integrated circuit, such as a computer processor. Instead of manufacturing a processor on a single silicon wafer with the desired number of cores, chiplets allow manufacturers to use multiple smaller chips to make up a larger integrated circuit. Multiple chiplets working together in a single integrated circuit are called a multi-chip module (MCM). Chips are becoming increasingly popular in different market segments due to the advantages they offer over larger monolithic chips, such as improved performance and higher wafer processing yields.

[0025] Chip-to-chip (C2C) communications (such as communications between corelets on a multi-chip package) typically occur on conductive traces on or within a package substrate to which the corelets or dies are attached. Die-to-die (D2D) communications are similar, and the term C2C ​​will be used hereinafter to refer to either or both C2C and D2D. In some aspects, the C2C link architecture implementation is influenced by the peripheral component interconnect express (PCIE) physical link layer and data link layer.

[0026] Figure 1 The top-level states of a serial link state machine (SLSM) are illustrated, and specifically the PCIE link training status and state machine (LTSSM) 100 is illustrated. The LTSSM states are defined by the PCIE specification. The LTSSM state machine and PCIE protocol are used herein as an illustrative example of how to implement serial communication between link partners, and are not limiting: the same principles described herein can be applied to other protocols and other state machines.

[0027] Referring now to the PCIE LTSSM as an illustrative example, the top-level states include Detect, Poll, Configure, L0, L0s, L1, L2, Recovery, Loopback, Hot Reset, and Disabled. Each state is composed of sub-states, which together make up the state. The first LTSSM state after reset is the Detect state. Detect, Poll, and Configure are link training states; Recovery is a link retraining state; L0, L1, and L2 are power management states; L0s is an active power management state, as is L1. Loopback, Hot Reset, and Disabled are other states.

[0028] Table 1 lists the LTSSM sub-states defined in the PCIE specification. The value assignments for the sub-states are implementation dependent and are included as an illustration of possible implementations. The value is a six-digit hexadecimal number.

[0029] Table 1- PCIE LTSSM sub-state

[0030]

[0031]

[0032] The PCIE states listed in Table 1 illustrate that a high-speed serial protocol may have the following characteristics: the protocol initially starts at a low rate on the serial link and then uses a state-based handshake mechanism to reach the highest data rate achievable under the current conditions, where the specific states and the order of state transitions are defined by the protocol. In some cases, the protocol may define a minimum time that a device must remain in a specific state before being allowed to enter the next state. In some cases, the protocol may define a maximum time that a device can remain in a specific state before entering an error recovery state.

[0033] High-speed, multi-gigabit interfaces that are critical to product functionality and reliability are extremely complex and challenging. Many registers must be tuned for optimal performance, and the serial link state machine needs to be probed for link startup, link debug, diagnostics, and link training optimization. In addition, serial link state monitoring that can be used to isolate power, system, thermal noise, or design sensitivity issues will be required for troubleshooting at the customer site. The link speed needs to be monitored as well as the underlying serial link state machine state and sub-state transitions and the time spent in each state, which directly affects the total initialization time and thus the system throughput.

[0034] However, a C2C or D2D link may not have external visibility for electrical analysis or protocol analysis, which means that a protocol analyzer cannot be used because the link traces are not visible on the chip package. If implemented as an alternative, a complex and expensive on-chip debug logic setup would be required.

[0035] Figure 2 A block diagram illustrating a C2C implementation with an on-die SLSM monitor and SLSM monitor logic according to aspects of the present disclosure is illustrated. Figure 2 In the embodiment, two cores, core 1 202 and core 2 204, are mounted on a substrate 200. Core 1 202 includes a first C2C serial transceiver C2C1 206 coupled to a first physical layer circuit PHY1 208. C2C1 206 includes a first SLSM circuit SLSM1 210. C2C1 206 also includes a first SLSM monitor SLSMM1 212 for monitoring SLSM1 210. Core 2 204 includes a second C2C serial transceiver C2C2 214 coupled to a second physical layer circuit PHY2 216. C2C2 214 includes a second SLSM circuit SLSM2 218. C2C2 214 includes a second SLSM monitor SLSMM2 220 220 for monitoring SLSM2 218. In some aspects, SLSM1 210 and SLSM2 218 may be PCIE LTSSMs, and SLSMM1 212 and SLSMM2 220 may be LTSSM monitors. In some aspects, a C2C link may be the same or similar to a PCIE, and the examples disclosed herein may reflect this similarity, but it should be noted that the same principles may apply to C2C links of other protocols.

[0036] Chip1 202 and Chip2 204 communicate with each other via a C2C link using conductive traces 222 and 224 on or embedded in substrate 200. Conductive traces 222 and 224 provide electrical connections between PHY1 212 and PHY2 220. The C2C link itself is not visible at the chip package and is completely contained within the substrate. The SLSM monitor is used to provide link status, link speed, and time spent in each state to both sides of the link partner using only 2 general purpose input / output ports (GPIO). Figure 2 In the example, SLSMM1 212 outputs this information on GPIO1 226 and SLSMM2 220 outputs this information on GPIO2 228. This feature is critical for link startup, link debugging, and link optimization.

[0037] The corresponding SLSM circuit maintains the C2C link training status and current state of its corresponding chiplet. For simplicity of description, only the operation of one of the SLSM monitors will be described, but it should be noted that the other SLSM monitor will perform the same function for the other chiplet.

[0038] Referring to C2C1 206 on coregrain 1 202, SLSM1 210 provides the current link state to SLSMM1 212. SLSMM1 212 also knows the core clock rate core1 clk rate of coregrain 1 202. Therefore, SLSMM1 212 maintains a running list of the states that SLSM1 210 is in and the length of time that SLSM1 210 is in each state based on the number of core clock cycles. Upon request, SLSMM1 212 can provide this information to entities external to coregrain 1 202, for example, via GPIO1 226. Similarly, SLSMM2 220 maintains the same kind of information about SLSM2 218 and can provide this information to entities external to coregrain 2 204 via GPIO2 228. In some aspects, this information can be provided in a unique form disclosed in more detail below, but other formats for providing this information are also within the scope of the present disclosure and are aspects of the present disclosure.

[0039] In this manner, the state of the C2C link between Chip1 202 and Chip2 204 can be made visible for monitoring and / or analysis external to Chip1 202 and Chip2 204, such as by a logic analyzer 230 (or other type of C2C link monitoring equipment) that analyzes the state of SLSM1 210 and SLSM2 218 by processing serial output data 232 received from GPIO1 226 and serial output data 234 received from GPIO2 228, respectively. In some aspects, the logic analyzer 230 can be part of another chip mounted to the substrate 200. In some aspects, the logic analyzer 230 can be a piece of bench test equipment that is electrically connected to the substrate 200 for testing and debugging purposes only. These examples are illustrative and not limiting. The present disclosure contemplates other specific implementations of the logic analyzer 230.

[0040] Analysis of the serial output data 232 from SLSMM1 212 and the serial output data 234 from SLSMM2 220 allows the logic analyzer 230 to monitor and debug both partners of the embedded C2C serial link simultaneously (e.g., one partner per GPIO) and in real-time or near real-time. For clarity, Figure 2 Details about triggering SLSMM1 212 or SLSMM2 220 to output serial output data 232 and serial output data 234, respectively, are not shown, but in some aspects, these serial output data can be output in response to an external request (e.g., from logic analyzer 230) or in response to an internal trigger (e.g., once it is available, in which case logic analyzer 230 can monitor these serial output data for a bit sequence indicating the beginning of a serial output data sequence). Other triggering modes are also contemplated by the present disclosure.

[0041] Figure 3 An example SLSM monitor according to aspects of the present disclosure is illustrated in more detail. Figure 3 In the example illustrated in FIG. 2 , the SLSM monitor SLSMM1 212 is divided into two clock domains, namely the core clock domain 300 and the serial clock domain 302. An asynchronous first-in-first-out (FIFO) buffer 304 operates in both clock domains. The main function of SLSMM1 212 is to monitor the SLSM of the C2C link (e.g., Figure 2 210) and outputs the states to a GPIO (e.g., GPIO 226) in a serial manner along with the number of cycles spent in each state. In some aspects, SLSMM1 212 includes a FIFO control block 306 for controlling the asynchronous FIFO 304. The FIFO control block 306 manages the write pointer 308 and the read pointer 310 as well as the write control signal (Wr) and the read control signal (Rd). The control block 312 controls SLSMM1 212.

[0042] Core clock domain. In some aspects, the core clock domain 300 stores information indicating the current state of the SLSM (SLSM state) 314 and information indicating the core clock rate (core clk rate) 316, and uses, for example, a counter 318 to track how long the SLSM remains in the current state. In some aspects, the core clock domain 300 begins monitoring when it is enabled (e.g., via a control signal received by the control block 312), and at least one of the following conditions is met: the SLSM state changes to a non-zero value; or the SLSM state is equal to a specific state, and the logic is configured to trigger in a specific state. In some aspects, once triggered, the core clock domain 300 will count the number of core clock cycles spent in the current state. In some aspects, the core clock domain stores the core clock rate, the current SLSM state, and the number of cycles spent in the state to the asynchronous FIFO 304. In some aspects, the asynchronous FIFO 304 is designed to accommodate SLSM state changes for typical link training scenarios. In the event that the SLSM state changes more frequently than would cause the asynchronous FIFO 304 to be full, a status indicating this error condition will be stored for transmission in the serial output.

[0043] Asynchronous FIFO. In some aspects, the asynchronous FIFO 304 writes data at the core clock frequency. In some aspects, the asynchronous FIFO 304 provides a "FIFO full" flag to the write control logic to avoid overwriting valid data. In some aspects, the asynchronous FIFO 304 reads data when valid data is available at the serial clock frequency. In some aspects, the asynchronous FIFO 304 provides a "FIFO empty" flag to the read control logic to indicate the availability of valid data. In some aspects, the read data is provided to the serializer 320, which sends the valid data out in a serial manner via the GPIO.

[0044] Serial clock domain. In some aspects, the serial clock domain 302 reads valid data (when available) from the asynchronous FIFO 304 and outputs the valid data via the GPIO 226. In some aspects, the data read from the asynchronous FIFO 304 includes the SLSM state and the number of cycles in the state. In some aspects, the serializer 320 serializes the data at the serial clock frequency. In some aspects, for example, in the case where C2C is based on PCIE and the SLSM is PCIE LTSSM, the serial clock domain 302 transmits the serial data as 32 bits to the GPIO in the following format:

[0045] Bit 31 30:28 27:26 25:20 19:0 Fields efficient PCIE gen rate situation LTSSM status / substatus Core clock cycle count

[0046] Wherein, the core clock cycle count refers to the number of core clock cycles spent in each LTSSM state or sub-state. In some aspects, when the SLSM or core clock rate remains unchanged for a long time, the logic will force serial output and the clock cycle count is set to the maximum value that can be configured, and the count continues with the next entry in the asynchronous FIFO 304. Some examples of serial output are shown in FIG. 4A to FIG. 4C Shown in.

[0047] Figure 4A , Figure 4B and Figure 4C are diagrams each illustrating an example SLSM monitor serial output using an example format for presenting SLSM state machine information in a serial manner according to aspects of the present invention. FIG. 4A to FIG. 4C In the example shown, the C2C link is based on PCIE and uses the PCIE LTSSM state machine, but the same principles can be applied to other C2C serial protocols. FIG. 4A to FIG. 4C The counter values ​​shown are illustrative and non-limiting; actual counter values ​​will typically be larger than FIG. 4A to FIG. 4C The examples shown in the figure have higher values.

[0048] Figure 4A10000000000001 1 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0049] Figure 4B 100} . Example SLSM monitor serial output showing clock rate conversion to gen 4 speed. The bits of the core_clk_rate field = {011}, which indicates a core clock rate of 500 MHz. The bits of the status field = {00}, which indicates OK (e.g., no errors or overflows occurred). The bits of the ltssm_state field = {001101}, which indicates the Recovery.Speed ​​substate. The bits of the core_clk_cycles field = {00000000000111011100}, which indicates 476 cycles.

[0050] Figure 4C Example SLSM monitor serial output showing an overflow condition. When an overflow condition occurs, the state counter will be forced to 0. The bits of the core_clk_rate field = {011}, which indicates a core clock rate of 500 MHz. The bits of the status field = {01}, which indicates an overflow has occurred. The bits of the ltssm_state field = {001101}, which indicates the Recovery.Speed ​​substate. Due to the overflow condition, the bits of the core_clk_cycles field are all set to zero.

[0051] In some aspects, the serial output can be captured with an analyzer on both link partners. The data can then be post-processed to provide visibility into link initialization and training. Several examples of post-processed serial outputs of two link partners (Partner 0 and Partner 1) are shown in the table below. These examples are for illustration only and are intended to show example LTSSM states or sub-state transitions seen from each end of an example C2C link based on PCIE, but the same principles can be applied to C2C links and state machines using protocols other than PCIE. The time values ​​are selected to be illustrative and do not necessarily reflect true link training values.

[0052] Table 2 shows the post-processed output of a first example of monitoring a C2C link between Partner 0 and Partner 1. Table 2 shows an example where each partner can establish a unidirectional link at a first link speed Gen1 and then increase the link speed to a second link speed Gen4. The time taken for each of these steps (i.e., the number of clock cycles that the link remains during each corresponding state or sub-state) is approximately the same for both partners.

[0053] Table 2

[0054]

[0055]

[0056] Of particular note is the time spent at Gen1 speed (state #3 in Table 2) before the link is able to change to Gen4 speed (state #4 in Table 2). In the example above, the link stayed at Gen1 for approximately 200 clock cycles before transitioning to Gen4.

[0057] Table 3 shows the post-processed output of the second example of monitoring a C2C link involving two partners.

[0058] Table 3

[0059]

[0060] In this example, Partner 0 remained at Gen1 speed for 4325 clock cycles (state #3 in Table 3) before switching to Gen4 speed, while Partner 1 was able to change to Gen4 speed after the typical 200 clock cycles or so. The longer time taken to change to Gen4 speed indicates that Partner 0's phase-locked loop (PLL) took longer to lock to the new speed. With this knowledge, the developer or debugger can determine that the PLL control registers should be adjusted to speed up the locking process. Without this knowledge, the developer or debugger may have little indication that optimizations can be made during the link establishment process.

[0061] Table 4 shows the post-processed output of the third example of monitoring a C2C link involving two partners.

[0062] Table 4

[0063]

[0064] In this example, Partner 0 remained at Gen1 speed for 4325 clock cycles before transitioning to Gen4 speed (State #3 in Table 4), while Partner 1 was able to change to Gen4 speed after the typical 200 clock cycles or so, again indicating that the PLL control registers on Partner 0 should be adjusted to speed up the locking process. However, in addition, the table above indicates that for Receiver Equalization Stage 3 (State #8 in Table 4), Partner 1 took significantly longer than Partner 0. This indicates that the default transmit equalization settings on Partner 0 are not optimal for Partner 1’s signal reception.

[0065] Figure 5 is a flow chart illustrating portions of an example process 500 associated with an on-die C2C link state monitor according to aspects of the present disclosure. In some implementations, Figure 5 One or more process blocks of may be performed by a C2C link circuit (eg, C2C serial transceiver 206, SLSMM1 212). In some implementations, Figure 5 One or more process blocks of may be performed by another device or a group of devices separate from or including the C2C link circuit. Additionally or alternatively, Figure 5 One or more process blocks of process 500 may be performed by one or more components of a device (such as a processor, memory, transceiver), any or all of which may be means for performing the operations of process 500.

[0066] like Figure 5 As shown, process 500 may include: at block 502, detecting a change in the link state of a C2C link from a first link state to a second link state. Since the change in the link state of the C2C serial link is reflected by the SLSM, the change in the link state corresponds to a change in the SLSM state. In some aspects, the means for performing the operation of block 502 may include an on-die C2C link circuit for communicating via the C2C link according to the C2C protocol, such as a C2C serial transceiver 206. For example, SLSMM1 212 may receive a notification of a new SLSM state from SLSM1 210. In some aspects, detecting a change in the link state of the C2C link includes detecting a change in the link state of a peripheral component express interconnect (PCIE) protocol link.

[0067] like Figure 5As further shown, process 500 may include, at block 504, storing link state change information in an on-die first-in, first-out (FIFO) buffer. Means for performing the operations of block 504 may include C2C serial transceiver 206. For example, SLSMM1 212 may receive the SLSM state from SLSM1 210 and, before storing it in the SLSM state 314, may create an entry in the asynchronous FIFO 304 that includes a previous value in the SLSM state 314 and a current value in the counter 318. Once the information is stored in the asynchronous FIFO 304 (e.g., by the FIFO control block 306), the write pointer 308 may be updated, the counter 318 may be cleared, and the received SLSM state may be stored in the SLSM state 314. Thus, in some aspects, the link state change information may include information indicating a first link state, a duration that the C2C link was in the first link state, and a speed of the C2C link in the first link state.

[0068] In some aspects, storing information indicating the first link state includes storing information indicating a PCIE LTSSM state or sub-state. In some aspects, storing information indicating a duration that the C2C link is in the first link state includes storing information indicating a number of clock cycles that the C2C link is in the first link state.

[0069] like Figure 5 As further shown, process 500 may include detecting a request for link state change information at block 506. Means for performing the operations of block 506 may include SLSMM1 212. Such a request may be received by control block 312, for example.

[0070] like Figure 5 As further shown, process 500 may include, at block 508, retrieving link state change information from the FIFO buffer. Means for performing the operations of block 508 may include SLSMM1 212. For example, control block 312 may trigger FIFO control block 306 to instruct asynchronous FIFO 304 to transfer the oldest entry (pointed to by read pointer 310) to serializer 320 and update read pointer 310.

[0071] like Figure 5 As further shown, process 500 may include, at block 510, sending link state change information to an output pin in a serial manner. Means for performing the operations of block 510 may include SLSMM1 212. For example, serializer 320 may then output the state change information received from asynchronous FIFO 304 via GPIO 226 in a serial manner.

[0072] In some aspects, sending link state change information serially to an output pin includes sending at least one of: one or more bits indicating that the serial output is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; one or more bits indicating a link state; or one or more bits indicating a duration that the C2C link is in a link state.

[0073] Process 500 may include additional implementations, such as any single implementation or any combination of implementations described below and / or in conjunction with one or more other processes described elsewhere herein. Figure 5 An example block diagram of process 500 is shown, but in some implementations, process 500 may include Figure 5 More blocks, fewer blocks, different blocks, or blocks arranged in a different manner than those depicted in the process 500. Additionally or alternatively, two or more blocks in the blocks of process 500 may be executed in parallel.

[0074] The on-die SLSM monitor disclosed herein has several technical advantages. For example, in a conventional multi-chip module, since link partners are embedded in the substrate, these link partners cannot be accessed, but the on-die SLSM monitor provides the ability to monitor link transitions after post-processing the serialized status data.

[0075] Since the C2C link operates at very high rates, it is not possible to simply provide electrical connections directly to the conductive traces on which the link operates without causing unacceptable degradation in link performance due to the additional impedance, inductance, and noise associated with the electrical connections to observable package pins. Even assuming that these technical challenges can be overcome, there remains the problem that analysis of the C2C link signals so obtained would require an expensive protocol analyzer.

[0076] In contrast, the on-die SLSM monitor disclosed herein pre-processes the link data to provide real-time status updates, which occur at a rate much lower than the data rate of the C2C link itself. Therefore, this information can be provided by the SLSM monitor via a GPIO pin or other output pin at a lower clock rate, i.e., this can be easily accommodated by observable package pins. Because the SLSM monitor disclosed herein provides an overview of the activity on the link, e.g., identifying the states that the link enters and the length of time that the link stays in each state, this eliminates a portion of the post-processing that an external protocol analyzer would need to perform, thereby simplifying the analysis of the data output by the SLSM monitor on the GPIO pin.

[0077] In addition, the on-die SLSM monitor provides the ability to simultaneously monitor the SLSM status and C2C link speed across both link partners of the C2C link in a serial manner using only two GPIOs. Since the SLSM status updates are provided using a single GPIO on each side compared to the entire bus, this saves valuable space on the substrate that would otherwise be occupied by signal traces.

[0078] Because there can be an SLSM monitor on each end of the C2C link, both sides of the C2C link can be monitored and debugged simultaneously, which can result in a significant reduction in test time. Based on the same principle, multiple C2C links can be monitored and debugged simultaneously, further reducing test time. Alternatively, the same GPIO pin on the same core can be used to monitor a collection of C2C links from one core to one or more other cores in sequence. For example, the GPIO pin on core A can be used to monitor the link from core A to core B, and then the same GPIO pin on core A can be used to monitor the link from core A to core C, and so on. Alternatively, one GPIO pin on core A can be used to monitor the C2C link with core B, and another GPIO pin on core A can be used to monitor the C2C link with core C, and so on.

[0079] Another advantage of the on-die SLSM monitor disclosed herein is that it can capture link status information in real time. Conventional methods such as monitoring control and status registers (CSRs) or determining statistics through firmware do not allow real-time monitoring. Real-time monitoring is critical for debugging transient events that affect platform design as well as power distribution network (PDN) and packaging characteristics.

[0080] For example, in some aspects, the time spent in each SLSM state provides a quantifiable means to fine-tune the various CSR registers of the C2C PHY. In some aspects, adaptive feedback to the PHY optimization loop is used to improve initialization and startup time in real time. In some aspects, such adaptive feedback is deployed as a self-correcting feature within the product life cycle. In addition, improving startup time and latency has a direct impact on power-performance metrics, which can be a key product differentiator. Optimization of link initialization based on information provided by the on-die SLSM monitor is useful in applications with strict operating time (such as autonomous drive link connection, where the vehicle needs to work within a few seconds) and in key cloud infrastructure solutions. Another benefit to the customer is that, using the on-die SLSM monitor disclosed herein, the LTSMM state can be a monitor at the customer site during startup using an inexpensive logic analyzer, for example, without the need to maintain an engineering-intensive firmware image for performing the logic analyzer function.

[0081] In the above detailed description, it can be seen that different features are grouped together in each example. This disclosure should not be understood as an intention that the example clauses have more features than the features explicitly mentioned in each clause. On the contrary, various aspects of the present disclosure may include less than all the features of the disclosed individual example clauses. Therefore, the following clauses should be deemed to be incorporated into the description accordingly, where each clause itself can be used as a separate example. Although each dependent clause can refer to a specific combination of one clause with other clauses in a clause, the aspects of the dependent clause are not limited to a specific combination. It should be understood that other example clauses may also include a combination of the subject matter of the dependent clause aspect with any other dependent clause or independent clause or any feature with other dependent clauses and independent clauses. The various aspects disclosed herein explicitly include these combinations, unless it is explicitly expressed or can be easily inferred that a specific combination is not intended to be used (e.g., contradictory aspects, such as defining an element as both an electrical insulator and an electrical conductor). In addition, it is also expected that various aspects of the clause may be included in any other independent clause, even if the clause is not directly dependent on the independent clause.

[0082] Specific implementation examples are described in the following numbered clauses:

[0083] Item 1. A method for on-die monitoring of a C2C serial link state, the method comprising: detecting a change in the link state of the C2C serial link from a first link state to a second link state; storing link state change information in an on-die FIFO buffer, the link state change information comprising information indicating the first link state, a duration for which the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; detecting a request for link state change information; retrieving link state change information from the FIFO buffer; and sending the link state change information to an output pin in a serial manner.

[0084] Clause 2. The method of clause 1, wherein storing information indicating the duration that the C2C serial link is in the first link state comprises storing information indicating a number of clock cycles that the C2C serial link is in the first link state.

[0085] Clause 3. The method of any one of clauses 1 to 2, wherein sending the link state change information to the output pin comprises sending the link state change information to a GPIO pin.

[0086] Clause 4. A method according to any one of clauses 1 to 3, wherein sending the link state change information serially to the output pin includes sending at least one of the following: one or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; one or more bits indicating a link state; or one or more bits indicating a duration that the C2C serial link is in the link state.

[0087] Clause 5. A method according to any one of clauses 1 to 4, wherein detecting the change in the link state of the C2C serial link includes detecting the change in the link state of the PCIE protocol link, and wherein storing information indicating the first link state includes storing information indicating a PCIE LTSSM state or sub-state.

[0088] Item 6. A device for on-die monitoring of C2C serial link training status and link status, the device comprising: a die, the die comprising: a C2C serial link circuit, the C2C serial link circuit comprising: a physical layer, the physical layer being configured to send and receive data via a C2C serial link according to a C2C communication protocol; an SLSM, the SLSM being configured to perform link training of the C2C serial link; and an SLSM monitor, the SLSM monitor being coupled to the SLSM and being configured to: detect a change in the link status of the C2C serial link from a first link state to a second link state; store link status change information in an on-die FIFO buffer, the link status change information comprising information indicating the first link state, the duration that the C2C serial link is in the first link state, and the speed of the C2C serial link in the first link state; detect a request for link status change information; retrieve link status change information from the FIFO buffer; and send the link status change information serially to an output pin of the die.

[0089] Clause 7. An apparatus according to Clause 6, wherein, in order to store information indicating the duration that the C2C serial link is in the first link state, the SLSM monitor is configured to store information indicating the number of clock cycles that the C2C serial link is in the first link state.

[0090] Clause 8. The apparatus of any of Clauses 6 to 7, wherein, to send the link state change information to the output pin of the die, the SLSM monitor is configured to send the link state change information to a GPIO pin.

[0091] Clause 9. An apparatus according to any one of clauses 6 to 8, wherein, in order to send the link state change information to the output pin in a serial manner, the SLSM monitor is configured to send at least one of the following: one or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating the status of the link state change information; one or more bits indicating a link state; or one or more bits indicating the duration that the C2C serial link is in the link state.

[0092] Clause 10. An apparatus according to any one of clauses 6 to 9, wherein the C2C communication protocol comprises a PCIE protocol, and wherein the serial link state machine comprises a PCIE LTSSM, and wherein the information indicating the first link state comprises information indicating an LTSSM state or sub-state.

[0093] Item 11. A device comprising: a component for detecting a change in the link state of a C2C serial link from a first link state to a second link state; a component for storing link state change information in an on-die FIFO buffer, the link state change information comprising information indicating the first link state, the duration that the C2C serial link is in the first link state, and the speed of the C2C serial link in the first link state; a component for detecting a request for link state change information; a component for retrieving link state change information from the FIFO buffer; and a component for sending the link state change information in a serial manner to an output pin.

[0094] Clause 12. The apparatus of Clause 11, wherein the means for storing the information indicative of the duration that the C2C serial link is in the first link state comprises means for storing information indicative of a number of clock cycles that the C2C serial link is in the first link state.

[0095] Clause 13. The apparatus of any of Clauses 11 to 12, wherein the means for sending the link state change information to the output pin comprises means for sending the link state change information to a GPIO pin.

[0096] Clause 14. An apparatus according to any one of Clauses 11 to 13, wherein the component for sending the link state change information in a serial manner to the output pin includes a component for sending at least one of the following: one or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; one or more bits indicating a link state; or one or more bits indicating a duration that the C2C serial link is in the link state.

[0097] Clause 15. An apparatus according to any one of clauses 11 to 14, wherein detecting the change in the link state of the C2C serial link includes detecting the change in the link state of the PCIE protocol link, and wherein storing information indicating the first link state includes storing information indicating a PCIE LTSSM state or sub-state.

[0098] Item 16. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by a device, cause the device to: detect a change in the link state of a C2C serial link from a first link state to a second link state; store link state change information in an on-die FIFO buffer, the link state change information including information indicating the first link state, the duration that the C2C serial link is in the first link state, and the speed of the C2C serial link in the first link state; detect a request for link state change information; retrieve link state change information from the FIFO buffer; and send the link state change information to an output pin in a serial manner.

[0099] Clause 17. A non-transitory computer-readable medium according to Clause 16, wherein the computer-executable instructions that, when executed by the device, cause the device to store information indicating the duration that the C2C serial link is in the first link state include computer-executable instructions that, when executed by the device, cause the device to store information indicating the number of clock cycles that the C2C serial link is in the first link state.

[0100] Clause 18. A non-transitory computer-readable medium according to any one of Clauses 16 to 17, wherein the computer executable instructions that, when executed by the device, cause the device to send the link status change information to the output pin include computer executable instructions that, when executed by the device, cause the device to send the link status change information to a GPIO pin.

[0101] Clause 19. A non-transitory computer-readable medium according to any one of Clauses 16 to 18, wherein the computer executable instructions that, when executed by the device, cause the device to send the link state change information in a serial manner to the output pin include computer executable instructions that, when executed by the device, cause the device to send at least one of the following: one or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; one or more bits indicating a link state; or one or more bits indicating a duration that the C2C serial link is in the link state.

[0102] Clause 20. A non-transitory computer-readable medium according to any one of clauses 16 to 19, wherein detecting the change in the link state of the C2C serial link includes detecting the change in the link state of the PCIE protocol link, and wherein storing information indicating the first link state includes storing information indicating a PCIE LTSSM state or sub-state.

[0103] Clause 21. An apparatus, comprising: a memory; a transceiver; and a processor, the processor being communicatively coupled to the memory and the transceiver, the memory, the transceiver, and the processor being configured to perform the method of any one of clauses 1 to 5.

[0104] Clause 22. An apparatus comprising means for performing the method of any one of clauses 1 to 5.

[0105] Clause 23. A non-transitory computer-readable medium storing computer-executable instructions, the computer-executable instructions comprising at least one instruction for causing a computer or a processor to perform the method according to any one of clauses 1 to 5.

[0106] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0107] In addition, it will be appreciated by those skilled in the art that the various exemplary logic blocks, modules, circuits, and algorithmic steps described in conjunction with the aspects disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, various exemplary components, frames, modules, circuits, and steps have been generally described in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints proposed for the entire system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such specific implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0108] The various illustrative logic blocks, modules, and circuits described in conjunction with the various aspects disclosed herein may be implemented or executed using a general purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0109] The methods, sequences and / or algorithms described in conjunction with the various aspects disclosed herein may be directly embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An example storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative solution, the storage medium may be integral with the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal (e.g., UE). In an alternative solution, the processor and the storage medium may reside in a user terminal as discrete components.

[0110] In one or more example aspects, the function can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the function can be stored on a computer-readable medium or sent by a computer-readable medium as one or more instructions or codes. Computer-readable media includes both computer storage media and communication media, and the communication media includes any medium that promotes the transmission of a computer program from one place to another. Storage media can be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if the software is sent from a website, a server or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwaves, then coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwaves are included in the definition of the medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0111] Although the foregoing disclosure shows the exemplary aspects of the present disclosure, it should be noted that various changes and modifications may be made herein without departing from the scope of the present disclosure as defined by the appended claims. In addition, the functions, steps and / or actions of the method claims according to the various aspects of the present disclosure described herein do not need to be performed in any particular order. In addition, although the elements of the present disclosure may be described or claimed in the singular, plural forms may also be considered unless explicitly stated to be limited to the singular.

Claims

1. A method for on-die monitoring of a chip-to-chip (C2C) serial link status, the method comprising: detecting a change in a link state of the C2C serial link from a first link state to a second link state; storing link state change information in an on-die first-in-first-out (FIFO) buffer, the link state change information comprising information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; detecting a request for link state change information; retrieving link state change information from the FIFO buffer; as well as The link status change information is sent to the output pin in a serial manner. 2 . The method of claim 1 , wherein storing information indicating the duration that the C2C serial link is in the first link state comprises storing information indicating a number of clock cycles that the C2C serial link is in the first link state. 3 . The method of claim 1 , wherein sending the link state change information to the output pin comprises sending the link state change information to a general purpose input / output (GPIO) pin.

4. The method of claim 1 , wherein sending the link state change information to the output pin in a serial manner comprises sending at least one of the following: One or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; One or more bits indicating link status; or One or more bits indicating a duration that the C2C serial link is in the link state.

5. The method of claim 1 , wherein detecting the change in the link state of the C2C serial link comprises detecting the change in the link state of a Peripheral Component Interconnect Express (PCIE) protocol link, and wherein storing information indicative of the first link state comprises storing information indicative of a PCIE Link Training Status and State Machine (LTSSM) state or sub-state.

6. An apparatus for on-die monitoring of chip-to-chip (C2C) serial link training status and link state, the apparatus comprising: A tube core, the tube core comprising: A C2C serial link circuit, the C2C serial link circuit comprising: a physical layer configured to send and receive data via a C2C serial link according to a C2C communication protocol; a serial link state machine (SLSM) configured to perform link training of the C2C serial link; and a SLSM monitor coupled to the SLSM and configured to: detecting a change in a link state of the C2C serial link from a first link state to a second link state; storing link state change information in an on-die first-in-first-out (FIFO) buffer, the link state change information comprising information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; detecting a request for link state change information; retrieving link state change information from the FIFO buffer; and The link status change information is sent to the output pin of the die in a serial manner.

7. The device according to claim 6, wherein: To store the information indicating the duration that the C2C serial link is in the first link state, the SLSM monitor is configured to store information indicating the number of clock cycles that the C2C serial link is in the first link state.

8. The device according to claim 6, wherein: To send the link status change information to the output pin of the die, the SLSM monitor is configured to send the link status change information to a general purpose input / output (GPIO) pin.

9. The device according to claim 6, wherein: To send the link state change information to the output pin in a serial manner, the SLSM monitor is configured to send at least one of the following: One or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; One or more bits indicating link status; or One or more bits indicating a duration that the C2C serial link is in the link state.

10. The apparatus of claim 6, wherein the C2C communication protocol comprises a Peripheral Component Interconnect Express (PCIE) protocol, and wherein the serial link state machine comprises a PCIE Link Training Status and Link State Machine (LTSSM), and wherein the information indicating the first link state comprises information indicating an LTSSM state or sub-state.

11. A device, comprising: means for detecting a change in a link state of a chip-to-chip (C2C) serial link from a first link state to a second link state; means for storing link state change information in an on-die first-in-first-out (FIFO) buffer, the link state change information comprising information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; means for detecting a request for link state change information; means for retrieving link state change information from said FIFO buffer; and A means for sending the link status change information to an output pin in a serial manner.

12. The apparatus of claim 11, wherein the means for storing the information indicating the duration that the C2C serial link is in the first link state comprises means for storing information indicating a number of clock cycles that the C2C serial link is in the first link state.

13. The apparatus of claim 11, wherein the means for sending the link state change information to the output pin comprises means for sending the link state change information to a general purpose input / output (GPIO) pin.

14. The apparatus of claim 11, wherein the means for sending the link state change information to the output pin in a serial manner comprises means for sending at least one of: One or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; One or more bits indicating link status; or One or more bits indicating a duration that the C2C serial link is in the link state.

15. The apparatus of claim 11, wherein detecting the change in the link state of the C2C serial link comprises detecting the change in the link state of a Peripheral Component Interconnect Express (PCIE) protocol link, and wherein storing information indicative of the first link state comprises storing information indicative of a PCIE Link Training Status and State Machine (LTSSM) state or sub-state.

16. A non-transitory computer-readable medium storing computer-executable instructions which, when executed by an apparatus, cause the apparatus to: detecting a change in a link state of a chip-to-chip (C2C) serial link from a first link state to a second link state; storing link state change information in an on-die first-in-first-out (FIFO) buffer, the link state change information comprising information indicating the first link state, a duration that the C2C serial link is in the first link state, and a speed of the C2C serial link in the first link state; detecting a request for link state change information; retrieving link state change information from the FIFO buffer; as well as The link status change information is sent to the output pin in a serial manner.

17. The non-transitory computer-readable medium of claim 16, wherein the computer-executable instructions that, when executed by the device, cause the device to store information indicating the duration that the C2C serial link is in the first link state include computer-executable instructions that, when executed by the device, cause the device to store information indicating a number of clock cycles that the C2C serial link is in the first link state.

18. The non-transitory computer-readable medium of claim 16, wherein the computer-executable instructions that, when executed by the device, cause the device to send the link state change information to the output pin include computer-executable instructions that, when executed by the device, cause the device to send the link state change information to a general purpose input / output (GPIO) pin.

19. The non-transitory computer-readable medium of claim 16, wherein the computer-executable instructions that, when executed by the device, cause the device to send the link state change information to the output pin in a serial manner include computer-executable instructions that, when executed by the device, cause the device to send at least one of: One or more bits indicating that the link state change information is valid; one or more bits indicating a core clock rate; one or more bits indicating a status of the link state change information; One or more bits indicating link status; or One or more bits indicating a duration that the C2C serial link is in the link state.

20. The non-transitory computer-readable medium of claim 16, wherein detecting the change in the link state of the C2C serial link comprises detecting the change in the link state of a Peripheral Component Interconnect Express (PCIE) protocol link, and wherein storing information indicative of the first link state comprises storing information indicative of a PCIE Link Training Status and State Machine (LTSSM) state or sub-state.

Images