Method and system for sending in-band cross-chip triggers to maintain high speed interconnects

Abstract

The present disclosure relates to methods and systems for sending in-band cross-chip triggers to preserve high-speed interconnects. A system includes a link having one or more data paths and a device coupled with the link, the device including a data link (DL) transmitter and a buffer. The device, in response to an indication, writes one or more bits corresponding to an operation to a first portion of a data frame, the data frame including a second portion, the second portion including data. In response to writing the one or more bits corresponding to the operation, the device transmits the first portion and the second portion of the data frame via the one or more data paths. In response to writing the one or more bits corresponding to the operation, the device stores the data frame at the buffer.

Description

[0001] Related applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 294,041, filed December 27, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0002] At least one embodiment relates to processing resources for performing and facilitating high-speed communication. For example, at least one embodiment relates to techniques for transmitting in-band cross-chip triggers to maintain and debug high-speed interconnects. Background Technology

[0003] Communication systems transmit signals from a transmitter to a receiver via a communication channel or medium (e.g., cable, printed circuit board, link, wireless, etc.). In some communication systems, errors can occur when transmitting signals from the transmitter to the receiver. Therefore, communication systems can assess the quality of the communication channel and perform debugging operations to ensure that the communication channel reliably transmits data. In traditional communication systems, devices can send cross-chip triggers (e.g., instructions to perform debugging operations) via sidebands—for example, via a communication channel or medium unrelated to the transmitted data. For example, a device can send a cross-chip trigger via a general-purpose pin dedicated to transmitting the trigger. However, general-purpose pins do not maintain timing with the data transmitted via the communication channel—for example, cross-chip triggers can be transmitted asynchronously relative to the data. This can increase latency and lead to inaccurate debugging operations. Attached Figure Description

[0004] Various embodiments according to this disclosure will be described with reference to the accompanying drawings, in which: Figure 1 These are examples of communication systems according to at least some embodiments; Figure 2 Example frames of communication in a communication system according to at least some embodiments are shown; Figure 3 This is an example communication system that transmits in-band cross-chip triggering according to at least some embodiments; Figure 4 This is an example communication system that transmits in-band cross-chip triggering according to at least some embodiments; Figure 5 This is a flowchart of a method for transmitting in-band cross-chip triggering in a high-speed interconnect system according to at least some embodiments; Figure 6 It is an exemplary computer system including a transceiver according to at least some embodiments, the transceiver including chip interconnects for transmitting in-band cross-chip triggered interconnects. Detailed Implementation

[0005] A communication system transmits signals from a transmitter to a receiver via a communication channel or medium (e.g., cable, printed circuit board, link, wireless, etc.). For example, a communication system may include a first device (e.g., a first integrated circuit (IC) or chip) and a second device (e.g., a second IC or chip) via a communication link—for example, the communication system may be a chip-to-chip (C2C) interconnect, where both devices include a transmitter and a receiver. In some communication systems, errors may occur when data is transmitted from the first device to the second device via a link. The communication system can perform debugging operations or evaluate the quality of the link. If a problem is found in the link, the communication system can further perform debugging operations to ensure that data is communicated reliably. For example, the first device may send a cross-chip trigger to the second device to evaluate the quality of the link. Cross-chip triggers allow debugging operations to occur simultaneously on both the first and second devices to assess where errors may occur. For example, the first device may send data frames to the second device, and the respective software stacks of the first and second devices can compare the data frames sent at the first device and the data frames received at the second device to evaluate the quality of the link. Conventional communication systems may send cross-chip triggers along sidebands—e.g., across communication channels or media unrelated to the data transmission. For example, a communication system may include links for transmitting data and dedicated general-purpose pins on each device for transmitting cross-chip triggers. However, each device may maintain the timing of these general-purpose pins independently of the timing of data transmission. Therefore, any one device can send or receive data and cross-chip triggers asynchronously—for example, a device may receive a data frame for debugging operations before or after receiving a corresponding cross-chip trigger. For example, a device may receive a cross-chip trigger hundreds of nanoseconds before or after the data associated with it. Asynchronous cross-chip triggers can increase latency or lead to inaccurate debugging operations. For example, a device may receive a cross-chip trigger first and then have to wait to receive data, resulting in increased latency. In some communication systems, a device may receive a first data frame corresponding to a cross-chip trigger, then another cross-chip trigger, and then a second data frame, and incorrectly perform debugging operations with the second data frame because it was received after the cross-chip trigger.

[0006] Advantageously, various aspects of this disclosure can address the aforementioned shortcomings and other challenges by providing a method and system for transmitting in-band cross-chip triggers. For example, a first device in a communication system can use cross-chip triggering logic to package cross-chip triggers within a data frame at the data link layer of a transmitter. In response to receiving a debug trigger (e.g., a trigger from the respective software stack of the device), in response to determining an internal trigger (e.g., data received at the data link layer of the transmitter from the transaction layer indicating the generation of a cross-chip trigger), or in response to an external trigger—e.g., a trigger from hardware outside the transmitter (e.g., error or debug circuitry)—the first device can package the cross-chip trigger into the data frame. After packaging the cross-chip trigger into the data frame, the device can transmit the data frame with the cross-chip trigger across the link (e.g., in-band) and store the data frame in a buffer at the data link layer of the transmitter.

[0007] In some embodiments, the second device may receive data frames at the data link layer of the receiver, decode the data frames, and determine that the data frames include cross-chip triggers—for example, identifying cross-chip triggers in the data frames. In such embodiments, the second device may store the data frames in a buffer at the data link layer of the receiver. The software stack of the second device may communicate with the software stack of the first device after the data frames with cross-chip triggers are stored in the buffer—for example, the software stack of the second device may read the data frames with cross-chip triggers. For example, the software stack of the second device may indicate pass or fail (e.g., the data frames with cross-chip triggers were received correctly or incorrectly), or send a message indicating the data content of the data frames with cross-chip triggers (e.g., via CRC signature). The software stack of the first device and the software of the second device may communicate via a second link—for example, a sideband link unrelated to the data transmission. Therefore, the communication system may perform debugging operations by transmitting cross-chip triggers in-band from the first device to the second device.

[0008] In some embodiments, a second device may receive a data frame with cross-chip triggering and send the data frame with cross-chip triggering back to a first device—for example, decoding the data frame with cross-chip triggering to generate a second data frame with cross-chip triggering, which is a copy of the data frame with cross-chip triggering, and sending the second data frame back to the first device. The first device may receive the second data frame with cross-chip triggering at the data link layer of a receiver. The data link layer of the receiver of the first device may decode the second data frame and determine that the second data frame includes cross-chip triggering. In such embodiments, the first device may store the second data frame in a buffer at the data link layer of the receiver. In some embodiments, the controller of the first device (e.g., a microcontroller or a finite state machine (FSM)) may read the data frame stored in the transmitter's buffer and read the second data frame stored in the receiver's buffer, compare the data frame and the second data frame, and perform debugging operations based on the comparison. For example, when the data frame differs from the second data frame, the controller may determine one or more errors and transmit an indication to the device's software stack. In such an example, the software stack may perform debugging operations in response to receiving the indication—for example, resetting the link based on determining one or more errors. In some embodiments, the software stack of the first device can read the data frame and the second data frame, perform a comparison, and perform debugging operations accordingly.

[0009] By utilizing in-band cross-chip triggers, communication systems can perform debugging operations more efficiently. For example, cross-chip triggers can be synchronized with data because both are included in the same data frame during cross-link transmission. Furthermore, in some embodiments, cross-chip triggers in data frames do not disrupt normal operation because they can be included in the data frame to be transmitted—e.g., a data frame in the device's data pipeline. Therefore, embodiments of this application allow for a more reliable method of transmitting in-band cross-chip triggers to maintain and debug high-speed interconnects.

[0010] Figure 1 An exemplary communication system 100 according to at least one exemplary embodiment is illustrated. The system 100 includes a host 102-a, a host 102-b, a first device 104-a, and a second device 104-b. The system 100 also includes a link 106 coupling the first device 104-a and the second device 104-b. Each device 104 may include a transceiver 125, which includes a transmitter 130, a receiver 135, a digital data source 140, and processing circuitry 145. Each transmitter 130 may include a transaction layer (TL) 108, a data link layer (DL) 110, and a physical layer (PL) 112, and each receiver 135 may include a TL 114, a DL 116, and a PL 118.

[0011] In at least one example, host 102 or device 104 may correspond to one or more of a personal computer (PC), laptop, tablet, smartphone, server, collection of servers, etc. In some examples, host 102 may correspond to any suitable type of device that communicates with other devices and is also connected to the public link 106. In some examples, host 102 may transmit commands or data to device 104. In such examples, devices 104 may communicate data with each other based on commands or data received by host 102. As another specific but non-limiting example, host 102 and device 104 may correspond to servers that provide information resources, services, and / or applications to user devices, client devices, or other hosts in system 100.

[0012] In at least one exemplary embodiment, the first device 104-a and the second device 104-b may be examples of chips—for example, system 100 may be an example of a multi-chip module or a chip-to-chip (C2C) interconnect. In such examples, device 104 may be a single chip or a stack of chips. In some examples, device 104 may include a graphics processing unit (GPU), a switch (e.g., a high-speed network switch), a network adapter, a central processing unit (CPU), etc., to execute commands or functions received from host 102. Each device 104 may include a transceiver 125 for transmitting and receiving signals, such as data signals. The data signals may be digital signals modulated with data or optical signals, or other signals suitable for carrying data. Each transceiver 125 may include a receiver 135 and a transmitter 130. Transmitter 130 includes suitable software and / or hardware for receiving digital data from digital data source 140 and outputting data signals based on the digital data for transmission via link 106 to receiver 135 of device 104-b. Device 104 of devices 104-a and 104-b may include suitable hardware and / or software for receiving signals, such as data signals from link 106.

[0013] In embodiments, device 104 can communicate bidirectionally—for example, from host 102-a to host 102-b or from host 102-b to host 102-a. In some examples, each receiver 135 or transmitter 130 of device 104 can operate independently and / or simultaneously. For example, receiver 135-a of first device 104-a can receive data from transmitter 130-b of second device 104-b, while transmitter 130-a of first device 104-a transmits data to receiver 135-b of second device 104-b.

[0014] Each transmitter 130 and receiver 135 in device 104 may include a transaction layer (TL). In some examples, TL 108 of transmitter 130 may be configured to request a transaction—for example, request the transmission of data. For example, TL 108 of transmitter 130 may communicate functionality with other components of device 104 or assemble data packets. In some examples, TL 108 of transmitter 130 may generate transaction layer packets (TLPs) that can be transmitted to DL 110 for further processing. In some examples, each receiver 135 in device 104 may also include a transaction layer 114. In some examples, TL 114 of receiver 135 may be configured to complete a transaction—for example, complete the transmission of data. For example, TL 114 of receiver 135 may receive functionality from other components of the receiver in each device 104 or decompose received data packets. In some embodiments, TL 114 of receiver 135 may validate incoming TLP data packets to ensure that the received packets are valid—for example, without errors.

[0015] Each transmitter 130 and receiver 135 in device 104 may also include a data link layer. In some examples, DL 110 and DL 116 may be configured to ensure that data transmitted over link 106 is correct and error-free. For example, DL 110 of transmitter 130 may assign error codes (e.g., CRC values) to its respective transmitted frames or packets. DL 116 of transceiver 125 may generate error codes based on received frames and decode the CRC embedded in the frame to compare the generated error codes with the transmitted CRC. In some examples, DL 116 performs error decoding to check if the received data is correct and error-free. In some examples, DL 110 of transmitter 130 may be configured to add a sequence number as a header to each transmitted frame or packet, and DL 116 of receiver 135 may be configured to also check the sequence number. In some examples, the DL 110 of transmitter 130 and the DL 116 of receiver 135 may include or be coupled to a controller or control flow unit to perform error decoding operations on received packets or frames—for example, processing circuitry 145.

[0016] Furthermore, each transmitter 130 and receiver 135 in device 104 may include a physical layer (PL). In some examples, PL 112 and PL 118 may be configured to transmit and receive data on link 106. For example, PL 112 and PL 118 may include input / output (I / O) buffers, parallel-to-serial and serial-to-parallel converters, impedance matching circuitry, logic circuitry, etc., to transmit and receive data packets or frames across link 106.

[0017] Each transceiver 125 may include a digital data source 140 and processing circuitry 145 for controlling the transceiver 125. The digital data source 140 may include suitable hardware and / or software for outputting data in digital format (e.g., binary code and / or thermometer code). The digital data output by the digital data source 140 may be retrieved from memory (not shown) or generated based on input (e.g., user input).

[0018] Processing circuitry 145 may include software, hardware, or a combination thereof. For example, processing circuitry 145 may include memory (which contains executable instructions) and a processor (e.g., a microprocessor) that executes the instructions on the memory. The memory may correspond to any suitable type of storage device or collection of storage devices configured to store instructions. Non-limiting examples of suitable storage devices that may be used include flash memory, random access memory (RAM), read-only memory (ROM), variations thereof, combinations thereof, or the like. In some embodiments, the memory and processor may be integrated into a common device (e.g., a microprocessor may include integrated memory). Additionally or alternatively, processing circuitry 145 may include hardware such as application-specific integrated circuits (ASICs). Other non-limiting examples of processing circuitry 132 include integrated circuit (IC) chips, central processing units (CPUs), general-purpose processing units (GPUs), microprocessors, field-programmable gate arrays (FPGAs), logic gates or collections of transistors, resistors, capacitors, inductors, diodes, or the like. Some or all of processing circuitry 145 may be provided on a printed circuit board (PCB) or a collection of PCBs. It should be understood that any suitable type of electrical component or collection of electrical components may be adapted to be incorporated into processing circuitry 145. The processing circuitry 145 can send and / or receive signals from other components of the transceiver 125 to control the overall operation of the transceiver 125.

[0019] Transceiver 125 or selected elements thereof may take the form of a pluggable card or controller of device 104. For example, transceiver 125 or selected elements thereof may be implemented on a network interface card (NIC).

[0020] Link 106 may be an example of a communication network that can be used to connect devices 104, such as an Internet Protocol (IP) network, an Ethernet network, an InfiniBand (IB) network, a Fibre Channel network, the Internet, a cellular communication network, a wireless communication network, a combination thereof (e.g., Fibre Channel over Ethernet), a Peripheral Component Interconnect Fast (PCIe), variations thereof, and / or the like. In a specific but non-limiting example, link 106 is a network that uses data signals (e.g., digital, optical, wireless signals) to enable data transmission between devices 104.

[0021] In embodiments, link 106 may be configured to transmit requests, data, functions, commands, etc., between first device 104-a and second device 104-b. In examples, link 106 may be a cable, printed circuit board, link, wireless, etc. In at least one embodiment, link 106 may be an example of a Ground Reference Signaling (GRS) interconnect. In such an example, link 106 may include an RC-dominant channel and an LC transmit line. Furthermore, GRS interconnects may be on-chip links, links across a substrate (such as an organic package), or link signaling across a printed circuit board (PCB). In some examples, GRS may use a ground network as a signal reference voltage—for example, ground may be return signaling. Although not explicitly shown, it should be understood that host 102 and device 104 may include other processing devices, storage devices, and / or communication interfaces typically associated with computing tasks such as sending and receiving data.

[0022] In some instances, each DL 110 of transmitter 130 and each DL 116 of receiver 135 includes chip triggering logic 150. In some embodiments, the chip triggering logic 150 of DL 110 of transmitter 130 is configured to package cross-chip triggers (e.g., indications of debug operations) in data frames. In such embodiments, the chip triggering logic 150 of DL 110 of transmitter 130 may store data frames with cross-chip triggers in a buffer and transmit the cross-chip triggers via link 106—for example, DL 110-a of transmitter 130-a may package cross-chip triggers in data frames and transmit the data frames to device 104-b via link 105. In some embodiments, the chip triggering logic 150 of DL 110 of transmitter 130 may package cross-chip triggers in response to receiving a debug trigger, receiving an external trigger, or generating an internal trigger, as referenced. Figure 3 and Figure 4 As described. In at least one embodiment, the cross-chip trigger 150 of the DL 116 of receiver 135 is configured to receive a data frame, decode the data frame, and determine whether the data frame includes a cross-chip trigger. In some embodiments, the DL 116 of receiver 135 is configured to store a data frame including a cross-chip trigger. In some embodiments, the cross-chip trigger can be sent from the DL 110 of the first device 104-a to the DL 116 of the second device 104-b, as described in reference... Figure 3 As described. In some embodiments, cross-chip triggering can be sent from DL 110 of the first device to DL 116 of the first device via the second device 104-b, as referenced. Figure 4 As described. In either case, the communication system 100 can send cross-chip triggers in-band—for example, via link 106 associated with data communication between the first device 104-b and the second device 104-b.

[0023] Figure 2 As shown in the reference Figure 1 Example frame 200 of communication in the described communication system 100. For example, frame 200 may be transmitted by transmitter 130 to receiver 135, from device 104-a to 104-b or from device 104-b to device 104-a. In embodiments, frame 200 may include “N” flits 202. For example, a given frame 200 may include ten (10) flits 202. In some examples, each flit 202 may include the same number of bits, for example, each flit 202 is “X” bit wide. For example, each flit 202 may be 128 bits wide. In some examples, DL 110 of transmitter 130 may transmit one (1) flit per clock cycle. Thus, each frame may be transmitted over “N” clock cycles based on the number of “N” flits 202. Furthermore, each frame 200 may include an error code CRC 208. DL 110 is configured to generate CRC 208 for the entire frame. In such an embodiment, the DL 116 of receiver 135 is configured to perform an error decoding operation on “N” micro-pieces 202 of each frame 200—for example, interrupt logic 150 is configured to perform the error decoding operation after receiving “N” micro-pieces 202 corresponding to the size or width of frame 200. That is, the error decoding operation is performed at the frame granularity—for example, the frame error rate (FER) is determined. In some embodiments, frame 200 may include one (1) micro-piece 202. In such an embodiment, the error decoding operation can be performed substantially at the micro-piece granularity because each frame 200 includes one (1) micro-piece 202. In some embodiments, the DL 110 of transmitter 130 may generate different error detection codes—for example, error detection codes other than CRC. For example, the DL 110 of transmitter 130 may generate parity or checksum. In either example, the error detection code selected by transmitter 130 may be embedded within frame 200 and transmitted to the DL 116 of receiver 135.

[0024] Each frame 200 may also include a header 204. In some embodiments, the header 204 may include a trigger 212 (e.g., cross-chip trigger) and other fields 210 (e.g., information associated with the frame 200). In some embodiments, the trigger 212 may be a reserved field for encoding trigger information. In at least one embodiment, the chip triggering logic 150 of the DL 110 may write one or more bits to the trigger 212 to indicate cross-chip triggering. That is, the trigger 212 may be a multi-bit field indicating the trigger type of the embedded data frame 200. For example, the trigger 212 may be a two (2) bit field indicating debug triggering, external triggering, internal triggering, or no triggering, as referenced. Figure 3 and Figure 4 As described. In other embodiments, trigger 212 may indicate system-level debugging operations or operations to evaluate the quality of link 106. Therefore, chip triggering logic 150 may write one or more bits to trigger 212 to indicate the type of trigger (or lack thereof) within data frame 200. In at least one embodiment, chip triggering logic 150 of DL 116 may receive an incoming data frame 200, decode header 204, and determine whether the data frame 200 includes a cross-chip trigger—for example, determining whether one or more bits of trigger 212 indicate a cross-chip trigger. By writing one or more bits to trigger 212, communication system 100 can transmit cross-chip triggers in-band. For example, chip triggering logic 150 writes a cross-chip trigger in trigger 212 that is included in data frame 200 transmitted between device 104-a and device 104-b via link 106.

[0025] Figure 3 An exemplary communication system 300 according to at least one exemplary embodiment is illustrated. In some embodiments, the communication system 300 may be a reference... Figure 1 An example of the described communication system 100. For example, system 300 may include references Figure 1 The description includes host 102-a, host 102-b, first device 104-a, and second device 104-b. In at least one embodiment, system 100 further includes a link 106 coupling the first device 104-a and the second device 104-b. Each device 104 may include a transaction layer (TL) 108, a data link layer (DL) 110, and a transmitter (e.g., reference 104-b). Figure 1 The physical layer (PL) 112 associated with the transmitter 130 described herein. Each device 104 may include a receiver (e.g., as described in reference 130). Figure 1The receiver 135 described relates to TL114, DL116, and PL118. In at least one embodiment, the transmitter's DL110 may include data pipeline 305-a, control 310-a, and buffer 315-a—for example, although not illustrated, the transmitter's DL110-b of device 104-b may also include data pipeline 305-a, control 310-a, and buffer 315-a. In some embodiments, the receiver's DL116 may include data pipeline 305-b, control 310-b, and buffer 315-b—for example, although not illustrated, the receiver's DL116-a of device 104-a may also include data pipeline 305-b, control 310-b, and buffer 315-b. In one embodiment, the communication system 300 may illustrate an example of transmitting data frames, including cross-chip triggering, from device 104-a to device 104-b for debug operations. Although not shown, device 104-b can also send cross-chip triggers from device 104-a for debugging operations.

[0026] In at least one embodiment, host 102-a is configured to communicate data with host 102-b. In such an embodiment, host 102-a may send data packets 316, which include data to be communicated to device 104-a (e.g., to transmitter TL108-a).

[0027] In some embodiments, the transmitter's TL 108-a is configured to receive data packets 316 from the host 102-a. In at least one embodiment, the transmitter's TL 108-a can assemble the data packets 316 received from the host 102-a into a data frame 318—for example, referring to... Figure 2An example of the described data frame 200. In at least one embodiment, a data packet 316 received from host 102-a may indicate that the data packet 316 is associated with a cross-chip trigger. In such an embodiment, transmitter TL 108-a may assemble data frame 318 and indicate that the data frame is associated with a cross-chip trigger. In some embodiments, the software stack of device 104-a may be programmed to TL 108-a to generate an indication for a specific data packet 316 (e.g., cross-chip trigger). In such an embodiment, transmitter TL 108-a may compare a first set of bits received in each data packet 316 with a specified data packet. When the first set of bits in the data packet 316 received from host 102-a matches (e.g., satisfies) the specified data packet, TL 108-a may generate an indication of a cross-chip trigger when assembling data frame 318. In at least one embodiment, transmitter TL 108-a may receive a stall signal 320 from the software stack, firmware, or controller (e.g., a finite state machine (FSM)) of device 104-a. In such an embodiment, the hold signal 320 may instruct TL 108-a to hold link 106 until the debugging operation is complete—for example, to hold link 106 until a second hold signal 320 instructing the resumption of operation on link 106 is received. When TL 108-a receives hold signal 320, the transmitter's TL 108-a is configured not to assemble data packets 316 received from host 102-a—for example, TL 108-a may suspend operation. When TL 108-a receives the second hold signal 320, the transmitter's TL 108-a may resume operation—for example, to resume assembling data packets 316 received from host 102-a. That is, in some embodiments, the software stack of device 104-a may perform debugging operations using data that has already been transmitted, thus avoiding holding link 106. In other embodiments, the software stack of device 104-a can perform debugging operations via lingering link 106 and utilize custom data (e.g., data generated by the software stack of device 104-a) for debugging operations, as described in reference data pipeline 305-a.

[0028] In at least one embodiment, data pipeline 305-a is configured to receive data frame 318 from transmitter TL 108-a. In some embodiments, data pipeline 305-a is configured to sample and decode data frame 318. In embodiments where TL 108-a assembles data frame 318 with an indication that the data frame is associated with a cross-chip trigger, data pipeline 305-a (e.g., a component of data pipeline 305-a) can identify the cross-chip trigger while sampling and decoding data frame 318. In such embodiments, data pipeline 305-a can write one or more bits into a trigger field indicating a cross-chip trigger (e.g., reference...). Figure 2 The described trigger 212 generates a data frame 340 that includes a cross-chip trigger. In some embodiments, a cross-chip trigger indicated by host 102-a (e.g., an indication in data frame 318) may be associated with system-level debugging. In such embodiments, data pipeline 305-a may write one or more bits to a trigger field to indicate a system-level debugging operation. In some embodiments, data pipeline 305-a may send data frame 340 to transmitter PL 112-a. When data pipeline 305-a sends data frame 340 (e.g., a data frame with a cross-chip trigger), data pipeline 305 may send trigger 360 to control 310-a, indicating that the data frame includes a cross-chip trigger.

[0029] In some embodiments, data pipeline 305-a is configured to receive debug trigger 325 from the software stack of device 104-a (e.g., from firmware). In at least one embodiment, the software stack may send debug trigger 325 to data pipeline 305-a to mark an incoming data frame 318 as a cross-chip triggered data frame—for example, debug trigger 325 may be a bit indicating that the received data frame 318 is a cross-chip triggered data frame. In such an embodiment, data pipeline 305-a may write one or more bits to the trigger field of the received data frame 318 to generate a data frame 340 with cross-chip triggering. Furthermore, in such an embodiment, the software stack of device 104-a may not cause link 106 to stall—for example, by not sending stall signal 320. That is, the software stack of device 104-a may perform debugging operations using data that is already being transmitted (e.g., already transmitted by host 102-a). Accordingly, debugging operations may occur in parallel with the data being transmitted, and the performance of communication system 300 is not degraded.

[0030] In at least one embodiment, the software stack of device 104-a can send a debug trigger 325 and a hangover signal 320. In such an embodiment, the software stack of device 104-a can generate data vectors transmitted from device 104-a to device 104-b to evaluate the quality of link 106. For example, the software stack of device 104-a can generate data vectors to test various pins of link 106 (e.g., data vectors designed to test a first pin of link 106 or data vectors designed to test a set of pins of link 106), generate data vectors to test data channels of link 106 (e.g., data vectors designed to test a first data channel of link 106), generate data vectors to test the transition of each pin of link 106, and so on. For example, the software stack of device 104-a can generate data vectors that send different data on each data channel to test whether each data channel of link 106 reliably communicates data without errors. When the software stack of device 104-a generates a data vector, the software stack can first send the data vector to TL 108-a, or send the data vector directly to data pipeline 305-a. For example, the software stack can send the generated data vector to data pipeline 305-a by debugging the 325 interface.

[0031] In at least one embodiment, the software stack of device 104-a can program data pipeline 305-a to generate a trigger 360 (e.g., an internal trigger) for a specific data frame 318. In such an embodiment, data pipeline 305-a can compare a first set of bits received in each data frame 318 (e.g., the first few bits of chip 202-a) with a specified data frame. When the first set of bits received from data frame 318 from transmitter TL 108-a matches (e.g., satisfies) the specified data frame, data pipeline 305-a can write one or more bits into a trigger field to generate data frame 340. In such an embodiment, data pipeline 305-a can further generate a trigger 360 (e.g., an internal trigger) and send the trigger 360 to control 310-a. In some embodiments, data pipeline 305-a can generate trigger 360 based on other conditions programmed into the software stack of device 104-a. For example, the software stack can be programmed to generate a trigger 360 for periodically evaluating the quality of link 106—for example, at a specified time period (e.g., after one hour) or after receiving a specified number of data frames 318 (e.g., after receiving five (5) data frames 318). In such an embodiment, data pipeline 305-a can also write one or more bits to the trigger field of data frame 318 to generate a data frame 340 with a cross-chip trigger and send data frame 340 to PL 112-a of the transmitter. In some embodiments, data pipeline 305-a is configured to generate an internal trigger 360 when replaying a data frame 340 that has been corrupted by link 106. For example, if a data frame 340 including a cross-chip trigger is corrupted by link 106 (e.g., fails to be properly received at device 104-b), data pipeline 305-a can replay the corrupted data frame 340. In such an embodiment, data pipeline 305-a can regenerate data frame 340 and generate internal trigger 360 to instruct control 310-a that the replayed data frame 340 includes a cross-chip trigger.

[0032] In some embodiments, data pipeline 305-a can be configured to identify no cross-chip trigger associated with the incoming data frame 318. In such embodiments, data pipeline 305-a can avoid generating trigger 360. Furthermore, data pipeline 305-a can write one or more bits to the trigger field to indicate that no cross-chip trigger is associated with the data frame 318 and send the frame to PL 112-a of the transmitter. In some embodiments, data pipeline 305-a may not write any bits to the trigger field—for example, the data frame 318 assembled by TL 108-a can already indicate no cross-chip trigger.

[0033] In at least one embodiment, control 310-a (e.g., a controller or FSM) is configured to receive trigger 360 from data pipeline 305-a. In at least one embodiment, control 310-a may generate a capture 365 command in response to receiving trigger 360. In some embodiments, control 310-a may generate a capture 365 command to instruct buffer 315-a to store data frames 340 transmitted by data pipeline 305-a—for example, control 310-a may generate the capture 365 command each time a data frame includes a cross-chip trigger. In at least one embodiment, control 310-a may be configured to receive external trigger 330. In some embodiments, control 310-a may receive external trigger 330 from peer blocks or hardware components not coupled to link 106 (e.g., hardware components of transceivers not coupled to device 104-a or device 104-b). For example, control 310-a may receive external trigger 330 from debug logic or error circuitry. In at least one embodiment, external trigger 330 can indicate to control 310-a a cross-chip trigger for data frame 318 received at data pipeline 305-a. In such an embodiment, control 310-a can send a command to data pipeline 305-a to write one or more bits into the trigger field of the incoming or received data frame 318—for example, to write one or more bits to indicate that the data frame 318 is associated with an external cross-chip trigger. In such an embodiment, data pipeline 305-a can write one or more bits to generate a data frame 340 indicating an external cross-chip trigger and send the data frame 340. Furthermore, control 310-a can generate a capture 365 command and transmit the capture 365 command to buffer 315-a.

[0034] In at least one embodiment, in response to receiving a capture 365 command from control 310-a, buffer 315-a is configured to store data frames 340 transmitted by data pipeline 305-a—for example, buffer 315-a is configured to store each data frame 340 including cross-chip triggers. In at least one embodiment, transmitter DL 110 may include more than one buffer 315-a—for example, DL 110 may include an arrangement of buffers 315-a configured to store data frames 340 including cross-chip triggers. In some embodiments, the size of buffer 315-a (e.g., the amount of data stored in buffer 315) may depend on the error magnitude of device 104-a. For example, if data pipeline 305-a accurately marks data frames 318 as cross-chip triggers, buffer 315-a may be shallow (e.g., have a smaller size)—for example, buffer 315-a may only store data frames 340 including cross-chip triggers when they are accurately determined. In at least one embodiment, the software stack of device 104-a can read data frame 340 stored in buffer 315-a, as described below. In at least one embodiment, buffer 315-a can be physically close to the transmitter's PL 112-a. Accordingly, the respective software stack of device 104-a can determine whether an error that has occurred is associated with link 106 (e.g., the physical layer) or whether the error occurred elsewhere.

[0035] In at least one embodiment, the transmitter's PL 112-a is configured to transmit the data frame 340 received across link 106 to device 104-b. In at least one embodiment, the receiver's PL 118-b of device 104-b is configured to receive the data frame 340 and transmit it to the receiver's DL 116-b. In at least one embodiment, link 106 is a reference... Figure 1 An example of a GRS link is described. In other embodiments, link 106 may be an example of an Ethernet network or a standardized interconnect (e.g., PCIe).

[0036] In some embodiments, the data pipeline 305-b of the receiver at device 104-b is configured to receive data frame 340 from PL118-b of the receiver. In at least one embodiment, the data pipeline 305-b is configured to sample and decode the received data frame 340. In at least one embodiment, the data pipeline 305-b is configured to identify cross-chip triggers when decoding the trigger field of the data frame 340. For example, the data pipeline 305-b may decode the trigger field and identify no trigger, external trigger, internal trigger, debug trigger, etc. In at least one embodiment, if the data pipeline 305-b determines that the data frame 340 includes a cross-chip trigger, the data pipeline 305-b may generate a trigger 350. For example, the data pipeline 305-b may send the trigger 350 to control 310-b to indicate that the received data frame 340 includes a cross-chip trigger.

[0037] In some embodiments, data pipeline 305-b is configured to perform error detection (e.g., error correction) operations on received data frames 340. For example, when data frame 340 includes a reference... Figure 2When performing the CRC 208 described, data pipeline 305-b can perform a Cyclic Redundancy Check (CRC) error operation. In at least one embodiment, data pipeline 305-a can decode (e.g., unpack) the trigger field before or after the CRC operation. In at least one embodiment, data pipeline 305-b can unpack the trigger field before performing an error detection operation. For example, if a debugging operation is associated with evaluating the quality of link 106, data pipeline 305-b can be configured to unpack the trigger field before performing an error detection operation (e.g., storing data frame 340 in buffer 315-b). That is, a debugging operation can be performed to determine the number of errors caused by link 106 (e.g., the number of flipped bits). Accordingly, data pipeline 305-b can store data frame 340 in buffer 315-b without performing an error detection operation to see the number of errors introduced by link 106. In at least one embodiment, data pipeline 305-b can be configured to unpack the trigger field after performing an error detection operation. For example, if the debugging operation is associated with system-level debugging, data pipeline 305-b may first perform an error detection operation and then unpack the trigger field. That is, a system-level debugging operation can be performed to determine whether a data frame was correctly sent from device 104-a to device 104-b, and evaluating the success of the error detection operation can be part of the system-level debugging operation. In some embodiments, data pipeline 305-b can be programmed by the software stack to unpack the trigger field before or after the error detection operation. In such an embodiment, the software stack of the device sending the data frame 340 with the cross-chip trigger can communicate with the software stacks of other devices to program data pipeline 305 to the correct configuration. For example, the software stack of device 104-a can communicate with device 104-b for a debugging operation to evaluate the quality of the link. In such an embodiment, the software stack of device 104-b can program data pipeline 305-b to perform the debugging operation after unpacking the trigger field. Similarly, the software stack of device 104-b can be programmed to perform debugging operations before the unpacking trigger field when the software stack of device 104-a instructs system-level debugging. In some embodiments, the software stack of device 104-a can communicate with the software stack of device 104-b via a sideband (e.g., a communication channel or link other than link 106).

[0038] In at least one embodiment, control 310-b (e.g., a controller or FSM) is configured to receive trigger 350 from data pipeline 305-b. In at least one embodiment, control 310-b may generate a capture 365 command in response to receiving trigger 350. In some embodiments, control 310-b may generate a capture 365 command to instruct buffer 315-b to store data frames 340 received by data pipeline 305-b—for example, control 310-b may generate the capture 365 command each time a data frame including a cross-chip trigger is received. In such an embodiment, buffer 315-b is configured to store received data frames 340 including cross-chip triggers.

[0039] In at least one embodiment, the software stacks of device 104-a and device 104-b are configured to perform debugging operations after data frame 340 has been stored in both buffers 315-a and 315-b (e.g., after a cross-chip trigger is sent and received). For example, the software stack of device 104-a can perform a read 335 from buffer 315-a via an interface, and the software stack of device 104-b can perform 335 from device 104-b. In some embodiments, the software stacks of device 104-a and device 104-b can communicate the results to complete the debugging operation. For example, the software stack of device 104-b can hash all entries in buffer 315-b and send the signatures corresponding to the entries to the software stack of device 104-a via a sideband link or communication channel—for example, since link 106 may be associated with errors, the software stack can avoid communicating via link 106. In such an embodiment, the software stack of device 104-a can compare entries in buffer 315-a with signatures (e.g., entries in buffer 315-b). Therefore, the software stack of device 104-a can evaluate the quality of link 106 or perform system-level debugging. For example, the software stack of device 104-a can determine the number of errors by comparing entries in buffer 315-a with signatures (e.g., entries in buffer 315-b). If the number of errors exceeds a threshold number of errors (e.g., the maximum number of errors associated with reliable link 106), the software of device 104-a can reset the link, retrain the link, or perform other operations to reduce the number of errors. In some embodiments, the number of errors can be associated with a count of bit errors or a count of chip errors. In other embodiments, the software stack of device 104-a can determine errors associated with a pin (e.g., a single pin or a group of pins) or a pin group of link 106 based on generating a data vector and comparing the data vector received at buffer 315-b. In at least one embodiment, the software stack of device 104-a can determine how far the data of interest has traveled in the system—for example, determining whether the data associated with a cross-chip trigger is stored in buffer 315-a or buffer 315-b. In some embodiments, the software stack of device 104-a or device 104-b can perform debugging operations at startup or runtime. In at least one embodiment, the software stack of device 104-a can send a second lingering signal 320 to TL 108-a after the debugging operation is completed—for example, the software stack of device 104-a can resume operation of link 106.

[0040] In at least one embodiment, the software stacks of devices 104-a and 104-b are configured to perform maintenance operations after data frame 340 is stored in buffers 315-a and 315-b (e.g., after cross-chip triggers are sent and received). In some embodiments, this operation corresponds to maintenance operations associated with a link, link quality assessment, or link reliability determination. For example, the software stack of device 104-a may utilize in-band cross-chip triggers to initiate maintenance operations. In at least one embodiment, the software stack of device 104-a may perform maintenance operations via cross-chip triggers to assess link quality / reliability. In at least one embodiment, the software stack of device 104-a may determine to partially or completely retrain the link—e.g., retrain the link without stopping functional data traffic. In other embodiments, if retraining fails, the software stack of device 104-a may restart the system, perform the next predetermined maintenance operation in advance based on heuristics (e.g., trends in link quality over time may help detect impending link failures), or take no action.

[0041] Figure 4 An exemplary communication system 400 according to at least one exemplary embodiment is illustrated. In some embodiments, the communication system 400 may be a reference Figure 1 and Figure 3 Examples of the described communication systems 100 or 300. For example, system 400 may include host 102-a, host 102-b, first device 104-a, and second device 104-b, as shown in reference. Figure 1 As described. In at least one embodiment, system 100 further includes a link 106 coupling the first device 104-a and the second device 104-b. Each device 104 may include a transaction layer (TL) 108, a data link layer (DL) 110, and a transmitter (e.g., reference 104-b). Figure 1 The physical layer (PL) 112 associated with the transmitter 130 described herein. Each device 104 may include a receiver (e.g., as described in reference 130). Figure 1 The receiver 135 described is associated with TL 114, DL 116, and PL 118. In at least one embodiment, the transmitter's DL 110 may include a data pipeline 305-a, a control 310-a, and a buffer 315-a, as referenced. Figure 3As described—for example, although not shown, the transmitter's DL 110-b of device 104-b may also include data pipeline 305-a, control 310-a, and buffer 315-a. In some embodiments, the receiver's DL 116 may include data pipeline 305-b, control 310-b, and buffer 315-b—for example, although not shown, the receiver's DL 116-b of device 104-b may also include data pipeline 305-b, control 310-b, and buffer 315-b. In one embodiment, the communication system 400 may illustrate an example of a debug operation in which data frames including cross-chip triggers are sent from device 104-a back to device 104-a. Although not shown, device 104-b may also send cross-chip triggers back to device 104-b for debug operations.

[0042] In at least one embodiment, communication system 400 illustrates an alternative method for in-band transmission of cross-chip triggers compared to communication system 300. Specifically, host 102-a, TL 108-a, data pipeline 305-a, control 310-a, buffer 315-a, data pipeline 305-b, control 310-b, and buffer 315-b can perform reference... Figure 3 The described operation. For example, data pipeline 305-a can write one or more bits to the trigger field of data frame 318 to indicate a cross-chip trigger in response to an indication in receive data frame 318, receive debug trigger 325, receive external trigger 330, or generate internal trigger 360, as described in the reference. Figure 3 As described. In such an embodiment, data pipeline 305-a can transmit trigger 360 to control 310-a and send data frame 340 to PL 112-a. Control 310-a can send a capture command 365 upon receiving trigger 360 and cause buffer 315-a to store data frame 340—for example, all data frames with cross-chip triggers. Furthermore, data pipeline 305-b can receive data frame 340, identify cross-chip triggers, and generate trigger 350. In such an embodiment, control 310-b can send a capture command 365 to buffer 315-b and cause buffer 315-b to store data frame 340—for example, all data frames with cross-chip triggers.

[0043] However, the communication system 400 shows that instead of storing the data frame 340 in a buffer in DL 116-b of device 104-b, the data frame 340 is stored in buffer 315-b in DL 116-a of the receiver of device 104-a. For example, the software stack of device 104-a can communicate with the software stack of device 104-b and instruct the data frame 340 (e.g., a data frame with cross-chip triggering) to be sent back to device 104-a. In such an embodiment, the software stack of device 104-b can program components of device 104-b to send the data frame 340 back to device 104-a. In other embodiments, data pipeline 305-a can write one or more bits in the trigger field of data frame 340 to indicate that data frame 340 should be sent back to device 104-a. In either case, the data pipeline of DL 116-b can identify and transmit data frame 340, including cross-chip triggering, to the receiver's TL 114-b without storing the data frame 340 in a buffer. The receiver's TL 114-b can then send data frame 340 to the transmitter's TL 108-b of device 104-b. In embodiments where the data in the data frame is part of normal operation (e.g., the data is not a data vector generated by the software stack of device 104-a), TL 114-b can also send the data to the host 102-b. In some embodiments, data frame 340 can then be sent back to the receiver's PL 118-a of device 104-a via link 106. In some embodiments, the transmitter of device 104-b can send the same data frame 340 received at the receiver of device 104-b back to device 104-a. In other embodiments, the transmitter of device 104-b may generate a second data frame 340, which is a copy of the data frame 340 received at the receiver of device 104-b.

[0044] In at least one embodiment, after data frame 340 is stored in buffer 315-b (e.g., after data frame 340 transmitted by device 104-a is also received and stored in buffer 315-b), the software stack of device 104-a can perform readout 335 at buffers 315-a and 315-b to perform as referenced. Figure 3The aforementioned debugging operations—for example, determining an error count or assessing the health of the link, and taking appropriate action (e.g., resetting the link, retraining the link, or other operations to reduce the number of errors). In at least one embodiment, since data frames 340 are stored in buffers 315-a and 315-b of device 104-a, the software stack of device 104-a can avoid communicating with the software stack of device 104-b when performing debugging operations—for example, the debugging operations can be internal to device 104-a. In some embodiments, device 104-a may include a microcontroller coupled to buffers 315-a and 315-b—for example, an on-chip microcontroller. In such embodiments, the microcontroller can perform speed assessments or debugging operations—for example, assessments performed when data frames 340 are stored in buffer 315-b. For example, the microcontroller can determine the number of errors by comparing data frames 340 stored in buffers 315-a and 315-b. In such embodiments, if the determined number of errors exceeds an error threshold, the microcontroller can send an indication to the software stack of device 104-a. In other embodiments, the microcontroller may perform reference... Figure 3 Other debugging operations described include, for example, identifying errors in pins, pin groups, or data channels. In such embodiments, the microcontroller can send the results to the software stack of device 104-a. In at least one embodiment, utilizing a microcontroller can reduce software intervention and shorten the cycle time for performing debugging operations.

[0045] Figure 5 An example flowchart of a method 500 for error rate interruption in hardware for high-speed interconnect is shown. Method 500 can be executed by processing logic including hardware, software, firmware, or any combination thereof. In at least one embodiment, method 500 is executed by reference... Figure 3 and Figure 4 The communication systems 300 and 400 described are executed by devices 104-a and 104-b—for example, by each device's TL108, data pipeline 305, control 310, buffer 315, and respective software stack or microcontroller. Although shown in a specific order or sequence, the order of processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be executed in different orders, and some processes may be executed in parallel. Furthermore, one or more processes may be omitted in different embodiments. Therefore, not all processes are required in every embodiment. Other illustrations illustrating methods for sending in-band cross-chip triggers to maintain and debug high-speed interconnects are possible.

[0046] In operation 505, the processing logic is configured to write one or more bits corresponding to a debug operation into a first portion of a data frame in response to an indication, the data frame including a second portion comprising data. For example, the processing logic may write one or more bits into a trigger field (e.g., trigger 212) to indicate a cross-chip trigger in response to a receive indication (e.g., receive debug trigger, internal trigger, or external trigger). In some embodiments, the processing logic may respond to receiving and decoding a second data frame received from a transaction layer transmitter component (e.g., TL 108-a) and identify the second data frame with a reference. Figure 3 The description of cross-chip triggering is associated with an indication to determine the writing of one or more bits. In some embodiments, the processing logic may be in a device coupled to a link comprising one or more data paths. In at least one embodiment, the processing logic may include or be coupled to a data link (DL) transmitter and a buffer. In some embodiments, data frames may be transmitted as part of normal operation—for example, a data frame may include data that a host (e.g., host 102) wants to send to a second portion of a second device. That is, cross-chip triggering may be transmitted concurrently with data. In other embodiments, the data in the second portion may be a data vector generated by the device's software stack. In some embodiments, this operation corresponds to a system debugging operation, a maintenance operation associated with a link, a link quality assessment, or a link reliability determination. For example, in-band cross-chip triggering may be used to initiate a maintenance operation. In at least one embodiment, a maintenance operation may begin from the processing logic by assessing the quality / reliability of the link via cross-chip triggering. In at least one embodiment, the maintenance operation may then determine to partially or completely retrain the link—for example, to retrain the link without stopping functional data traffic. In other embodiments, if retraining fails, the maintenance operation may restart the system, proceed with the next planned maintenance operation based on heuristics (e.g., trends in link quality over time may help detect impending link failures), or take no action.

[0047] In operation 510, the processing logic may send a first portion and a second portion of a data frame via one or more data paths of a link in response to writing one or more bits corresponding to a debug operation. That is, the processing logic may send a cross-chip trigger in-band (e.g., on the same link associated with the transmitted data). In some embodiments, when the data in the second portion is a data vector generated by a software stack, the processing logic may hold the link before sending the data vector to a second device, as referenced. Figure 3 As described.

[0048] In operation 515, the processing logic may store the data frame in a buffer in response to writing one or more bits corresponding to a debug operation in the first portion of the data frame. For example, the processing logic may, in response to writing one or more bits—for example, in response to determining that the data frame is associated with a cross-chip trigger—capture a command (e.g., as referenced). Figure 3 The captured 365 command described is sent to the buffer. Therefore, the buffer can store the data frame in response to receiving the captured command. In at least one embodiment, the processing logic may not receive an indication—for example, the processing logic may not receive a debug trigger, external trigger, or internal trigger. In such an embodiment, the processing logic may not write one or more bits to the first portion, or may write one or more bits to the first portion to indicate that the data frame is not associated with a debug operation. In such an embodiment, the processing logic may not store the data frame in the buffer.

[0049] In operation 520, the processing logic can receive a data frame. In some embodiments, the data frame can be received at a receiver of a second device coupled to the link via one or more data paths. In at least one embodiment, the processing logic can receive a second data frame associated with the data frame at the receiver of the device. In such embodiments, the second device can receive the data frame via one or more data paths. In some embodiments, the second device can determine that one or more bits in the first portion correspond to a debugging operation in response to receiving the data frame. In some embodiments, the second device can generate a second data frame, including the one or more bits, in response to determining that the one or more bits correspond to a debugging operation. In such embodiments, the second device can transmit the second data frame via one or more data paths in response to determining that one or more bits in the first portion correspond to a debugging operation. That is, the second device can send a copy of the data frame back to the device, as referenced. Figure 4 As described. In some embodiments, the second device may send the received data frame back to the device without creating a copy—for example, sending the original received data frame back to the device.

[0050] In operation 525, the processing logic may store the data frame in a second buffer. For example, the processing logic may decode the data frame in response to receiving it. The processing logic may determine that one or more bits in the first portion correspond to a debug operation, and store the data frame in response to determining that one or more bits correspond to a debug operation. In at least one embodiment, the second buffer is in the receiver of a second device (e.g., device 104-b), as referenced... Figure 3 As described. In at least one embodiment, the second buffer is in, as referenced Figure 4In the receiver of the described device (e.g., device 104-a), for example, processing logic may decode a second data frame received at the device, determine that one or more bits of the second data frame correspond to a debug operation, and store the second data frame in a second buffer. In some embodiments, the processing logic may determine that the received data frame includes one or more bits in a first portion that do not correspond to a debug operation. In such embodiments, the processing logic may not store the data frame in the second buffer.

[0051] In operation 530, the processing logic can execute a debugging operation indicated in one or more bits. In some embodiments, if the data frame is stored in a second buffer in the second device, the debugging operation can be performed at both the device and the second device. For example, the device may include a first set of components associated with a first software stack (e.g., the software stack of device 104-a), and the second device may also include a second set of components associated with a second software stack (e.g., the software stack of device 104-b). In such an embodiment, the second set of components (e.g., the second software stack) can send a message to the first software stack via a second link (e.g., a sideband link) coupled to the device and the second device, the message being associated with receiving data including one or more bits corresponding to the debugging operation. For example, the second set of components can send a signature corresponding to an entry in the second buffer to the device, as referenced. Figure 3 As described. In some embodiments, the first software stack may compare the entries in the first buffer with the signature and perform a reference... Figure 3 The aforementioned debugging operations—for example, retraining or resetting the link when the number of detected errors exceeds a threshold number of errors—may occur. In some embodiments, if the second data frame is stored in a second buffer of the device, the debugging operations can occur internally to the device, as described in the reference. Figure 4 As described. For example, the device may include a controller (e.g., a microcontroller) coupled to a buffer and a second buffer. In some embodiments, the controller may compare a data frame with a second data frame in response to the second data frame being stored at the second buffer, and perform debugging operations in response to comparing the data frame and the second data frame. In other embodiments, the device's software stack may perform internal debugging operations. For example, the device may include a set of components associated with the software stack, and that set of components may perform debugging operations in response to the second data frame being stored at the second buffer. In any embodiment (e.g., the data frame is stored at a receiver of the second device, or the second data frame is stored at a receiver of the device), the processing logic may also perform debugging operations associated with a pin of the link, a set of pins of the link, or a data channel of the link, as referenced. Figure 3 As described.

[0052] Figure 6A computer system 600 according to at least one embodiment is shown. In at least one embodiment, the computer system 600 includes... Figure 1-3 The system is publicly available in China, and can be executed. Figure 4 This is part of all the processes disclosed in 400. For example, computer system 600 may be from... Figure 1 CPU 102. In at least one embodiment, computer system 600 may be a system with interconnected devices and components, a System-on-a-Chip (SoC), or some combination thereof. In at least one embodiment, computer system 600 is formed by processor 602, which may include execution units for executing instructions. In at least one embodiment, computer system 600 may include, but is not limited to, components such as processor 602, which employs execution units including logic to execute algorithms for process data. In at least one embodiment, computer system 600 may include processors such as the PENTIUM® processor family, Xeon™, Itanium®, XScale™ and / or StrongARM™, Intel® Core™ or Intel® Nervana™ microprocessors available from Intel Corporation of Santa Clara, California, although other systems (including PCs, engineering workstations, set-top boxes, etc.) may also be used. In at least one embodiment, the computer system 600 may execute a version of the Windows operating system available from Microsoft Corporation of Redmond, Washington, although other operating systems (such as UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0053] In at least one embodiment, the computer system 600 can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol (IP) devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, the embedded application can include a microcontroller, a digital signal processor (“DSP”), a SoC, a network computer (“NetPC”), a set-top box, a network hub, a wide area network (“WAN”) switch, or any other system capable of executing one or more instructions according to at least one embodiment. In embodiments, the computer system 600 can be used in devices such as graphics processing units (GPUs), network adapters, central processing units, and network devices, such as switches (e.g., high-speed direct GPU-to-GPU interconnects, such as NVIDIA GH100 NVLINK or NVIDIA Quantum 2 64-port InfiniBand NDR switches).

[0054] In at least one embodiment, computer system 600 may include, but is not limited to, processor 602, which may include, but is not limited to, one or more execution units 608 configured to execute Computing Unified Device Architecture (“CUDA”) programs (CUDA® is developed by NVIDIA Corporation in Santa Clara, California). In at least one embodiment, a CUDA program is at least a portion of a software application written in the CUDA programming language. In at least one embodiment, computer system 600 is a single-processor desktop or server system. In at least one embodiment, computer system 600 may be a multiprocessor system. In at least one embodiment, processor 602 may include, but is not limited to, a CISC microprocessor, a RISC microprocessor, a VLIW microprocessor, a processor implementing instruction set combinations, or any other processor device, such as a digital signal processor. In at least one embodiment, processor 602 may be coupled to a processor bus 610, which may transmit data signals between processor 602 and other components in computer system 600.

[0055] In at least one embodiment, processor 602 may include, but is not limited to, a Level 1 (“L1”) internal cache memory (“cache”) 604. In at least one embodiment, processor 602 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory may reside external to processor 602. In at least one embodiment, processor 602 may include a combination of internal and external caches. In at least one embodiment, register file 606 may store different types of data in various registers, including but not limited to integer registers, floating-point registers, status registers, and instruction pointer registers.

[0056] In at least one embodiment, an execution unit 608, including but not limited to logic performing integer and floating-point operations, is also located within the processor 602. The processor 602 may also include a microcode (“ucode”) read-only memory (“ROM”) for storing microcode for certain macro instructions. In at least one embodiment, the execution unit 608 may include logic for processing a packaged instruction set 609. In at least one embodiment, by including the packaged instruction set 609 in the instruction set of the general-purpose processor 602, along with associated circuitry for executing the instructions, packaged data in the general-purpose processor 602 can be used to perform operations used by numerous multimedia applications. In at least one embodiment, many multimedia applications can be executed more quickly and efficiently by using the full width of the processor’s data bus to perform operations on the packaged data, which may eliminate the need to send smaller data units on the processor’s data bus to perform one or more operations on a data element at a time.

[0057] In at least one embodiment, the execution unit may also be used in a microcontroller, embedded processor, graphics device, DSP, and other types of logic circuitry. In at least one embodiment, the computer system 600 may include, but is not limited to, memory 620. In at least one embodiment, memory 620 may be implemented as a DRAM device, SRAM device, flash memory device, or other storage device. Memory 620 may store instructions 619 and / or data 621 represented by data signals that can be executed by processor 602.

[0058] In at least one embodiment, the system logic chip may be coupled to the processor bus 610 and the memory 620. In at least one embodiment, the system logic chip may include, but is not limited to, a memory controller hub (“MCH”) 616, and the processor 602 may communicate with the MCH 616 via the processor bus 610. In at least one embodiment, the MCH 616 may provide a high-bandwidth memory path 618 to the memory 620 for instruction and data storage, as well as for storage of graphics commands, data, and textures. In at least one embodiment, the MCH 616 may initiate data signals between the processor 602, the memory 620, and other components in the computer system 600, and bridge data signals between the processor bus 610, the memory 620, and the system I / O 622. In at least one embodiment, the system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, the MCH 616 may be coupled to the memory 620 via the high-bandwidth memory path 618, and the graphics / video card 612 may be coupled to the MCH 616 via an Accelerated Graphics Port (“AGP”) interconnect 614.

[0059] In at least one embodiment, the computer system 600 may use system I / O 622 as a proprietary hub interface bus to couple MCH 616 to I / O controller hub (“ICH”) 630. In at least one embodiment, ICH 630 may provide direct connectivity to certain I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may include, but is not limited to, a high-speed I / O bus for connecting peripheral devices to memory 620, chipset, and processor 602. Examples may include, but are not limited to, an audio controller 629, a firmware hub (“Flash BIOS”) 628, a wireless transceiver 626, a data storage 624, a conventional I / O controller 623 including user input 625 and a keyboard interface, a serial expansion port 627 (e.g., USB), and a network controller 634. Data storage 624 may include a hard disk drive, floppy disk drive, CD-ROM device, flash memory device, or other mass storage device. In one embodiment, transceiver 626 includes a restricted FFE 608.

[0060] In at least one embodiment, Figure 6 The system shown includes interconnected hardware devices or "chips" in transceiver 626—for example, transceiver 626 includes chip-to-chip interconnects that include a first device 104-a and a second device 104-b, as referenced. Figure 1 As described. In at least one embodiment, Figure 6 An exemplary SoC can be shown. In at least one embodiment, Figure 6 The device shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof, and utilizes link 106, as referenced. Figure 1 As described. In at least one embodiment, one or more components of system 600 are interconnected using a compute-fast link (CXL) interconnect. In an embodiment, transceiver 626 may include, as described in reference... Figure 1 The described chip trigger 150. In such an embodiment, chip trigger logic 150 enables devices 104-a and 104-b to send cross-chip triggers within the band (e.g., across link 106 associated with transmitting data). Therefore, chip trigger logic 150 can be used in methods and systems for sending cross-chip triggers within the band to maintain and debug high-speed interconnects.

[0061] Other variations are within the spirit of this disclosure. Therefore, although the disclosed technology is readily adaptable to various modifications and alternative constructions, certain embodiments thereof are illustrated in the accompanying drawings and have been described in detail above. However, it should be understood that the disclosure is not intended to be limited to one or more specific forms disclosed, but rather, it is intended to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of this disclosure as defined in the appended claims.

[0062] Unless otherwise stated or obviously contradicted by the context, the terms “a,” “an,” and “the,” and similar references, used in the context of describing the disclosed embodiments (particularly in the context of the appended claims), should be interpreted as encompassing both singular and plural forms, rather than as definitions of terms. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (meaning “including, but not limited to”). “Connection” (referring to a physical connection where not modified) should be interpreted as partially or wholly contained, attached to, or linked together, even with some intervention. Unless otherwise indicated herein, references to numerical ranges herein are intended only as a way of abbreviating each individual value falling within that range, and each individual value is incorporated into the specification as if it were separately described herein. In at least one embodiment, unless otherwise indicated or contradicted by the context, the use of the terms “set” (e.g., “item set”) or “subset” should be interpreted as a non-empty set comprising one or more members. Furthermore, unless otherwise indicated or contradicted by the context, the term “subset” of the corresponding set does not necessarily mean an appropriate subset of the corresponding set, but rather that the subset and the corresponding set can be equal.

[0063] Unless otherwise explicitly stated or clearly contradicted by the context, connective phrases such as “at least one of A, B, and C” or “at least one of A, B, and C” are understood in the context to generally refer to items, terms, etc., which can be A or B or C, or any non-empty subset of the set A, B, and C. For example, in an illustrative example of a set with three members, the connective phrases “at least one of A, B, and C” and “at least one of A, B, and C” refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Therefore, such connective language is generally not intended to imply that some embodiments require the presence of at least one of A, at least one of B, and at least one of C. Additionally, unless otherwise stated or contradicted by the context, the term “multiple” indicates a plural state (e.g., “multiple items” means multiple items). In at least one embodiment, the number of items in the multiple items is at least two, but may be more if explicitly indicated or indicated by the context. Furthermore, unless otherwise stated or clearly understood from the context, the phrase “based on” means “at least partially based on” rather than “based on only”.

[0064] Unless otherwise stated herein or clearly contradicted by the context, the operations of the processes described herein may be performed in any suitable order. In at least one embodiment, processes such as those described herein (or variations thereof and / or combinations thereof) are executed under the control of one or more computer systems configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more application programs) that is executed jointly on one or more processors via hardware or a combination thereof. In at least one embodiment, the code is stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transient signals (e.g., propagating transient electrical or electromagnetic transmissions) but includes non-transitory data storage circuitry (e.g., buffers, caches, and queues). In at least one embodiment, code (e.g., executable code or source code) is stored on one or more non-transitory computer-readable storage media (or other memory for storing executable instructions) on which executable instructions are stored, which, when executed by one or more processors of a computer system (i.e., as a result of execution), cause the computer system to perform the operations described herein. In at least one embodiment, the set of non-transitory computer-readable storage media comprises a plurality of non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media lack all the code, but the plurality of non-transitory computer-readable storage media collectively store all the code. In at least one embodiment, the executable instructions are executed such that different instructions are executed by different processors.

[0065] Therefore, in at least one embodiment, the computer system is configured to implement one or more services that perform the operations of the processes described herein, either individually or collectively, and such a computer system is configured with suitable hardware and / or software to enable the implementation of the operations. Furthermore, the computer system implementing at least one embodiment of this disclosure is a single device, and in another embodiment it is a distributed computer system comprising multiple devices operating in different ways, such that the distributed computer system performs the operations described herein, and that a single device does not perform all the operations.

[0066] The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate embodiments of this disclosure and does not constitute a limitation on the scope of the disclosure unless otherwise required. No language in the specification should be construed as indicating that any unclaimed element is essential to the practice of the disclosure.

[0067] All references cited in this article, including publications, patent applications and patents, are incorporated herein by reference as if each reference were individually and specifically indicated to be incorporated herein by reference and the entire contents of which are described herein.

[0068] The terms “coupled” and “connected”, and their derivatives, may be used in the specification and claims. It should be understood that these terms may not be intended to be synonyms with each other. Rather, in certain examples, “connected” or “coupled” may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. “Coupled” may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0069] Unless otherwise expressly stated, it will be understood that throughout this specification, terms such as “processing,” “computing,” “determining,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that process and / or convert data represented as physical quantities (e.g., electrons) in the registers and / or memory of the computing system into other data represented as physical quantities in the memory, registers, or other such information storage, transmission, or display devices of the computing system.

[0070] Similarly, the term "processor" can refer to any device or part of memory that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. A "computing platform" can include one or more processors. As used herein, a "software" process can include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Likewise, each process can refer to multiple processes that execute instructions sequentially or intermittently, or in parallel. In at least one embodiment, the terms "system" and "method" are used interchangeably herein, provided that a system can embody one or more methods, and a method can be considered a system.

[0071] In this document, reference may be made to obtaining, acquiring, receiving, or inputting analog or digital data to a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog and digital data can be accomplished in various ways, such as by receiving data as a parameter to a function call or a call to an application programming interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data via a serial or parallel interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data from a providing entity to an acquiring entity via a computer network. In at least one embodiment, reference may also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, the process of providing, outputting, transmitting, sending, or presenting analog or digital data can be implemented by transmitting data as an input or output parameter to a function call, an application programming interface, or an inter-process communication mechanism.

[0072] While the description given herein illustrates exemplary embodiments of the described technologies, other architectures may be used to implement the described functionality and are intended to fall within the scope of this disclosure. Furthermore, although specific assignments of responsibilities may be defined above for descriptive purposes, various functions and responsibilities may be assigned and divided in different ways depending on the circumstances.

[0073] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter claimed in the appended claims is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms for implementing the claims.

Claims

1. A system comprising: A link, which includes one or more data paths; A device coupled to the link and including a data link DL transmitter and a buffer, the device being used for: In response to an instruction, one or more bits corresponding to the operation are written into a first portion of a data frame, the data frame including a second portion comprising data; In response to writing the one or more bits corresponding to the operation, the first and second portions of the data frame are transmitted via the one or more data paths; as well as In response to writing one or more bits corresponding to the operation, the data frame is stored in the buffer; as well as A second device, coupled to the link and including a second buffer, is used for: The data frame is received via one or more data paths; In response to receiving the data frame, the data frame is decoded; Determine that one or more bits in the first part correspond to the operation; as well as In response to determining that one or more bits correspond to the operation, the data frame is stored in the second buffer.

2. The system according to claim 1, wherein: The device also includes a first set of components associated with a first software stack; and The second device also includes a second set of components associated with the second software stack, the second set of components being used for: A message associated with receiving the data frame is sent to the first software stack via a second link coupled to the device and the second device, the data frame including one or more bits corresponding to the operation.

3. The system of claim 1, wherein the device further comprises a data link DL receiver and a second buffer, the device being used for: Receive a second data frame associated with the data frame; In response to receiving the second data frame, the second data frame is decoded; In response to decoding the second data frame, determine one or more bits of the second data frame that correspond to the operation; as well as In response to determining that one or more bits of the second data frame correspond to the operation, the second data frame is stored in the second buffer.

4. The system of claim 3, wherein the device further comprises a controller coupled to the buffer and the second buffer, the controller being configured to: In response to the second data frame being stored in the second buffer, the data frame is compared with the second data frame; and The operation is performed in response to comparing the data frame and the second data frame.

5. The system of claim 3, wherein the device further comprises a set of components associated with a software stack, the set of components being used for: The operation is performed in response to the second data frame being stored in the second buffer.

6. The system of claim 3, further comprising a second device coupled to the link, the second device being configured to: The data frame is received via one or more data paths; In response to receiving the data frame, determine that one or more bits in the first portion correspond to the operation; In response to determining that one or more bits correspond to the operation, a second data frame is generated, the second data frame including the one or more bits; as well as In response to determining that one or more bits in the first portion correspond to the operation, the second data frame is transmitted via the one or more data paths.

7. The system of claim 1, wherein the operation corresponds to system debugging operation, maintenance operation associated with the link, link quality assessment, or link reliability determination.

8. The system of claim 1, wherein the device is further configured to: Receive a second data frame including the indication from the transaction layer transmitter, wherein writing the one or more bits is in response to receiving the second data frame.

9. The system of claim 1, wherein the device is further configured to: Receive the instruction, wherein writing the one or more bits is in response to receiving the instruction.

10. A method comprising: At the data link DL transmitter of the device, in response to an instruction, one or more bits corresponding to the operation are written into a first part of a data frame, the data frame including a second part including data; In response to writing the one or more bits corresponding to the operation, the first and second portions of the data frame are transmitted via one or more data paths of a link coupled to the device; In response to writing one or more bits corresponding to the operation, the data frame is stored in the buffer of the device; At the second device, the data frame is received via one or more data paths; At the second device, in response to receiving the data frame, the data frame is decoded; At the second device, it is determined that one or more bits in the first portion correspond to the operation; as well as In response to determining that one or more bits correspond to the operation, the data frame is stored in a second buffer of the second device.

11. The method of claim 10, further comprising: Messages are sent from a first set of components associated with a first software stack of the second device to a second set of components associated with a second software stack of the device, wherein the messages are sent via a second link coupled to the device and the second device.

12. The method of claim 10, further comprising: At the DL receiver of the device, a second data frame associated with the data frame is received; In response to receiving the second data frame, the second data frame is decoded; In response to decoding the second data frame, determine one or more bits of the second data frame that correspond to the operation; as well as In response to determining that one or more bits of the second data frame correspond to the operation, the second data frame is stored in a second buffer.

13. The method of claim 12, further comprising: At the controller coupled to the buffer and the second buffer, in response to the second data frame being stored in the second buffer, the data frame is compared with the second data frame; as well as The operation is performed in response to comparing the data frame and the second data frame.

14. The method of claim 13, further comprising: The operation is performed at a set of components associated with the software stack of the device in response to the second data frame being stored in the second buffer.

15. The method of claim 13, further comprising: At the second device, the data frame is received via one or more data paths; At the second device, in response to receiving the data frame, it is determined that one or more bits in the first portion correspond to the operation; At the second device, in response to determining that the one or more bits correspond to the operation, a second data frame is generated, the second data frame including the one or more bits; as well as In response to determining that one or more bits in the first portion correspond to the operation, the second data frame is transmitted from the second device to the device via the one or more data paths.

16. A system comprising: A link, which includes one or more data paths; A first device, coupled to the link and including a data link (DL) transmitter and a buffer, is used for: In response to an instruction, one or more bits corresponding to the operation are written into a first portion of a data frame, the data frame including a second portion comprising data; In response to writing the one or more bits corresponding to the operation, the first and second portions of the data frame are transmitted via the one or more data paths; as well as In response to writing one or more bits corresponding to the operation, the data frame is stored in the buffer; as well as A second device, coupled to the link and including a DL receiver and a second buffer, is used for: In response to the first device sending the data frame, receiving the data frame, wherein the second device is further configured to: In response to receiving the data frame, the data frame is decoded; In response to decoding the data frame, determine that one or more bits in the first portion of the data frame correspond to the operation; as well as In response to determining that one or more bits correspond to the operation, the data frame is stored in the second buffer.

17. The system of claim 16, wherein the second device is further configured to: In response to decoding the data frame, determine that one or more bits of the first portion of the data frame correspond to the operation; In response to determining that one or more bits correspond to the operation, a second data frame associated with the data frame is generated, the second data frame including the one or more bits; and In response to generating the second data frame, the second data frame is sent to the first device via the one or more data paths.

18. A system comprising: A link, comprising one or more data paths; as well as A device, coupled to the link and including a data link DL transmitter and a buffer, is used for: In response to receiving a trigger signal from any of the software stack, internal logic, or external hardware, one or more bits indicating a cross-chip debug trigger are written into a first portion of a data frame, the data frame also including a second portion comprising data; The data frame comprising the first part and the second part is transmitted within the synchronization zone of one or more data paths of the link; as well as In response to writing one or more bits, the data frame is stored in the buffer.

19. The system of claim 18, further comprising a second device coupled to the link and including a second buffer, the second device being configured to: The data frame is received via one or more data paths; In response to receiving the data frame, the data frame is decoded; The first portion is determined to include the one or more bits that indicate the cross-chip debug trigger; as well as In response to determining that the first portion includes the one or more bits indicating the cross-chip debug trigger, the data frame is stored in the second buffer.

20. The system of claim 18, wherein the device further comprises a data link DL receiver and a second buffer, the device being configured to: Receive a second data frame associated with the data frame; In response to receiving the second data frame, the second data frame is decoded; Determine one or more bits in the second data frame to indicate a second cross-chip debug trigger; as well as In response to determining that one or more bits of the second data frame indicate the second cross-chip debug trigger, the second data frame is stored in the second buffer.

Images