Streaming fabric interface

Abstract

To provide an apparatus in which a link trained in two non-flit modes (NFM) is communicated, and packet integrity is not damaged while simplicity of receiver decode in cut-through routing and flit mode (FM) format is maintained.SOLUTION: An interface for coupling an agent to a fabric supports a load / store interconnect protocol which includes an FM and a NFM. In the case of the FM, a set of FM header formats is used, and in the case of the NFM, a set of NFM header formats is used. Interface logic determines that a link is trained to the NFM, generates a header according to the set of FM header formats. One or more fields of the set of FM header formats are reused in the header to carry the one or more NFM fields before sending the modified header over the interface.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] This disclosure relates to computing systems, and particularly (though not exclusively) to point-to-point interconnects. [Background technology]

[0002] Advances in semiconductor processing and logic design have enabled an increase in the amount of logic that can reside in an integrated circuit device. As a natural consequence, computer system configurations have evolved from a single or multiple integrated circuits in a system to multiple cores, multiple hardware threads, and multiple logical processors on individual integrated circuits, as well as other interfaces integrated with such processors. A processor or integrated circuit typically comprises a single physical processor die, which may include any number of cores, hardware threads, logical processors, interfaces, memory, controller hubs, etc.

[0003] Smaller computing devices have become more popular as a result of their greater ability to fit more processing power in a smaller package. Smartphones, tablets, ultra-thin notebooks, and other user equipment have grown exponentially. However, these smaller devices rely on servers for both data storage and complex processing that exceeds their form factor. As a result, the need in the high performance computing market (i.e., server space) has also increased. For example, in today's servers, there are typically multiple physical processors (also referred to as multiple sockets) to increase computing power, rather than just a single processor with multiple cores. However, as processing power increases along with the number of devices in a computing system, communication between the sockets and other devices becomes more important.

[0004] In fact, interconnects have grown from traditional multi-drop buses that primarily handled electrical communications to full-blown interconnect architectures that facilitate high-speed communications. Unfortunately, there is a need for future processors that are much more efficiently utilized, and a corresponding demand is being placed on the capabilities of existing interconnect architectures. [Brief description of the drawings]

[0005] [Figure 1] 1 is a simplified block diagram illustrating an example embodiment of a system-on-chip (SoC) device.

[0006] [Diagram 2] 1 is a simplified block diagram of a Streaming Fabric interface.

[0007] [Diagram 3] 1 is a simplified block diagram of another example Streaming Fabric interface.

[0008] [Figure 4] 1 is a simplified block diagram illustrating an exemplary Compute Express Link (CXL) topology.

[0009] [Diagram 5] 1 is a simplified block diagram of a channel of an exemplary Streaming Fabric interface.

[0010] [Figure 6] 1 is a simplified block diagram illustrating an exemplary receiver buffer.

[0011] [Figure 7] 13 is a representation of example fields in metadata for a header channel of a Streaming Fabric interface.

[0012] [Figure 8]1 is a timing diagram illustrating an example data flow on a header channel of an example streaming fabric interface.

[0013] [Figure 9] 4 is a timing diagram illustrating an example data flow on a data channel of an example streaming fabric interface.

[0014] [Figure 10] 1 is a diagram illustrating an example initialization state machine for an example flexible on-die fabric interface.

[0015] [Figure 11] 1 is a timing diagram illustrating an example flexible on-die fabric interface initialization.

[0016] [Figure 12] 1 is a timing diagram illustrating a first example of a disconnect flow in an exemplary flexible on-die fabric interface.

[0017] [Figure 13] 11 is a timing diagram illustrating a second example of a disconnect flow in an exemplary flexible on-die fabric interface.

[0018] [Figure 14] 1 is a simplified block diagram of an exemplary computing system.

[0019] [Figure 15A] 1 illustrates an exemplary portion of a flit mode packet header. [Figure 15B] 1 illustrates an exemplary portion of a flit mode packet header. [Figure 15C] 1 illustrates an exemplary portion of a flit mode packet header. [Figure 15D]1 illustrates an exemplary portion of a flit mode packet header.

[0020] [Figure 16] 1 illustrates an exemplary packet header.

[0021] [Figure 17] 1 illustrates an exemplary computing system including a credit gasket.

[0022] [Figure 18] 1 is a timing diagram illustrating an exemplary use of a credit gasket.

[0023] [Figure 19A] 11 is a timing diagram showing an additional exemplary use of a credit gasket. [Figure 19B] 11 is a timing diagram showing an additional exemplary use of a credit gasket.

[0024] [Figure 20] 1 is a diagram illustrating the logic of an example credit gasket.

[0025] [Figure 21] 1 is a timing diagram illustrating an exemplary use of a credit gasket.

[0026] [Figure 22A] 11 is a timing diagram showing an additional exemplary use of a credit gasket. [Figure 22B] 11 is a timing diagram showing an additional exemplary use of a credit gasket.

[0027] [Figure 23] 1 is a simplified block diagram of an exemplary computing system including a bufferless arbiter.

[0028] [Figure 24A] 1 is a timing diagram illustrating an exemplary use of an arbiter in a computing system. [Figure 24B] 1 is a timing diagram illustrating an exemplary use of an arbiter in a computing system. [Figure 24C] 1 is a timing diagram illustrating an exemplary use of an arbiter in a computing system. [Figure 24D] 1 is a timing diagram illustrating an exemplary use of an arbiter in a computing system.

[0029] [Diagram 25] FIG. 1 illustrates an embodiment of a block diagram for a computing system including a multi-core processor.

[0030] [Figure 26] 1 illustrates another embodiment of a block diagram for a computing system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] In the following description, numerous specific details are described to provide a thorough understanding of the present disclosure, such as examples of specific types of processors and system configurations, specific hardware structures, specific design and micro-design details, specific register configurations, specific instruction types, specific system components, specific dimensions / heights, specific processor pipeline stages, and operations. However, it will be apparent to those skilled in the art that these specific details need not be employed to practice the embodiments of the present disclosure. In other examples, well-known components or methods, such as specific and alternative processor architectures, specific logic circuits / code for the described algorithms, specific firmware code, specific interconnect operations, specific logic configurations, specific manufacturing techniques and materials, specific compiler implementations, specific expressions for the algorithms in the code, specific power-down and gating techniques / logic, and other specific operational details of computer systems, are not described in detail to avoid unnecessarily obscuring the present disclosure.

[0032] The following embodiments may be described with reference to efficient high-speed data transmission and configurability in certain integrated circuits, such as computing platforms or microprocessors, although other embodiments are applicable to other types of integrated circuits and logic devices. Similar techniques and teachings of the embodiments described herein may be applied to other types of circuits or semiconductor devices that may also benefit from better energy efficiency and energy conservation. For example, the disclosed embodiments may be applied to computing systems embodied as servers, blades, desktop computer systems, system-on-chip (SoC) devices, handheld devices, tablets, set-top boxes, in-vehicle computing systems, computer vision systems, gaming systems, machine learning systems, and embedded applications. As will be readily apparent in the following description, the method, apparatus, and system embodiments described herein (whether related to hardware, firmware, software, or a combination thereof) are beneficial to the development of high-performance computer interconnects and their respective systems.

[0033] As computing systems advance, the components therein become more complex. As a result, the interconnect architectures for coupling and communicating between components also become more complex to ensure that bandwidth requirements for optimal component operation are met. Furthermore, different market segments require different aspects of the interconnect architectures to meet market needs. For example, servers require higher performance, while the mobile ecosystem may in some cases sacrifice overall performance for power savings. Moreover, the sole purpose of many fabrics is to provide the highest possible performance while maximizing power savings. The interconnects described below could potentially benefit from aspects of the solutions described herein.

[0034] An exemplary interconnect fabric architecture includes the Peripheral Component Interconnect (PCI) Express (PCIe) architecture. The primary goal of PCIe is to enable components and devices from different vendors to interoperate in an open architecture across multiple market segments, such as clients (desktop and mobile), servers (standard and enterprise), and embedded communication devices. PCI Express is a high-performance, versatile I / O interconnect defined for a variety of future computing and communication platforms. Several PCI attributes, such as its usage model, load-store architecture, and software interface, have been maintained throughout its revisions, whereas the previous parallel bus implementation has been replaced by a highly scalable fully serial interface. More recent versions of PCI Express leverage the advantages of point-to-point interconnects, switch-based technology, and packetized protocols to deliver new levels of performance and features. Power management, quality of service (QoS), hot-plug / hot-swap support, data integrity, and error handling are some of the advanced features supported by PCI Express.

[0035] Conventional streaming interfaces for coupling fabrics to protocol agents may generally include proprietary interfaces (e.g., Intel® On-Chip System Fabric (IOSF®)), interfaces developed for coherent or unordered protocols, and other interfaces that are poorly adapted to scale to address evolving data rates in modern protocols and architectures. For example, proprietary interfaces may carry custom or use-case specific information or features that prevent standardization of the interface or fail to scale to next generation bandwidths. Meanwhile, other conventional interfaces may be defined in a more general manner, for example, as a data bus for carrying packets. However, conventional bus definitions and interfaces may introduce receiver decoding complexity, especially in the presence of multiple flow control classes or virtual channels, especially as data rates increase and more packets can be processed per clock cycle. As an example, if four (or more) packets of any channel or flow control can potentially arrive in a given clock cycle and are accessing a shared buffer, a corresponding four (or more) logical write ports may need to be provisioned at the receiver, resulting in excess surface area being allocated to providing such logic (and buffers). In some cases, conventional interfaces address the use case of multiple packets per cycle (of different flow control classes) by simply stamping multiple copies of the interface (e.g., one for each flow control class), resulting in a high pin count. Additionally, conventional streaming interfaces have headers and data packets that follow each other on the same physical wire, limiting the possibilities for latency optimization. Some conventional interfaces fail to provide an effective and flexible mechanism for credit flow, among other exemplary shortcomings.

[0036] In some implementations, an improved scalable streaming interface may be defined between agent logic on a device and the fabric, such as between protocol layers and other devices coupled to the fabric (e.g., CPUs, endpoint devices, switches, etc.). The streaming interface may support load / store protocols such as PCIe, Compute Express Link (CXL) (e.g., CXL.io), among other load / store protocols. The improved streaming interface may define interface rules and interface channels to enable large chip area and latency benefits during implementation, while providing power-efficient bandwidth scaling benefits that will become even more important as protocols approach higher speeds, such as moving to 32.0 GT / s in PCIe Gen5, or 64.0 GT / s data rates starting with PCIe Gen6 and CXL3.0 and beyond, among other examples. Such an interface may optimize the best balance between pin count and receiver decode complexity. In some implementations, the improved streaming interface discussed herein may enable a reduced number of logical write ports on a receiver buffer, where the receiver buffer is shared among multiple virtual channels and flow control classes. Additionally, the improved streaming interface may fork packet headers and data onto independent physical channels (e.g., a header channel and a data channel), thereby allowing the receiver to begin processing the header while the data is still being streamed, thereby helping to reduce overall latency and buffer size and complexity. Additionally, the improved streaming interface discussed herein may be standardized to allow the ecosystem of IP blocks to adopt and evolve to a scalable, standardized interface rather than a traditional proprietary interface, enabling more options for interoperability, among other example features and advantages such as those discussed herein.

[0037] Referring to the simplified block diagram 100 of Figure 1, a simplified example of a system-on-chip (SoC) device 105 is shown. An SoC may be implemented as an integrated circuit incorporating multiple components or computing blocks (or intellectual property (IP) blocks) of a computer. Such blocks (e.g., 110, 115, 120, 125, 130, 135, 140, 145) may include components such as one or more CPU components 110, 115, 120, 125 (e.g., microprocessors or microcontrollers), application specific processors 130, 135 (e.g., graphics processing units (GPUs), image signal processors (ISPs), tensor processing units, accelerator devices, etc.), memory components, input / output (I / O) ports, secondary storage blocks, and other compute blocks on a single die, such as a silicon die, or on a substrate.

[0038] The compute blocks (e.g., 110, 115, 120, 125, 130, 135, 140, 145) of the exemplary SoC 105 may be interconnected by a SoC fabric (e.g., 150). The fabric 150 may itself be implemented using a set of one or more IP blocks that facilitate communication between the compute blocks (e.g., 110, 115, 120, 125, 130, 135, 140, 145). In some implementations, the fabric 150 may be implemented as a network-on-chip (NOC), such as a NOC implemented with one or more circuit blocks.

[0039] Communications by the various blocks (e.g., 110, 115, 120, 125, 130, 135, 140, 145) may be facilitated through protocol agents (e.g., 160a-h) provided on the blocks (e.g., 110, 115, 120, 125, 130, 135, 140, 145). Each agent (e.g., 160a-h) may include logic (e.g., implemented in hardware circuitry, firmware, and / or software) to implement all or a subset of the layers of one or more interconnect protocols (e.g., PCIe, Compute Express Link (CXL), Gen-Z, OpenCAPI, In-Die Interface, Cache Coherent Interconnect for Accelerators (CCIX), UltraPath Interconnect (UPI), etc.) through which the corresponding compute block communicates with other compute blocks in the system. As discussed herein, the agents may couple to fabric 150 via their respective interfaces. While such agents may traditionally be coupled to the fabric via a proprietary wire interface, one or more agents (e.g., 160a-h) may utilize respective instances of a configurable and flexible on-die wire interface that may be deployed to support different protocols for different agents of the SoC 105. In other cases, the interfaces between the agents (e.g., 160a-h) may be to support non-coherent and / or load / store streaming protocols, and corresponding streaming fabric interfaces may be defined and implemented on the blocks (e.g., 110, 115, 120, 125, 130, 135, 140, 145) and fabric 150, among other example implementations.

[0040] As introduced above, an improved Streaming Fabric Interface Architecture (SFI) may be provided in components of a system (e.g., IP blocks and components implementing the system's fabric) to map load / store protocols (e.g., PCIe, CXL.io) between agents and the fabric. The SFI interface may provide a scalable streaming interface that can sustain the high bandwidth requirements of load / store protocols (including new next-generation speeds for such protocols). The SFI interface may enable easy implementation on both the sender and receiver sides when transmitting such high data rates. Additionally, logic implementing the SFI interface may embody, realize, and enforce rules for communication on the interface (e.g., beyond those defined in the protocols supported by the interface), which may greatly simplify storage overhead in the context of read / write ports on a receiver, among other exemplary features.

[0041] The SFI interface may be employed both in the context of a host CPU (e.g., through a root complex) or in the context of a device endpoint. In both cases, the SFI functions to carry protocol layer (transaction layer) specific information between different processing entities. As an example, on the device side, the SFI may be used to interface between a PCIe controller and an application layer (e.g., a fabric or gasket layer between the controller and the fabric). Similarly, on the host side, the SFI may be used to interface between a PCIe root port and the CPU fabric. Configurable parameters may be defined in the SFI interface, allowing an instance of the interface to be parameterized broadly enough and to carry multiple packets in a single transmission according to the supported protocols and system use cases. On a given SFI interface, data transmission may be unidirectional. Thus, in some implementations, a pair of SFI interface instances may be provided (one in each direction) to facilitate implementations utilizing bidirectional data transmission between communicating blocks. Therefore, many of the examples herein discuss a transmitter (TX) and receiver (RX) pair for a single instance of an SFI interface.

[0042] Using the SFI as an intermediary interface, different configurations may be enabled. For example, the SFI interface may make no assumptions regarding the protocol or application-specific roles of the interface's transmitters and receivers. Rather, the SFI interface may simply provide mechanisms and rules for high-bandwidth packet transmission. For example, FIG. 2 is a simplified block diagram 200 illustrating an example implementation of coupling a controller 210 (e.g., of an agent) to an application layer 215 (e.g., implemented through a fabric) via two SFI interface instances 205a, 205b. The controller 210 may include protocol circuitry or other logic to establish a link 220 according to a particular interconnect protocol (e.g., PCIe) and participate in initialization, training, and communication over the link 220. The example of FIG. 2 may represent an example use case of the SFI in a PCIe application. The SFI instance 205a may treat the PCIe controller 210 as a transmitter and the application layer element 215 as a receiver. Thus, application layer elements 215 may include receiver buffers for SFI interface 205a for use in maintaining flow control credits (including shared credits for the channels of SFI interface 205a). Similarly, SFI interface 205b may view application layer elements 215 as a transmitter and PCIe controller 210 as a receiver (and controller 210 may include corresponding receiver queues or buffers 225 for use with interface 205b).

[0043] Although some implementations of the SFI may utilize the semantics and header formats of PCIe-based protocols, the SFI is not limited to supported PCIe-based protocols. Furthermore, the SFI does not include new protocol definitions. The SFI semantics may be used to support a variety of different protocols, provided that the protocols can be mapped or adapted to the flow control (FC) and virtual channel (VC) semantics that the SFI provides, among other example features. For example, the SFI supports advertisement of zero or multiple shared credit pools for receiver queues (such as those discussed in more detail below).

[0044] Referring to FIG. 3, a simplified block diagram 300 is shown illustrating a conventional root complex stack utilizing an SFI interface. For example, the SFI interfaces 205a, 205b may be used to couple protocol stack logic (e.g., 305, 310) to a non-coherent-to-coherent protocol converter 315 (e.g., may be located between the protocol stack logic and the interconnect fabric 215 of the system). For example, the protocol stack logic may be embodied as an agent or controller for a particular non-coherent, load / store interconnect protocol (e.g., PCIe, CXL.io, etc.) and may include lower level layer logic 305 (e.g., implemented in a circuit) including physical layer logic and link layer logic. Transaction layer logic 310 may also be provided and may be a layer that interfaces with the converter 315 through the SFI interfaces (e.g., 205a, 205b). Buffers 225 (e.g., I / O / Queue (IOQ)) may be provided and used to hide physical link latency between the device and the host. The depth of such buffers 225 is typically shallow, and the number of logical write ports required is the number of simultaneous packets available from the link in one clock cycle. For example, in one example, for PCIe Gen5 speeds (32GT / s), up to four packets can arrive in one 1 GHz cycle, so four logical ports are required in such an example to handle these packets simultaneously, assuming that these packets can potentially be of different flow control classes and / or virtual channels. On the other hand, fabric-side buffers 230 (e.g., ProcQ buffers) can be implemented as deep buffers used to hide CPU fabric latencies (e.g., for inbound writes, which translate to latencies of fetching ownership requests and committing data to the coherency domain). They can include one or more write ports.In a split-queue implementation, the SFI semantics may enable further optimizations (e.g., to perform "batching" of transactions on the ProcQ side). Indeed, the SFI semantics have been applied to improve buffer implementations in various system configurations, providing a balance of receiver complexity with bandwidth scaling capabilities, among other exemplary features.

[0045] Among the exemplary features employed in the exemplary improved SFI interface, receiver decode may be simplified, with an interface that scales to support a wide range of data payloads (e.g., from as small as 4B to as large as 4KB (or larger)). The improved streaming interface may allow multiple packets to be delivered in the same cycle, allowing for a scalable interface across a variety of payload sizes while maintaining a common set of semantics and ordering (e.g., PCIe-based, etc.). Configurable parameters may include the number of logical write ports at the receiver (e.g., 1 or 2), which may be supported by defining rules for the interface that limit the number of different packets or headers sent in a clock cycle to the use of a corresponding number of flow control classes and / or virtual channels. Reducing the number of logical write ports at the receiver may result in significant area and complexity savings. Additionally, as noted above, the improved streaming interface may allow header processing to begin at the receiver while data is being streamed (e.g., of headers received over a dedicated header channel) to improve latency (e.g., in the case of a CPU host, helping to overlap ownership request latency onto the incoming data stream).

[0046] Compute Express Link or CXL is a low-latency, high-bandwidth discrete or on-package link that supports dynamic protocol multiplexing (or muxing) of coherency protocols (CXL.cache), memory access protocols (CXL.mem), and I / O protocols (CXL.io). CXL.cache is an agent coherency protocol that supports device caching of host memory, CXL.mem is a memory access protocol that supports device-attached memory, and CXL.io is a PCIe-based non-coherent I / O protocol with enhancements for accelerator support. CXL is intended to provide a rich set of protocols to support a wide range of devices, such as accelerator devices. Depending on the particular accelerator usage model, all or only a subset of the CXL protocols (CXL.io, CXL.mem, CXL.cache) may be enabled to provide a low-latency, high-bandwidth path for the corresponding computing block or device (e.g., accelerator) to access the system.

[0047] As described above, in some implementations, agents utilized to implement the CXL.io protocol may couple to a system fabric utilizing an SFI interface such as those described herein. For example, referring to FIG. 4, a simplified block diagram 400 is shown illustrating an exemplary CXL agent and the fabric coupling to such an agent. FIG. 4 illustrates an exemplary system topology for a port supporting a CXL link 415. For example, the CXL link 415 may couple a CPU host device 405 to another device 410 (e.g., a memory device or an accelerator device). Each agent (on devices 405, 410) may include link layer logic (e.g., 420a-b, 425a-b) to support each of the CXL sub-protocols (e.g., CXL.io, CXL.mem, CXL.cache). For CXL.mem and CXL.cache, a common controller (e.g., 425a-b) may be utilized. For CXL.io, controllers (420a-b) may be provided that are separate from the coherent CXL.mem and CXL.cache protocols. Protocol multiplexing may be facilitated through CXL arbitration / multiplexing logic (e.g., 425a-b implemented in hardware circuitry) that interfaces with the Flex Bus® physical layer (e.g., 430a-b). The Flex Bus may be implemented as a flexible high-speed port that is statically configured to support either PCIe or CXL. The Flex Bus allows either PCIe or CXL protocols to be transmitted over a high-bandwidth off-package link. Protocol selection in the Flex Bus PHYs 430a-b may occur during boot time via auto-negotiation based on the application.

[0048] Continuing with the example of FIG. 4, a first interface type 450a, 450b is used for coherent protocols such as CXL.cache and CXL.mem, where another different wire interface definition (e.g., 205', 205") (e.g., SFI interface) is used for load / store protocols such as PCIe and CXL.io. In one example, the SFI 205', 205" can function as an intermediary interface that makes no assumptions regarding protocol or application specific roles between the transmitter and receiver, providing a scalable streaming interface that can sustain the high bandwidth requirements of load / store protocols (e.g., PCIe, CXL.io, etc.). The SFI does not include a standalone protocol definition, SFI semantics provided to support different protocols that can be mapped to flow control, and virtual channel semantics provided by the SFI definition, among other examples and interface implementations.

[0049] As shown in Figure 4, the system may employ exemplary interfaces 450a, 450b to enable wire sharing in the fabric and achieve wire efficiency around the fabric and agents by allowing different coherent protocols to share common wires. For example, channels of various protocols emanating from agents may be carefully mapped to a minimum set of physical and virtual channels so that the bandwidth and channel isolation requirements of the agents and protocols are met with the lowest total wire count. Interfaces 450a, 450b may map these multiple protocols to a common set of channels and use common flow control and virtualization features on these channels, among other exemplary implementations.

[0050] In some implementations, an improved streaming interface may be implemented that is adapted to support, at least in part, a load / store protocol based on PCIe or PCIe semantics (e.g., PCIe or CXL.io). For example, the supported protocol may utilize a packet format based on a PCIe-defined format. Additionally, the concept of a flow-controlled virtual channel may be extended from the PCIe definition. It should be understood that other additional protocols (e.g., non-PCIe or CXL protocols) may also be supported by such an SFI interface. Indeed, while many of the examples discussed herein refer to PCIe or CXL.io-based protocols and implementations, it should be understood that the principles, features, and solutions discussed herein may be applied more generally to various other streaming or load / store protocols, among other example systems, for example.

[0051] In some implementations, the SFI interface may have separate header (HDR) and data buses or channels, each of which may carry multiple packet headers or payloads simultaneously. Furthermore, formalized rules may be set and employed in the agent's logic to govern how packets are packed / unpacked on the header and data interfaces. For example, an additional metadata channel, or bus, may be provided on the improved interface to carry metadata to enable the receiver to identify how to unpack the header / data transmitted on the separate header and payload data channels, respectively. Through separate, parallel header and data channels, the system (e.g., the CPU host root complex) may enjoy latency benefits, for example, by receiving multiple headers potentially before the corresponding payload is received. This resulting lead time may be used by the system to process the headers and initiate fetching ownership for cache lines for multiple header requests while the data for those requests is still being streamed. This helps stack latencies and reduce buffer residency, among other exemplary features.

[0052] Referring to FIG. 5, a simplified block diagram 500 illustrating an example implementation of an SFI interface is shown. For example, in each instance of an SFI interface, a set of physical lanes (e.g., wires or other conductors) may be provided and assigned to various channels, thereby embodying a logical set of signals defined for the interface and assigned to each physical lane of the interface. Each device may, for example, possess pins (implemented in hardware circuits and / or software) and corresponding SFI logic to implement its end of the interface (transmitter or receiver) and couple to the physical lanes that embody the connections between the transmitter and receiver on the interface. The SFI interface instance may additionally define two channels for transmitting packets or other data transmission messages from the transmitter to the receiver. Specifically, in some implementations, the SFI interface 205 may include a header (HDR) channel that embodies a set of signals (e.g., 505, 515, 520), each of which is a first set of multiple lanes of the interface used to transmit header data for packets. The SFI interface additionally includes a data (DATA) channel embodying another set of signals (e.g., 510, 525, 530) used to transmit payload data for messages that are mapped to an additional set of lanes of the interface 205. The signals of the HDR channel may include a main HDR signal 505 for carrying the header itself, a header metadata signal 515, and a header credit return signal 520 (directed from the receiver to the transmitter). Similarly, the DATA channel may also include a main data signal 510 for carrying the payload data, a data metadata signal 525, and a data credit return signal 530 (also directed from the receiver to the transmitter), among other exemplary signals. In some implementations, the SFI interface 205 may additionally include a global channel or layer (e.g., 550) that includes bidirectional control signals that apply across all physical channels (e.g., HDR and DATA) of the interface.For example, the global channel may carry a set of global control signals that may be used to perform initialization or shutdown of the interface and to communicate control or parameters for the interface, among other features.

[0053] Each of the HDR and DATA channels may carry multiple packets in the same cycle of transmission. Because most load / store protocols may rely on ordering semantics, the SFI assumes an implicit ordering when multiple packets are transmitted in the same cycle. Packets may be ordered, for example, from the least significant position to the most significant position. For example, if TLP0 starts at byte 0 of the header signal 505 and TLP1 starts at byte 16 of the header signal 505, the receiver will consider TLP1 to be ordered after TLP0 when such ordering rules are applied. Transmissions between different clock cycles follow the ordering rules of the associated protocol (e.g., the SFI carries forward all PCIe ordering rules when used for PCIe). In case of link subdivision (e.g., splitting an entire lane of a link into two or more smaller width links (e.g., associated with respective root ports)), different ports from the controller's perspective map to different virtual channels on the SFI. For example, in such cases, an implementation may support multiple port configurations within the same physical block (e.g., implemented as an agent or controller.) In these cases, among other example implementations, the same physical channel of the SFI may be used to transport packets for different ports, with each port mapped to its own set of virtual channels (e.g., one or more virtual channels per port).

[0054] A set of parameters may be defined for an instance of an SFI interface to configure aspects of the instance. For example, metadata signals for HDR and DATA channels may be based on one or more of the configurable parameters. For example, among other example information, the parameters may identify how the metadata signals carry metadata to convey information about the location of different packets in a single transmission. For example, in an SFI, a packet header with data associated with it is transmitted on the HDR channel and the associated data is transmitted separately on the DATA channel. There may be no timing relationship guarantee between the DATA and HDR channel transmissions. The receiver is assumed to track the associated data length for each received header and process only the associated data size. The data size may be transmitted along with the packet header information (e.g., a PCIe implementation using a PCIe packet header format identifies the amount of data in the length field of the PCIe TLP header to indicate how many 4-byte chunks of data are associated with that header). Information in the metadata transmitted through the metadata signal can also be used by the receiver to determine which headers to map to which data (e.g., through a combination of flow control and virtual channel ID), parity information, information about the header format (e.g., header size), among other example information.

[0055] A global layer or channel of signals (e.g., 550) may carry signals that apply across all physical channels of interface 205, such as control signals, vendor-defined signals, and other signals that enable other example functionality. For example, global channel 550 may carry signals that are also used for interface initialization and shutdown (such as in the examples discussed below). Table 1 describes an example implementation of signals of the global channel of an example SFI interface. [Table 1] Table 1: Global layer signals

[0056] The HDR channel carries the header of a request message from a transmitter to a receiver. Various information may be encapsulated in the (protocol-specific) fields of the header transmitted using the HDR channel, including addresses and other protocol level command information. Table 2 describes an example implementation of the signals of the HDR channel of an example SFI interface. [Table 2] Table 2: HDR Layer Fields

[0057] The header size may be a predefined parameter based on the expected or required peak sustained bandwidth of the system. The SFI interface (and corresponding logic) may enforce rules for the HDR channel, such as packet headers starting and ending on the same cycle of transmission. Multiple packet headers may nevertheless be transmitted on the same cycle by transmitting one of the packet headers on a first subset of the header signal lanes and the other packet header on another subset of the header signal lanes. However, the interface may define that the first packet on a valid header transmission starts on the lane of the header signal corresponding to byte 0 of the header field (logically represented by the header signal lanes).

[0058] A header valid signal (hdr_valid) may be asserted to indicate a corresponding valid value on a lane of the header signal. In some implementations, the number of lanes of the header signal may be logically divided into byte-wise subsets (e.g., 16 or 32 bytes of lane width in each subset) corresponding to the size of one of the protocol headers carried on the header signal. Furthermore, each header valid lane may be mapped to one of the subsets to indicate that valid header data is being sent to a corresponding one of the subsets of lanes of the header signal. Additionally, a header metadata signal (hdr_info_bytes) may carry metadata (e.g., aligned with one of the headers carried on the header signal) describing key attributes that may be used by a receiver to decode the corresponding header.

[0059] The DATA physical channel of the SFI interface may be used to carry payload data for all requests that have data associated with it. In the SFI, there may be no explicit timing relationship or requirement between the HDR channel and the associated data carried on the DATA channel. However, the transmitter may include logic to check both HDR channel and DATA channel credits before scheduling either header data on the HDR channel or payload data on the DATA channel. Table 3 describes an example implementation of the signals of the DATA channel of an example SFI interface. [Table 3] Table 3: Fields of the DATA channel

[0060] In an implementation of the SFI interface, payload data may be transmitted on the data signals of the DATA channel according to multi-byte granularity (e.g., 4-byte granularity). Thus, data for any payload may be identified as ending at a particular "chunk" of data (e.g., a particular 4-byte chunk). As an example, if the width of the data signal D is 64 bytes, then the number of potential data end locations is DE=64 / 4=16, where data_end[0] corresponds to data bytes [3:0], data_end[1] corresponds to data bytes [7:4], data_end[DE-1] corresponds to data bytes [D-1:D-4], etc. The start of the data signal (data_start) may utilize the same or different granularity as the end of the data signal. An instance of the SFI interface may be parameterized to support a maximum number of starting DS in a clock cycle (and limit the number of payload starts accordingly). As an example, if the width of the data signal bus D is 64 bytes and an instance of the SFI interface is configured to limit the number of starts in a cycle to 2, then DS=2, effectively splitting the data bus into two 32-byte chunks in which a new payload transmission may begin. For example, in an example where D=62 and DS=2, among other examples, data_start[0] corresponds to a chunk of data starting at data byte[0] and data_start[1] corresponds to a chunk of data starting at data byte

[32] (including examples having lower or higher granularity in the start of data and the end of a data chunk (e.g., DS>2), smaller or larger data bus sizes, etc.).

[0061] In one exemplary implementation of the DATA channel of the SFI interface, the width of the data start signal may be equal to DS, and the signal may effectively act as a mask to identify each corresponding chunk of data on the data signal that corresponds to the start of a respective payload (e.g., aligned on the same clock cycle). Additionally, each data start bit may have an associated data_info_byte signal transmitted therewith that indicates metadata for the corresponding payload. In some implementations, among other exemplary implementations, the data_info_byte is transmitted only once for a given payload (e.g., with the corresponding data start chunk and data_start_bit), while in other cases metadata may be transmitted (e.g., retransmitted) corresponding to all chunks of data in the same payload. In one implementation, the data_info_byte signal may indicate the respective FC ID and VC ID of the corresponding packet (e.g., using 4 bits carrying the FC ID (e.g., data_info_byte[3:0]) and another 4 bits carrying the VC ID (e.g., data_info_byte[7:4])), among other example information, for use by the receiver in processing the data payload transmitted over the data signal bus.

[0062] Unlike HDR channels, in some implementations of the DATA channel, data chunks from the same packet may be transmitted over multiple cycles. For example, the raw data bus width may be implemented as 64B per cycle, allowing a 128B data packet to be transmitted over two clock cycles. In some implementations, when transmission of a payload begins, the transmitter may ensure that all relevant data chunks in the payload are transmitted consecutively from LSB to MSB over consecutive clocks (e.g., without any gaps or bubbles). In some implementations, only one packet of a particular FC ID / VC ID combination may be transmitted on the interface at a time (an FC ID / VC ID combination is reused only after the previous packet using that combination has completed transmission). In some implementations, packets with different FC ID / VC ID combinations, among other examples, may be interleaved on the SFI interface (e.g., a packet of one FC ID / VC ID combination is interrupted to transmit at least a portion of a packet with another FC ID / VC ID combination).

[0063] The granularity of the credits on the DATA channel may also be configurable (e.g., at design compile time) and may correspond to a multiple of N bytes. For example, in one example, the granularity may need to be a multiple of 4 bytes. If the credit granularity is selected to be 16 bytes, among other example implementations, even a 4-byte data packet transmitted uses one 16-byte credit.

[0064] FIG. 6 is a simplified block diagram 600 illustrating an example implementation of a receiver buffer for use with an example SFI interface. In one example, the receiver buffer may be implemented as a linked list with a single write port and shared between two virtual channels (e.g., VC0 and VC1) of one flow control class (FC0). In this example, four headers per cycle may be received on the corresponding SFI interface. The linked list is managed in blocks (e.g., 630a-c) of four header positions at a time. Although the linked list may appear logically contiguous in memory, the physical blocks may be implemented non-contiguous or even in separate storage elements. In one example, all positions in a given block (e.g., 630a) are filled before moving to the next block (e.g., 630b). The buffer is allocated by the receiver one block at a time, and thus the corresponding shared credits may also be block granular. In an implementation, if a set of 4 headers (e.g., 605a-d) in a block (e.g., 630a) is actually composed of separate storage structures, each of these storage columns can be successfully implemented with only a single write port. For example, each of the columns represented in the linked list buffer of FIG. 6 can be physically implemented as a separate buffer / storage element with a respective single write port. Furthermore, timing relief and pipelining potential can be unlocked by using a "block" management of the linked list pointers (e.g., 615, 620, 625), because the next block pointer (in the example of FIG. 6) only needs to be looked up once every 4 headers. In some streaming protocols, the receiver cannot assume only one FC / VC combination per cycle in the general case, and therefore can be designed with multiple write ports (e.g., tails of different FC / VCs can collide in the same storage column).

[0065] As discussed above, the SFI interface (and the corresponding logic and buffer tracker utilized by the sender and / or receiver to implement its half of the interface) may enable pipelining of header processing while data is being streamed. Indeed, the latency savings realized thereby translate directly into buffer savings at the receiver in terms of header processing. In the context of a load / store protocol, the receiver is assumed to internally separate headers and data in any manner, since headers are heavily consumed by the control path, where most of the data is sequestered to the data path. By splitting the header and DATA channels on the exemplary SFI interface, the header of a later request may even bypass the data of an earlier request, which may enable the receiver to start processing the header while the data transmission is being completed. In the context of a host CPU processing inbound (device to host) writes, among other exemplary use cases and benefits, this may provide a head start to acquire ownership of the associated cache line. In fact, since fetching ownership is one of the biggest drivers of latency when processing writes, layering it into the data stream can help reduce overall latency and buffering in the CPU. Deadlock is avoided by ensuring that the sender checks both header and data credits before sending either the header or the data.

[0066] In some implementations, each VC and FC defined for the SFI interface is to use credits to send any message and collect credit return from the receiver. The source may consume the full credit required for the message to complete. The transmitter checks both HDR channel and DATA channel credits before sending the corresponding message on the respective channel to the receiver. The granularity of the HDR and DATA channel credits is predefined between TX and RX. For example, the granularity of the credit on the DATA channel may be configured (e.g., at design compile time) to be only a multiple of N bytes. For example, in one example, the granularity may need to be a multiple of 4 bytes. If the credit granularity is selected to be 16 bytes, even a 4-byte data packet transmitted uses one 16-byte worth of credit, among other example implementations. In one example, the FC ID may be based on PCIe semantics (e.g., 4'h0=posted, 4'h1=non-posted, 4'h2=completed), among other example implementations. Additionally, each of the physical channels (e.g., DATA and HDR) may have a dedicated credit return wire (different from the rest of the channel flows from the receiver to the transmitter). For example, during operation, the receiver returns a credit whenever it processes a message (or guarantees a buffer position for the next transaction).

[0067] In some implementations, the SFI allows two schemes to support sharing of buffers between different FC and VC IDs. In both schemes, the receiver advertises the minimum number of dedicated resources necessary to guarantee forwarding progress. For large packet transmissions, this means that the maximum payload size is based on the dedicated credit advertisement. If shared credits are used, the sender and receiver predetermine which credit type or scheme will be used. In some implementations, this decision may be made at design time. In alternative implementations, among other examples, the credit scheme may be dynamically determined (e.g., based on parameters written to corresponding configuration registers).

[0068] The first of the two schemes for credit sharing may be managed by the sender. In this scheme, the sender is responsible for managing the shared buffer at the receiver. One or more shared credit pools are advertised or consumed with spare VC ID / FC ID encoding. When the sender consumes a shared credit pool credit, it transmits a packet using the corresponding VC ID / FC ID encoding. When the receiver de-allocates a transaction that used the shared credit, it performs a credit return for the corresponding VC / FC ID combination. In some implementations, a bit may be provided in the header (along with a corresponding signal on the HDR channel) to indicate whether the credit is a shared credit or not. Thus, among other examples, the receiver may need to further decode the header packet to explicitly determine the actual VC ID or FC ID of the packet.

[0069] In one exemplary implementation of transmitter-managed credit sharing, an exemplary shared credit pool mapping advertised by the receiver (e.g., in a PCIe-based implementation) may support two VCs on the link and employ the following exemplary mapping shown in Table 4: [Table 4] Table 4: Example encodings for shared credits

[0070] Another of the two credit sharing schemes may be receiver-managed. In the receiver-managed scheme, the receiver is responsible for managing the shared buffer. Only dedicated credits are advertised to the sender. Typically, the advertised dedicated credits cover the point-to-point credit loop across the SFI, and the shared credits are used to cover larger credit loops (e.g., CPU fabric or application layer latency). After a particular FC / VC ID transaction is received and shared credits become available, credits may be returned for that FC / VC ID combination (e.g., without waiting until the transaction is deallocated from the receiver queue). This implicitly grants a shared buffer spot to that FC / VC ID. Internally, the receiver tracks credits returned to the sender based on FC / VC, and also tracks credits currently consumed by the sender. This tracking allows the receiver to ensure that a maximum number of buffers per FC / VC are used. The receiver may guarantee the necessary dedicated resources for forwarding progress guarantees, among other exemplary implementations.

[0071] Error handling in case of abnormal flow control may result in undefined behavior. Therefore, the SFI interface logic on the agent and fabric may check for abnormal cases and trigger assertions in the RTL and also log / signal fatal errors to enable post-silicon debugging. For example, the SFI may maintain consistency between the HDR and data streams, meaning that the transmitter sends data payloads in the same order as it sends the corresponding headers, and vice versa. In some implementations, the receiver logic may include functionality to detect and flag fatal errors for violations, among other example error handling features. In some implementations, the SFI provides for data poisoning to be sent at the end of the data transmission. In case of occasional errors, among other examples, the ownership request may be discarded / written back without correction, or the host may choose to poison the associated cache line and write the updated data.

[0072] Referring to FIG. 7, a representation of an example metadata format 700 that may be carried on a lane of a header metadata signal is shown. The least significant byte and bit are shown to the right. P (705) is the parity bit for the corresponding header. In some implementations, support for parity bits may be optional (e.g., and parity bit 705 treated as an additional reserved bit). When supported, parity may be supported, for example, by XORing at least all of the bits of the packet header. In some implementations, among other examples, both the bits of the associated metadata 700 and the non-parity bits may be XORed to determine parity. Bit D (710) indicates whether the header has corresponding payload data associated with it. All reserved bits (e.g., 715) may be ignored by the receiver or may need to be driven to 0 by the transmitter. In some implementations, it may be necessary for the switch / fabric router to propagate the reserved bits 715 as is without any modification. In some implementations, among other examples, the reserved bits 715 may be utilized for vendor-defined encoding or future information. The header size (HDR SIZE) 725 in the example metadata 700 may specify the size of the header (e.g., in 4-byte granularity). When calculating the header size, the length of the header metadata (700) may be ignored (not considered part of the header).

[0073] In an implementation of an SFI interface, the maximum number of packet headers that can be transmitted in one cycle on the interface can be predetermined (e.g., and recorded in the configurable parameters of the interface). The maximum packet headers per cycle can be determined by the width (or number of lanes) of the header signal (H) and the maximum packet header size. The SFI interface can be implemented (and designed) such that the header width (H) allows a common case usage to sustain maximum throughput. As an example, assuming a common case application header size is 16 bytes (e.g., mapping to a 4 D-word header in PCIe) and the interface sustains two headers per cycle, then H=2*(16)=32 bytes. A corresponding enable signal (and lane) can be included in the HDR channel that corresponds to the desired number of headers per cycle. As an example, if it is desired for an interface to support a maximum of two headers per cycle, a corresponding number of valid lanes M=2 may be defined, supporting one valid signal for each of the two potential headers in a cycle (e.g., hdr_valid[0] corresponds to the header start in byte 0 of the header signal, hdr_valid[1] corresponds to the header start in byte 16 of the header signal). In some cases, one or more of the header formats of the supported protocols may be too large to transmit on only one of the subset of lanes defined in the header signal (and be assigned to a respective one of the valid signal lanes), meaning that such headers may utilize two or more of the subset of lanes in the header signal for transmission (and only the first (least significant bit) of the two or more associated valid signals may be asserted). In such cases, among other examples, when the maximum headers per cycle is set to two, if a larger header format is transmitted on the header signal, only one header may be transmitted in that cycle and hdr_valid[1] is not asserted.

[0074] Continuing with the example of FIG. 7, the header metadata may additionally include information for use in flow control of the header (and associated packets). For example, the metadata may include a virtual channel (VC) identifier (ID) 720 for the header and a flow control class (FC) ID 730 for the header. In some cases, packet ordering may be according to the VC ID and FC ID (e.g., a combination of VC ID and FC ID) of the packet. In some implementations, parameters of the SFI interface may be configured to set a predetermined maximum number of FC and VC ID combinations for the interface that are available for use in any given transmission cycle (e.g., clock cycle) of the interface. This maximum number of FC-VC combinations may be advertised or otherwise set in the interface logic of both the transmitter and receiver (e.g., at design compile time). This maximum may be set to help minimize write ports in the receiver storage, for example, when receiver buffers are shared among supported FCs and / or VCs. As an example, the interface may be parameterized to accept up to two different FC-VC combinations in a cycle, such that in any given cycle, all packet headers transmitted belong to up to two different FCs in the same VC, the same FC for two different VCs, or the same FC-VC combination.

[0075] The transmitter may utilize credits associated with an FC, VC, or FC-VC combination to determine whether a packet can be transmitted over a channel. For example, if a packet header has data associated with it, the packet header is transmitted on the HDR channel and the associated data is transmitted on the DATA channel. Prior to transmitting a header or payload data, the transmitter may check (e.g., keep track of records in a local memory) the available credits for both the header and payload data (and the corresponding HDR and DATA channels) before scheduling the header or payload data transmission. In some implementations, the credit granularity for the header channel may be set to the maximum supported header size. For example, if the maximum supported header size is 20 bytes, one credit on the header channel may correspond to 20 bytes of storage at the receiver. In some cases, one full credit corresponding to the full 20 bytes is consumed even if only a 16-byte header is transmitted, among other exemplary similar alternative flow control and credit implementations.

[0076] Referring to FIG. 8, a simplified timing diagram 800 is shown illustrating an example of header transmission using a header channel of an exemplary SFI interface. The header channel may include a clock lane, one or more lanes dedicated to header valid signals (e.g., 810, 825), lanes dedicated to communication header metadata (e.g., 815, 830), and lanes dedicated to implementing multiple bytes of the header bus (e.g., 820, 835). In the example of FIG. 8, multiple valid signals are provided to govern the transmission of valid header data on corresponding subsections of the header bus. For example, the header lane 810 may carry a valid signal corresponding to the lanes (e.g., 820) implementing bytes 0-15 of the header bus, and the header lane 825 may carry a valid signal corresponding to the lanes (e.g., 835) implementing bytes 16-31 of the header bus. Thus, valid signal 810 may be asserted as long as valid data is transmitted on bytes 0 through 15 of the header bus (e.g., in clock cycles 1, 2, and 4), and similarly, valid signal 825 may be asserted corresponding to valid data transmitted on bytes 16 through 31. In one example, corresponding header data may be transmitted aligned (e.g., on the same clock cycle) as corresponding asserted valid signals, as in Figure 8, while in alternative implementations a delay between assertion of the valid signal and transmission of the header data may be defined, among other example features and implementations.

[0077] Continuing with the example of FIG. 8, subsections of the lanes implementing the header bus may also be associated with respective header metadata (or hdr_info) signals (e.g., 815, 830). For example, header bytes 0-15 (e.g., 820) may be associated with a first header metadata signal 815, and header bytes 16-31 may be associated with a second header metadata signal 830. The header metadata signals may carry data (e.g., 8 bytes) per cycle that describes attributes of the header carried on the corresponding header bus lane. In some cases, both subsections of the header bus may be utilized to carry a larger header, resulting in fewer than the maximum number of headers per cycle being transmitted in a given cycle (e.g., clock cycle 4). When two or more subsections of the header bus are used to transmit a single header, in some implementations, only one of the corresponding metadata signals (e.g., the signal corresponding to the least significant byte of the header) may carry data, while the remaining metadata signals do not carry any metadata. In this manner, among other examples, the receiver may identify that more than one subset of the header bus lanes are used to transmit a single header (e.g., assertion of one or both of the valid signals (e.g., 810, 825) corresponding to the subsection of the header bus used to communicate the header).

[0078] In the particular simplified example of Figure 8, headers of five transaction layer packets (TLPs) are shown to be transmitted over the exemplary SFI header channel. For example, header bus subsections 820, 835 may each carry headers of two separate TLPs in clock cycles 1 and 2 (e.g., headers of TLP0 (840) and TLP1 (845) in cycle 1, and headers of TLP2 (850) and TLP3 (855) in cycle 2). This may be possible based on the header sizes of these respective packets. Additionally, corresponding header metadata (e.g., 865, 870, 875, 880) may be transmitted on corresponding header metadata signals 815, 830 in cycles 1 and 2. Valid signals 810, 825 may be deasserted in cycle 3, causing no additional header data to be transmitted during this cycle.

[0079] In cycle 4, a header for another TLP, TLP4, is transmitted. In this example, the size of the TLP4 header requires transport on both header bus subsections 820, 835 to communicate the header on the HDR channel in a single clock cycle. For example, the headers for TLP0-3 (e.g., 840, 845, 850, 855) may be of size HDR_SIZE=4, while the size of the TLP4 header is HDR_SIZE=5. Thus, in this example, the bytes of the TLP4 header (860a-b) are transmitted on both lanes of the header bus subsections 820 and 835. In this example, only the valid signal 810 corresponding to the subsection (or byte) of the header bus carrying the start (or least significant byte) of the header is asserted high (890), while the other valid signals 825 remain deasserted in clock cycle 4. Similarly, only one of the header metadata signals (e.g., 815) may be used to carry metadata information for the TLP4 header, with the metadata signal (e.g., 830) corresponding to the most significant byte of the header carrying null or other signals. In one example, the headers of TLP0-4 may follow a PCIe-based protocol. In such a case, the TLP Hdr byte follows the format described in the PCI Express Base specification. In this example, among other example implementations, hdr_start[0] is associated with header byte[0], and hdr_start[1] is always associated with header byte

[16] .

[0080] In some implementations, the SFI interface may be implemented as a synchronous interface, where both sides of the interface run on the same clock. Despite this, the transmitter and receiver may not need to coordinate a reset at each respective device. Instead, in some implementations, the initialization flow defined for the interface may define a separate handshake to ensure that the transmitter and receiver exchange information about interface reset and flow control before traffic begins on the interface.

[0081] Referring to Figure 9, a simplified timing diagram 900 is shown illustrating an example of data transmission using the DATA channel of an exemplary SFI interface. In this example, the DATA channel includes a clock 905, a single valid signal 910 (e.g., on a single lane of the channel), and a set of lanes (e.g., 915, 920) that implement one or more subsections of a data bus. In this particular illustrative example of Figure 9, an X-1 subsection is shown. When the valid signal 910 is asserted (e.g., at 945), the data appearing on the data bus (and supporting signals (e.g., 925, 930, 935, 940)) is considered valid. When valid 910 is deasserted (e.g., at 966), the transmission of data on the data bus stops or stalls until valid is reasserted.

[0082] In some implementations of the SFI DATA channel, a start of data (or data_start) signal may be provided that is implemented on a set of lanes to implement a corresponding number of bits of the data_start signal. For example, the data_start signal may be implemented as a bit vector with corresponding data_start lanes (e.g., 925, 926, 928, etc.) that map to respective bytes or spans of bytes in the data bus. For example, each data_start lane (e.g., 925, 926, 928, etc.) may map to a corresponding one of the X+1 subsections of the data bus. For example, in an example where there are eight subsections of the data bus, the start of data signal may be comprised of eight bits or lanes, with each bit mapped to one of the subsections. When the first byte of the payload (e.g., measured from least significant byte to most significant byte) is communicated in a particular clock cycle, the corresponding start of data signal (e.g., 925) may be asserted (e.g., at 954) to identify the subsection (or chunk) of the data bus in which that first payload byte can be found. Through this, the receiver can identify the boundary between two payloads communicated on the channel.

[0083] As with the HDR channel example, the SFI DATA channel may also carry metadata on a dedicated metadata (data_info) signal lane (e.g., 930, 935) to describe the corresponding payload data being transmitted on the data bus. In some implementations, metadata for a payload may be communicated on the DATA channel in association with the start of that payload (e.g., aligned with the first byte of the payload and the corresponding data_start signal). Indeed, multiple metadata signals may be defined and carried on the DATA channel, one corresponding to each of a corresponding number of subsections (e.g., 915, 920) of the data bus. In some implementations, the subsections or chunks may correspond to the same logical chunk utilized in the data_start signal (and / or data_end signal 940). For example, when a particular chunk carries the first byte of a new payload, the corresponding one of the metadata signals (e.g., 930, 935) is responsible for carrying the corresponding metadata for that payload. 9, at clock cycle 1, the start of the payload of TLP0 (950) may begin on a first subsection (e.g., 915) of the data bus. Thus, the data_start signal 925 may indicate that the subsection (e.g., corresponding to byte 0 of the data bus) carries the start of a new payload (954). The other subsection (e.g., 920) of the data bus may be used to communicate additional chunks (e.g., 952) of the same payload. Additionally, one of the metadata signals (e.g., 930) corresponding to the first subsection (e.g., 915) of the data bus may be encoded with metadata (e.g., 956) for the TLP0 payload.

[0084] Continuing with the example of FIG. 9, the payload data of TLP0 (e.g., 950, 952, 960) may continue to be transmitted over multiple clock cycles on the data bus until all have been transmitted. An end of data (or data_end) signal 940 may operate in a similar manner to the start of data signal, with multiple lanes (e.g., 940, 942) being mapped to identify the subsection of the data bus where the last chunk of payload data was transmitted in the corresponding clock cycle. The granularity of the subsection or chunk referenced by the data_end signal may be higher, lower, or the same as that used in the data_start and metadata signals. In one example, among other examples, the subsection or chunk referenced by the data_end signal 940 may be 4 bytes. In the particular illustrative example of FIG. 9, the last byte / bit of the payload of TLP0 is transmitted in subsection "N" of data bus bytes and lanes as measured according to the granularity configured for the data_end signal. Thus, a lane of the data_end signal mapped to subsection N (e.g., 940) may be asserted to identify that subsection N carries the end of the payload. In some implementations, the granularity of the data bus subsections and chunks used by the start of data, end of data, and metadata signals, among other examples, may be configurable through corresponding parameter sets for the SFI interface.

[0085] Continuing with the example of FIG. 9, the second payload of the second packet (e.g., TLP1) can be transmitted on the data bus (e.g., the start of the second packet is transmitted in data chunk / subsection "Y"). In some implementations, the payloads of multiple packets can be transmitted simultaneously on the data bus (e.g., using respective subsections of the data bus). In this example, in clock cycle 2, the payload of TLP0 ends and the payload of TLP1 starts. Thus, both the data end signal (e.g., 970) and the data start signal (e.g., 964) are transmitted in the same cycle, and the data start signal 964 indicates the subsection or chunk of the data bus where the start of the payload appears (e.g., subsection "Y", where 0 < Y < X). In the example of FIG. 9, the start of the data signal is at the 1-byte granularity, specifically, it can identify the byte in the data bus where the payload starts (e.g., byte 0 for the payload of TLP0, byte Y for the payload of TLP1, etc.). Thus, the widths of the data start and data end signals can be based on the respective granularities used. Further, in the example of FIG. 9, the payload of TLP1 starts transmission in clock cycle 2, but the transmission can be temporarily interrupted through the de-assertion of the valid signal 910 (966), and the remaining bytes (e.g., 972, 974) are transmitted when the valid 910 is re-asserted. In other examples, among other exemplary implementations, the valid may need to remain asserted until all bytes of the payload in transit are transmitted. The examples of FIGS. 8 and 9 are provided, among other examples, as simplified, non-limiting examples that are intended to illustrate more general principles (and alternative implementations) that can be implemented in exemplary SFI HDR and DATA channels.

[0086] In some implementations, state machines or other logic may be provided on the agent and fabric devices to participate in the defined connect and disconnect flows in the SFI interface. For example, such flows may be invoked during boot / reset, when entering a low power mode, among other example states or events. In some implementations, the SFI defines an initialization phase in which information about credit availability at the receiver (RX) is communicated to the transmitter (TX) after a connection is established. In some cases, reset may be independently deasserted between the agent and fabric sides of the SFI. For an independent reset, the initialization signal may be driven to the disconnected state (e.g., on a global channel) upon reset, and no traffic may be sent until the initialization reaches the connected state. The disconnect flow may additionally be supported by the agent, for example, to reconfigure credits to achieve power savings. Without this flow, all SFI credits may be configured to their final values ​​before the first connection can proceed.

[0087] At initialization, the transmitter and receiver sides of the SFI interface (e.g., the agent and fabric sides) may come out of reset at approximately the same time or simultaneously. One end of the interface (e.g., after coming out of reset) may have no implicit requirements for when the other end should come out of reset. In some implementations, the SFI may define an explicit handshake during initialization between the agent and fabric to ensure that both endpoints (and all pipeline stages between them) come out of reset before any credits or transactions are sent on the UFI interface. Thus, after reset, the receiver may begin sending credits for use by the transmitter.

[0088] 10 is a diagram 1000 illustrating an example state machine for initialization states in an example implementation of an SFI interface. The states may include a disconnected state 1010 (which may be entered upon reset 1005), a connecting state 1015, a connected state (1020, 1035), a disconnecting state 1025, and a rejecting state 1030. A combination of values ​​of the txcon_req signal, the rxcon ack signal, and the rxdiscon_nack signal may indicate a respective initialization state. As an example, in the disconnecting state 1025, the txcon_req signal may be LOW, the rxcon ack signal may be HIGH, and the rxdiscon_nack may be LOW. Changing a particular one of the signal values ​​may cause a transition from one initialization state to another. For example, as shown in the example state machine of Figure 10, among other examples, in the Disconnecting state 1025, changing the rxcon_ack signal from HIGH to LOW may cause a transition to the Disconnected state 1010, while changing the rxdiscon_nack signal from LOW to HIGH may cause a transition to the Rejected state 1030. In the UFI interface, each initialization state is used to determine the actions to be performed by the receiver and transmitter, such as the example actions described in Table 5 below. [Table 5] Table 5: Initialization State Actions

[0089] Signaling rules may be defined for a global initialization signal set. In one example, a txcon_req signal may be defined such that a transition from 0 to 1 reflects a connection request and a transition from 1 to 0 reflects a disconnection request. Credit return signals may be provided, for example, along with a credit valid (crd_valid) signal and a credit shared (crd_shared) signal. In one example, crd_valid=1 may be defined to mean release dedicated message credits for a protocol ID and virtual channel ID, while crd_shared=1 means release shared credits (this may occur in parallel with dedicated message credit return). In some implementations, credit return behaves in the same manner during the first initialization of credits as during a runtime return of credits. An rx_empty signal indicates that all channel credits returned from the receivers and all receiver queues are empty (although, among other example issues, this may not take into account messages in transit or in intermediate buffers such as clock crossing queues). In some implementations, the transmitter may check rx_empty before initiating a disconnection. Checking increases the probability that the disconnection will be accepted quickly (e.g., when there are no possible in-flight requests not yet registered at the receiver). In some implementations, to further increase the probability of disconnection acceptance, the transmitter may implement a timer delay after the last valid message is sent so that the receiver pipeline has time to drain into the receiver queue, among other example features. In some implementations, during initialization, the transmitter may send messages as soon as any credit is available, without relying on rx_empty assertion. Alternatively, the transmitter may stall sending any packets after initialization until rx_empty is asserted, and the transmitter may use the received credit as an indication of the total credit the receiver has advertised.In an exemplary implementation of the SFI interface, a transmitter may transmit a packet when it receives sufficient credits from a receiver. The transmitter may identify that a packet is to be transmitted and determine that there are sufficient HDR and data credits, respectively, for the packet before starting transmission.

[0090] As a further example of signaling rules that may be defined in a UFI implementation, a connection request may always be followed by a connection ACK. As described above, a connection request may be signaled by a txcon_req transitioning from 0 to 1. This transition serves as an indication that the transmitter Tx is ready to receive credits and is operating normally. An ACK may be signaled by an rxcon_ack transitioning from 0 to 1. The ACK may be stalled for any amount of time until the receiver is ready to complete. Similarly, a disconnection request may be defined followed by a disconnection ACK or NACK. A disconnection request may be signaled by a txcon_req transitioning from 1 to 0. A disconnection ACK may be signaled by an rxcon_ack transitioning from 1 to 0. A disconnection NACK may be signaled by an rxdiscon_nack transitioning from 0 to 1. Among other example policies and implementations, a rule may be defined to require the receiver to respond with either an ACK or a NACK to each disconnection request it receives.

[0091] Referring to Figure 11, an example timing diagram 1100 for initialization of an SFI interface from reset to a connected state is shown. In the particular example shown in Figure 11, an example initialization flow utilizing initialization signals in a global channel of the SFI interface is shown. As shown in Figure 11, the initialization signal set may include a receiver disconnect NACK signal 1110, a receiver connect ACK signal 1115, and a transmitter connect request signal 1120. Additional signals are shown to indicate certain characteristics, including a receiver reset signal 1130 (which causes the agent to enter a reset state), a transmitter reset signal 1135 (which causes the fabric to enter a reset state). A representation of at least one of the SFI channel credit return signal set 1125 (e.g., a credit signal set for one or more of the HDR and DATA channels) is also shown.

[0092] When the transmitter comes out of reset to enter the connected state, it may assert the txcon_req signal 1120 to identify a request to the receiver. Similarly, when the receiver comes out of reset, it waits for a connection request on the txcon_req signal 1120. The assertion of the connection request may be any number of cycles after the reset (e.g., 1130) is asserted. The txcon_req signal 1120 remains asserted until the connection is complete, and is only deasserted as part of a disconnect flow. Upon receiving a connection request on the txcon_req signal 1120, the receiver may assert the rxcon_ack signal 1115 to acknowledge the request. The rxcon_ack signal 1115 may be asserted after receiver and transmitter reset and assertion of the txcon_req signal 1120. The rxcon_ack signal 1115 remains asserted and is first deasserted in the disconnect flow only.

[0093] This sequence may allow the initialization link state 1105 to progress from disconnected to connecting to connected. Upon entering the connected state (and sending the rxcon_ack signal), the receiver may immediately begin returning credits (e.g., on the credit return wire 1125). In fact, the receiver begins returning credits immediately upon assertion of the rxcon_ack signal 1115. Thus, the transmitter (e.g., agent) is ready to accept credit return when it asserts the txcon_req signal 1120 (e.g., at clock cycle x4). Because, for example, the credit return may be observed before the observation of the A2F_rxcon_ack due to intermediate buffers or clock crossings. After receiving the minimum credits to send a packet, the transmitter may begin transmitting a packet or message on the channel. The reconnect flow may be implemented similarly to the connection from reset flow discussed herein; however, among other example implementations, to begin a new credit initialization, the receiver first resets its credit counter to a reset value and the transmitter resets its counter of available credits to zero.

[0094] Referring to FIG. 12, an example timing diagram 1200 is shown illustrating an example disconnection and reconnection flow for an example SFI interface. In this example, the transmitter may deassert the txcon_req signal 1120 at time x3 to expedite the disconnection. In some implementations, the rxdiscon_nack signal 1110 is deasserted before the txcon_req signal 1120 is deasserted to allow the disconnection to proceed. When a disconnection is requested, the transmitter does not send any further messages on any channel (e.g., indicated by CHAN_is_valid bit assertion). Based on initiation of the disconnection flow by the transmitter, the receiver decides whether to acknowledge (ACK) or negatively acknowledge (NACK or reject) the disconnection. To acknowledge the disconnection, the receiver may deassert the rxcon_ack signal 1115 after ensuring that all pipelines are empty (e.g., at clock cycle x4), marking entry into the disconnected state (as reflected by the link status indicator 1105). In some cases, the receiver may also ensure that all credits are returned.

[0095] Diagram 1200 of FIG. 12 illustrates the case where the disconnect request is positively acknowledged by the receiver. FIG. 13 illustrates the opposite example where the receiver responds with a negative acknowledgement (or NACK). For example, to send a negative acknowledgement, the receiver may instead assert the rxdiscon_nack signal 1110 (e.g., at clock cycle x4). A negative acknowledgement may be selected, for example, if the receiver determines that it is unable to drain its pipeline without risking deadlock, among other example reasons. After the NACK, the transmitter reasserts the txcon_req signal 1120 (e.g., at clock cycle x6). Upon observing this valid acknowledgement by the transmitter of the receiver's NACK, the rxdiscon_nack signal 1110 may be deasserted (e.g., as shown at clock cycle x6 in the example of FIG. 13).

[0096] In some implementations, connection and disconnection flows are expected to complete within a few microseconds after initiation. In some implementations, timeouts may be defined, either explicitly or implicitly. For example, a receiver may be configured to respond with an ACK or NACK within a defined or recommended window of time. For example, an agent, fabric, or system (e.g., SoC) may define a timeout or time window to enforce this expectation.

[0097] In some cases, an agent or fabric element may be reset while the SFI interface is in the connected state, resulting in a surprise reset. For example, the defined or recommended flow may be to enter disconnect before resetting. As an example, the rxcon_ack signal may transition from 1 to 0 due to a surprise reset at the receiver side of the link while the transmitter's txcon_req signal has a value of 1. In such a case, the transmitter may force itself into the disconnected state and resume initialization. If this occurs when the transmitter is in an idle state, it may recover without message loss. As another example of a surprise reset, if the txcon_req signal transitions from 1 to 0 due to a surprise reset at the transmitter side of the link while rxcon_ack is 1, the standard disconnect flow may be followed. If this occurs when the receiver is in an idle state, the disconnect should receive an ACK and reach the disconnected state cleanly, provided the transmitter remains in reset. However, if the disconnect is rejected (NACKed) by the receiver, a fatal or abnormal link condition (e.g., an unrecoverable error) may result. In the event of a surprise reset, if traffic is active (eg, not idle), loss of protocol messages may occur, which may be fatal to continued normal operation.

[0098] As discussed above, the SFI interface in a system may be configurable according to various parameters. For example, a set of parameters may be specifically defined according to the use case, characteristics, protocols, and topology of a given system, such as a particular SoC design. Such parameters may, for example, define the maximum number of headers that may be transmitted in a single cycle, the maximum header size, the maximum number of different packet payloads that may be transmitted in a single cycle, among other example parameters. Parameter values ​​may be defined and stored, for example, in configuration registers or other data structures for use and reference by agents and fabric components connected through the interface. Table 6 presents example parameters that may be set in an example SFI interface. [Table 6] Table 6: Supported parameters Non-Flit Mode Tunneling for End-to-End Encryption

[0099] As introduced above, even when a PCIe / CXL.io link trains in non-flit mode (NFM), the SFI may utilize PCIe flit mode (FM) header format and semantics. For example, a PCIe flit mode may define a transaction layer packet (TLP) grammar with (1) zero or more 1 data word (1DW) locally vendor-defined TLP prefix, followed by (2) a TLP header base having a size indicated by the Type[7:0] field, followed by 0-7 DW of orthogonal header content (OHC) indicated by the OHC[4:0] field in the TLP header base. (3) A TLP data payload of 0-1024 DW may follow the TLP header base, followed by (4) a TLP trailer (if present as indicated by the TS[2:0] field in the header base), followed by (5) zero or more 1DW end-to-end suffixes. When a link trains to NFM, an I / O fabric or interconnect using the SFI utilizes a transaction layer to perform FM / NFM conversion to ensure that only FM format is carried over the SFI. However, because not all NFM fields, including NFM reserved fields, have an FM equivalent mapping, this may prevent attempts to encrypt the corresponding data (e.g., according to CXL TLP Integrity and Data Encryption (IDE) or the like). In conventional implementations, NFM-NFM communication through IDE encryption is only available with non-streaming interfaces. In some implementations, logic (e.g., implemented in circuitry implementing the interface) may allow two NFM-trained links to communicate while retaining the benefits of an SFI fabric and not compromising packet integrity while retaining the benefits of cut-through routing and receiver decoding simplicity of the flit mode format, among other example benefits. For example, a flit format extension may be defined to tunnel NFM-specific header information through the FM header structure, enabling NFM-FM-NFM end-to-end encryption.This may include, among other example features, all NFM reserved fields that have no FM equivalent, all NFM non-reserved fields that have no FM equivalent, and new hints to appropriately signal destinations for changes to the decoded packet format.

[0100] 14, a simplified block diagram 1400 of an example topology for implementing an SFI-based fabric 1405 for connecting two PCIe links (e.g., between PCIe endpoint 1410 and PCIe endpoint 1415) trained in non-flit mode (NFM) is shown. Since the SFI packet format follows the PCIe flit mode (FM) packet format, the source (e.g., 1410) is responsible for mapping the NFM header to FM format (e.g., using NFM-to-FM converter 1420) before sending through SFI 205a. Similarly, the destination (e.g., 1415) is responsible for mapping the FM header received through SFI 205b back to NFM format (e.g., using FM-to-NFM converter 1435).

[0101] Conventional devices do not support selective IDE stream for an NFM device (e.g., 1410) that communicates with another NFM device (e.g., 1415) through an SFI-based fabric (e.g., 1405). Converter logic (e.g., 1420, 1425) may be provided in such devices to tunnel all NFM-specific fields through existing SFI functionality, allowing end-to-end encryption without adding decoding complexity to the fabric while still maintaining the streaming benefits of the SFI interface. In one example, an NFM-FM converter circuit (e.g., 1420) identifies all NFM format reserved fields (e.g., as defined in PCIe 6.0 or later), identifies all (e.g., PCIe 6.0) NFM format fields without FM equivalents, and defines SFI format extensions to tunnel the NFM fields to a destination device (e.g., 1415).

[0102] According to one example implementation, Table 7 lists the PCIe 6.0 NFM reserved fields for all formats and prefixes and how they map to the SFI format. [Table 7] Table 7: Mapping of reserved NFM fields to SFI

[0103] Similarly, Table 8 lists the PCIe 6.0 NFM fields for all formats and prefixes that do not have an FM equivalent, and how they map to the SFI format in one example. [Table 8] Table 8: Mapping of NFM fields with no FM equivalent to SFI

[0104] 15A, an example NFM prefix definition used to tunnel the NFM field through the SFI is shown. Table 9 includes a description of the fields shown in the example NFM prefix 1505. [Table 9] Table 9: Description of fields in the NFM prefix

[0105] To tunnel additional bits of the exemplary NFM packet to a destination using the SFI, an NFM prefix (e.g., defined as a new PCIe FM vendor-defined local TLP prefix) may be utilized. For example, when tunneling is supported, an SFI NFM prefix (e.g., 1505) may be inserted after any other local TLP prefixes and before the remainder of the protocol header (e.g., PCIe flit mode base header). In one example, an NFM field 1506 is included in the NFM prefix 1505. When the NFM field is encoded with a value of "1", the Address[1:0] position in the base header has the same definition as the NFM format (e.g., for all format types). Additionally, when NFM=1, the Address Type (AT) bits for the flit are provided from the AT field 1507 of the SFI NFM prefix 1505. However, if the NFM field 1506 is encoded as '0', the base header and OHC format of the flit strictly follows the defined (eg, PCIe) FM format.

[0106] In some implementations, in addition to the definition of a new prefix, NFM / FM translation in the SFI may be further supported through the definition of a new SFI-specific extension of the PCIe 6.0 FM format, which may be used when the NFM field 1506 of the NFM prefix (e.g., when NFM=1) indicates the NFM format of the data. For example, Figures 15B to 15D show example extensions to the PCIe FM definition Orthogonal Header Content (OHC). For example, in Figure 15B, an extension to the PCIe OHC-A4 definition is shown, where one or more bits (e.g., bits 20-21) are reused to carry NFM information, such as an Execution Request (ER) field 1512 from the NFM PASID prefix for the message field, and a Privileged Mode Request (PMR) field 1514 from the NFM PASID prefix for the message field. 15C, an extension to the PCIe OHC-A5 definition is shown where one or more bits (e.g., reserved bit 29) are reused to carry NFM information, such as NFM completion header reserved bit (byte 11, bit 7) 1516. As another example, as shown in FIG. 15D, among other examples (such as those discussed above), an extension to the PCIe OHC-B definition is shown where one or more bits (e.g., reserved bits 7:0) are reused to hold NFM information, such as NFM TPH prefix reserved field (byte 3, bits 7:0) 1522. FIG. 16 shows, among other examples, an exemplary SFI header transmission 1600 consisting of two 64-bit memory write protocol headers 1605, 1610, both of which have inserted NFM prefixes 1505a, 1505b and associated SFI extensions (e.g., 1615, 1620, 1625, 1630, etc.) to a base header 1615 and an OHC-B 1620. Bufferless Shared / Dedicated Credit Gasket

[0107] As introduced above, SFI may support a mix of shared and dedicated credits for communication between received SFI and SFI transmitters over the SFI interface. An SFI receiver that implements a shared buffer and operates with block-sized operations to take advantage of the streaming benefits of SFI may, in some implementations, pair and communicate with an SFI transmitter that cannot share credits and instead relies exclusively on dedicated credits. To address such situations, a store-and-forward approach may be attempted, but both such solutions are costly in terms of area and latency (e.g., store and forward credits that have a size of at least the minimum packet size for all supported combinations of flow control (FC) or virtual channel (VC)). For example, in PCIe, six types of information (e.g., Posted Request Header (PH); Posted Request Data Payload (PD); Non-Posted Request Header (NPH); Non-Posted Request Data Payload (NPD); Completion Header (CplH); and Completion Data Payload (CplD)) may be tracked by flow control for each virtual channel, resulting in six FC / VC combinations for each VC.

[0108] In an improved implementation, a lightweight credit conversion gasket (e.g., implemented in circuitry of a device implementing an SFI interface) may be provided that is bufferless and manages the conversion of shared credit pools from receivers to dedicated credits to the sender while implementing anti-starvation control, and QoS and bandwidth shaping algorithms. For example, an SFI receiver may advertise shared credits to the credit gasket logic. The credit gasket may accumulate credit returns from the receivers and track the credits required by each enabled FC / VC at the sender to achieve a desired level of link utilization. The credit gasket may identify and account for receiver occupancy and throttle dedicated credit returns at FC / VC granularity for anti-starvation protection. By tracking and assigning receiver credits in block sizes, the credit gasket may eliminate storage needs and minimize impact to header / data latency. Link utilization monitoring may also be used to adjust credit assignment and control dynamic bandwidth allocation, reducing receiver storage requirements. Such a solution may be provided as a plug-and-play extension to the receiver to provide flexibility in communicating with agents of varying credit functions while minimizing area, latency costs, and implementation complexity.

[0109] FIG. 17 is a simplified block diagram 1700 illustrating an example topology with an SFI-compliant credit gasket 1715 that may bridge the gap between an example SFI receiver (e.g., on SFI agent B 1710) and an SFI transmitter (e.g., on SFI agent A 1705), where the transmitter only supports dedicated credits (e.g., with a dedicated credit pool for each FC / VC combination implemented on the link) while avoiding the use of costly buffers for storage and forwarding. The credit gasket 1715 in some implementations may be implemented as a separate circuit block from that used to implement the corresponding SFI receiver (e.g., on SFI agent B 1710) and may be coupled to the agent block 1710 using a shared credit interface 1720 based on the SFI. In other implementations, the credit gasket 1715 logic may be implemented in the receiver itself (e.g., enabled / disabled based on the transmitter's ability to communicate through the coupled corresponding SFI interface 1715). The credit gasket 1715 may act as a conversion tool between shared and dedicated credit regimes in the SFI. Thus, the credit gasket 1715 may interface with transmitters that only support dedicated credits (based on the SFI) using a dedicated credit interface 1720. The credit gasket 1715 may accept shared credit returns from corresponding SFI receivers (e.g., 1710), manage the conversion of these shared credit returns to dedicated credit returns, and then distribute the converted dedicated credits to corresponding SFI transmitters (e.g., 1705) to their enabled FC / VC combinations.

[0110] The credit gasket 1715 may expose a programmable interface (e.g., to system software or firmware) that allows the credit gasket to be informed which FC / VC combinations at the SFI transmitter are active and how many dedicated credits should be allocated to each (e.g., credits for HDR and data layers, respectively). The number of dedicated credits allocated may depend on the desired link utilization for each FC / VC and the delay characteristics of the link, among other example considerations. For example, as a simplified example, among other (potentially much more complex) examples, if only one FC / VC is active for an SFI link configured to transmit two headers per cycle with a credit loop latency of four cycles and no additional constraints, the gasket is configured to track eight dedicated credits for that FC / VC.

[0111] To facilitate the conversion of shared credits to dedicated credits, among other example information, credit gasket 1715 may track, for each FC / VC, the credit deficit and pending credit returns for each FC / VC, along with the size of the remaining shared credit pool. A credit "deficit" is the mechanism that credit gasket uses to track how many credits to provide to a given FC / VC, and relates to the difference between the credits currently requested by the FC / VC and what has been allocated. Once the deficit is met, no other credits should be assigned (until another deficit is identified). When the deficit increases for a given FC / VC, credit gasket determines that more credits should be allocated to the FC / VC than were previously or originally allocated. In one example, credit gasket 1715 tracks this per FC / VC deficit by initializing a counter to a programmed value, then decrementing for every credit return and incrementing for every valid SFI header or data transmission for that FC / VC combination.

[0112] 18 is a timing diagram 1800 illustrating an example of the involvement of a credit gasket 1715 disposed between an exemplary transmitter 1705 and receiver 1710, where the transmitter only supports dedicated credits and the credit gasket tracks deficits based on valid SFI transmissions. For example, the transmitter 1705 may transmit a first pair of headers (e.g., 1805, 1810) for FC / VC[1][0] followed by a second pair of headers (e.g., 1815, 1820) for FC / VC[1][2]. The credit gasket detects the transmission of headers 1805, 1810 from SFI A 1705 to SFI B 1710 and counts the number of credits (e.g., 2 credits) relative to those allocated for [1][0] (1825), and similarly identifies the transmission of headers 1815, 1820 and counts the number of credits used (e.g., 2 credits) relative to those allocated for [1][2] (1830). Because no transmission has been detected utilizing header credits for FC / VC[1][1], no adjustment is made to the credit tracker for FC / VC[1][1] (1835). In some implementations, the SFI signal may include a signal to identify whether shared or dedicated credits are used. Because the transmitter 1705 sends a header with information (e.g., in hdr_info_bytes[1] (1840)) indicating that it is using dedicated credits, the credit gasket 1715 may modify the header information before delivery to the receiver 1710 to indicate that shared credits are being used instead (thereby allowing the receiver to operate independently of the dedicated credit regime used by the transmitter or the manipulation of the credit regime by the credit gasket). For example, in one example, a "header uses shared credits" bit in the SFI-defined hdr_info_bytes may indicate whether dedicated or shared credits are being used.In this case, the credit gasket may flip the "header used shared credit" bit from a value indicating "dedicated" (e.g., hdr_info_bytes "0x8140") to another value indicating "shared" (e.g., hdr_info_bytes "0xA140") before forwarding the signal upstream to the SFI receiver 1710, among other examples.

[0113] As described above, the credit gasket 1715 may identify the number of credits initially assigned to each FC / VC combination and continuously monitor whether the FC / VC belongs to a credit deficit. For example, if any FC / VC combination is programmatically determined to be in deficit, the credit gasket may then act to arbitrate the credit assignment by assigning additional block-sized chunks of credit. In the example of FIG. 19A, a scenario is illustrated in which the deficit for an active FC / VC is smaller than the block size, showing that the extra credits are assigned from the shared pool (as tracked by the shared pool counter 1930) to the return pending counter 1925 at 1935. For example, as shown in timing diagram 1900a of FIG. 19A, a receiver (e.g., 1710) may send multiple credit returns (e.g., 1905), and the credit gasket 1715 may identify the credit returns and initially return the credits to the shared credit pool (as tracked using the shared credit pool counter 1925). When a FC / VC's dedicated credit pool is adjusted (along with a corresponding credit deficit counter, e.g., 1920), a credit deficit condition may be identified for the FC / VC by the credit gasket (e.g., at 1918). A credit deficit may occur due to an initialization or adjustment of credit allocations, or due to a transmitter sending a valid packet (and the credit gasket recognizes that its credits should be replenished). Credits from the shared pool may be reallocated to make up for the deficit (e.g., one credit (at 1922)). If any excess credits remain from the block-sized chunks after the deficit is filled, these may be stored in a separate return pending counter 1925 that tracks shared credits that have been assigned to a particular FC / VC (e.g., FC / VC[1][2]) but have not yet been returned to the SFI transmitter 1705.Before arbitrating to consume more shared credit resources, future shortfalls for that particular FC / VC may consume credits from the return pending counter.

[0114] Referring to FIG. 19B, a timing diagram 1900b is shown illustrating an example credit initialization and release using an example credit gasket 1715. For example, during initialization, the SFI receiver 1710 advertises shared credits using SFI reserved FC / VC combinations such as those described in the SFI specification. The credit gasket intercepts this advertisement and uses these returns to increment an internal shared credit pool (1940). For example, in the example of FIG. 19B, an example credit exchange during initialization (e.g., an initial 40 credits (e.g., 4x cycles of "0x10" credit return)) is shown as a dedicated credit, with subsequent releases (e.g., advertising 4 credits in two consecutive cycles to cover an 8 credit shortfall). After initialization, the shared credit return is first incremented into a per-FC / VC accumulator that is intended to collect credits in SFI block-sized chunks. When a block-sized chunk is collected, the gasket checks if there is a shortage of FC / VC. If there is, the block-sized chunk of credits is assigned directly to the FC / VC by adding the credits to its FC / VC return pending count. If there is no shortage, the credits are used to increment an internal shared credit pool using a programmable mapping to convert the FC / VC combination into a shared pool.

[0115] Referring to FIG. 20, a simplified block diagram 2000 is shown illustrating an example logic flow within an implementation of the credit gasket. A first interface 2005 may implement a dedicated credit domain to interface with agents having senders that only support dedicated credit. A second interface 2010 may implement a shared credit domain to interface with agents having receivers that support shared credit (and potentially a mix of shared and dedicated credit). A credit conversion logic 2015 may determine how credits are converted between dedicated and shared credit regimes. The credit conversion logic 2015 may include a credit return arbitration logic 2020, which may be utilized to arbitrate the provisioning of credits from a shared credit pool of the receiver to a dedicated credit pool of potential multiple FC / VC combinations as understood by the sender (and virtualized by the credit gasket). A transmission, such as a header, may be received from the sender in the credit gasket (2025) indicating that dedicated credit is applied. The credit gasket may overwrite such fields (2030) before forwarding them to the receiver over the appropriate channel of the SFI interface (2032), indicating to the receiver that shared credits will be used instead. Additionally, upon receiving a data transmission, depending on the size of the data transmission (and the FC / VC of the data transmission), the credit gasket may update the credit deficit tracked for that particular FC / VC by increasing the deficit by the number of credits used for the data transmission (2035). Based on this update, the credit gasket may determine (2040) whether the FC / VC dedicated pool is in a position of deficit or not.

[0116] Continuing with the example of FIG. 20, the receiver may issue a credit return (2045) that is converted by the credit gasket and forwarded to the sender (2050). When credits are returned by the receiver, a credit return accumulator is incremented based on the number of credits returned. A block size may be determined (e.g., during initialization) for the FC / VC, and the credit gasket may determine (2060) whether the number of credits is greater than or equal to the block size. If the accumulated credits for the FC / VC are greater than the block size, the credit gasket then determines (2040) whether the FC / VC is deficient or not. If the FC / VC is deficient, the returned credits (e.g., above a block size threshold) are used to increment (2065) a credit return pending counter (indicating the number of credits awaiting return for the issued FC / VC). However, if there is no FC / VC shortage, this may indicate an FC / VC in-use condition, and as a result, the credits are instead reallocated to the shared pool (the counter is incremented accordingly (2070)).

[0117] In some implementations, the credit gasket may include a configurable credit return arbitrator 2020 to evaluate the status of the credit gasket counters, arbitrate the issuance of credits to various FC / VC combinations supported by the transmitter, and ensure that the shared credit pool supported by the receiver is dynamically allocated in an efficient manner. For example, the credit return arbitrator 2020 may determine (2085) whether to again issue a credit return to the transmitter from the pending return credit counter (2075) and the shared pool counter (2080), e.g., when an update to an FC / VC's credit deficit counter occurs. The credit return arbitrator 2020 may determine (2050) whether to grant the credit return or, alternatively, to withdraw credits from those allocated to the FC / VC (2035), among other exemplary implementations. The credit return arbiter 2020 may be implemented in hardware circuitry and may provide an interface through which software or firmware can configure the algorithms utilized by the credit return arbiter 2020 in determining how to arbitrate credit returns by the credit gasket, among other exemplary implementations.

[0118] In some implementations, the initialized deficit for an enabled FC / VC should cover at least one full packet (e.g., one header credit, maximum payload size (MPS) size data credit) for deadlock avoidance and may be static or dynamic (e.g., as dictated by software). In the static case, the initialized deficit should be set as high as necessary to achieve the desired link utilization. When multiple FC / VC combinations are active, this may result in larger storage demands for the receiver. This may be exacerbated when activity per FC / VC is not uniform, with some FC / VC combinations experiencing periods of high and low activity. To mitigate this, in some implementations, a credit gasket may dynamically adjust the tracked credit deficit over time depending on FC / VC activity, reducing receiver storage demands and area. This allows dynamic credit and bandwidth allocation among active FC / VCs. In this scenario, the deficit is initialized only to cover MPS-sized packets at reset to allow initial traffic flow. When a link is utilized by an FC / VC (e.g., detected by counting the number of valid packets from a given FC / VC in a configurable time window), the credit gasket may choose to maintain, increase, or decrease the deficit. If the deficit increases due to increased activity from the FC / VC, this may result in credit gasket arbitration assigning more shared pool credit resources to the FC / VC. If the deficit decreases due to reduced activity from the FC / VC, the credit gasket may result in reducing the number of credits assigned to the FC / VC. For example, timing diagram 2100 shown in FIG. 21 illustrates a deficit increase scenario due to increased activity from the FC / VC.For example, based on detecting higher traffic on FC / VC[1][0], the credit gasket may increase the credit allocation by increasing the deficit (2105), thereby allowing a higher number of credits to be returned to and used by the sender.

[0119] If the credit gasket determines that credits were initially over-allocated to a given FC / VC, any credits already released to the FC / VC undergoing the deficit reduction cannot be retrieved by the existing SFI mechanism. If an implementation warrants future packet transmissions from the reduced FC / VC, it may choose to simply track the credit reduction and deduct it from future credit returns. Alternatively, credits may be retrieved on-demand in a different manner using an extension of the SFI. For example, a receiver-transmitter pull may be defined to incorporate a new signal added to the SFI, allowing the receiver to request credits back from the sender, and for the sender to acknowledge the pull (or reduction). In one example, as shown in the example of FIG. 22A, the receiver 1710 may signal (2210) to the transmitter 1705 that it wishes to reclaim the previously advertised credits by asserting a new *crd_rtn_pull signal (e.g., in place of the existing *crd_rtn_valid), reusing the existing SFI credit return wires 2205 (e.g., *fc_id, *vc_id, *value, *ded, etc.). The transmitter 1705 may respond to the pull request 2210 with an acknowledgment (e.g., using signal 2220) if the pull is successful, or with a rejection (e.g., using signal 2225) if it is unsuccessful. Upon a successful acknowledgment (e.g., 2230), the receiver 1710 may reclaim the previously assigned credits and reassign them as necessary. Upon a failed or rejected response, the receiver may choose to retry the pull at a later time if it is unable to reassign the credits and conditions still apply. For example, in the example of Figure 22B, the receiver may request (2235) a credit deallocation that the transmitter 1705 may initially reject (2240), but may later retry (2245). This second credit deallocation request 2245 may then be acknowledged (2250) by the transmitter, allowing the receiver 1710 to then successfully recover the credit(s). Table 10 shows exemplary SFI signals that may be utilized to facilitate receive credit pulls. [Table 10] Table 10: Exemplary Credit Pull Signals

[0120] In one example implementation, the wires used to implement signals 2205, 2220, 2225, etc. may be shared between credit return and credit pull, allowing only one of those two events to occur in any given cycle in such cases (e.g., *crd_rtn_pull and *crd_rtn_valid are mutually exclusive events). As a result, in such implementations, the receiver 1710 may have only one outstanding pull request at a time. In alternative implementations, the deallocation of credits from the FC / VC may instead be initiated by the transmitter, for example, through a transmitter-return credit return signal. For example, a transmitter-return credit return signal may be added to the SFI, allowing the transmitter to initiate credit return to the receiver if it independently detects or predicts a decrease in activity. In some implementations, the transmitter-return credit return signal may mirror the existing RX→TX credit return signal, but in the opposite direction. In this option, among other exemplary implementations, the receiver takes a more passive role in credit collection and relies on the sender to self-monitor. Bufferless Arbiter

[0121] Conventional streaming interfaces, such as those defined in existing versions of the SFI specification, define a one-to-one physical interface for coupling a single transmitter to a single receiver. In some implementations, a one-to-many streaming interface may be implemented using an arbiter or arbitration circuit. A buffer arbiter may be developed to facilitate such an interface, however, the use of a buffer arbiter may have large area and latency due to the store-and-forward architecture. For example, among other complexities, the buffer arbiter stores and forwards MPS-sized credits to allow for a transmitter that can transmit packets whenever credits are available and handle the burst rules for a one-to-many interface. In improved implementations, a bufferless arbiter may instead be utilized to enable the I / O fabric to implement a one-to-many connection using the SFI mechanisms without the use of buffers for store-and-forward. For example, a bufferless arbiter may utilize the established SFI mechanisms of early enable, block, and data interleaving to enable many-to-one lightweight bufferless, time division multiplexing without the need for store-and-forward. Indeed, among other exemplary features, the bufferless arbiter may represent a lightweight solution through a common interface that uses existing SFI interface mechanisms and very small area fabric switches, supporting scalability of system-on-chip (SoC) devices.

[0122] For example, Figure 23 is a simplified block diagram 2300 illustrating an exemplary system in which an SFI arbiter 2305 is provided to enable multiple devices (e.g., 2310, 2315, 2320) while maintaining the streaming benefits of the SFI specification. For example, the arbiter 2305 may be implemented as a bufferless arbiter to establish one-to-many SFI compliant connections, thereby reducing physical wire connections. The arbiter 2305 may use SFI mechanisms to arbitrate between multiple agents (e.g., 2310, 2315, 2320, etc.), avoiding the use of buffers for store and forward. In the example of FIG. 23, the arbiter 2305 establishes upstream connections between SFI agent A 2310 and SFI agent C 2320, and between SFI agent B 2310 and SFI agent C 2320, and downstream connections between SFI agent C 2320 and SFI agent A 2310, and between SFI agent C 2320 and SFI agent B 2315. The arbiter may implement SFI early valid support, block support, and data interleaving support, among other example features. In one example, the arbiter may utilize early valid indications from the SFI agents (e.g., on both the HDR and DATA channels) to implement arbitration requests to use the corresponding SFI physical channels. Block assertions from the arbiter 2305 to the SFI agents on both the HDR and DATA channels may be used to stall transmission of packets until arbitration is won. When arbitrating between simultaneously active data streams from multiple SFI agent transmitters, data interleaving support by the receiver on the DATA channel can be used. By utilizing this combination of features, an exemplary SFI-compliant arbiter eliminates the need for storage by ensuring there are no collisions between different transmitters, while always avoiding any additional latency by utilizing the streaming benefits of SFI.The arbiter 2305 may implement a variety of different arbitration algorithms, which may depend on the quality of service and fairness policies of the application, and in some cases may be configurable to allow different arbitration algorithms to be employed at different times within the system, among other example features.

[0123] 24A-24D, timing diagrams 2400a-d are shown illustrating an example use of a bufferless SFI arbiter circuit (e.g., 2305) to implement a one-to-many SFI interface (e.g., in an SoC, switch, network processing device, etc.). It should be understood that the one-to-two interface illustrated in the illustrations of FIGS. 23-24D is provided as a simplified example to illustrate general principles that may be employed within implementations of bufferless streaming interface arbiters, including arbiters that accommodate multiples of larger interfaces (e.g., one-to-three, one-to-four, one-to-ten, etc.). For example, in the example of FIG. 24A, in an example scenario where traffic in the upstream direction is shown, SFI agent A 2310 and SFI agent B 2315 are sending headers to SFI agent C 2320. In this example, when only SFI agent A 2310 is active and SFI agent B is idle, as indicated by hdr_early_valid signals 2405, 2410, the arbiter grants full bandwidth to SFI agent A. This is accomplished by maintaining the hdr_block signal 2415 for SFI agent B as long as SFI agent A wins arbitration. However, when both agents 2310, 2315 are active (2420), such as indicated through the assertion of both hdr_early_valid signals 2405, 2410, the arbiter alternates between winning grants between the two agents (2425) and alternately sets and clears the hdr_block signals 2430, 2415 for agents 2310, 2315 according to each arbiter grant. This results in efficient utilization of the output stream 2435 to SFI agent C 2320 which is able to service headers from both sources (eg, 2310, 2315).

[0124] In implementations where SFI *_early_valid is used as an arbitration request, this may impose additional requirements on SFI transmitters to be more efficient in early valid assertion, especially closer to the time of actual packet transmission, to risk efficiency loss. For example, FIG. 24B shows an example similar to that of FIG. 24B, but where SFI agent B 2315 behaves greedily or otherwise inefficiently in its hdr_early_valid assertion, leaving its hdr_early_valid signal 2410 asserted without even transmitting any header when given the opportunity (e.g., 2440). This introduces a bubble in the output to SFI agent C, because the cycle granted to SFI agent B is not utilized for packet transmission. To address such situations, in some implementations, the arbiter may include logic to further detect such cases. For example, in one example, the arbiter may maintain a counter that increments every cycle that is granted to an agent and is not utilized to transmit data (e.g., DATA or HDR data). A policy or threshold may be defined such that when the arbiter counts the number of unused cycles (e.g., above a programmable threshold), the arbiter may utilize the counter in conjunction with an arbitration algorithm to bias future arbitration decisions against inefficient agents that have been granted "wasted" cycles to it. For example, in the example of FIG. 24C, the arbiter 2305 may detect that SFI agent B has asserted early validity and been granted an arbitrated transmit cycle to transmit a header (e.g., 2450) and is waiting for these opportunities in a repetitive manner (e.g., failing to transmit a header in successive cycles, failing to transmit a header too many times over a particular period of time, etc.). In response, the arbiter may not grant arbitration to the offending agent (e.g., in this example, SFI agent B 2315) and instead diverts (2455) these cycles to other agents coupled to the arbiter (e.g., SFI agent A 2310).

[0125] Similar principles may be applied to arbitrate one-to-many DATA physical channels using a bufferless arbiter 2305, such as that shown in diagram 2400d of FIG. 24D. For example, each SFI transmitter may provide FC / VC information at the beginning of the stream (e.g., as defined by the SFI specification). Since the transmitter is not aware that the arbiter interleaves streams between multiple sources, the arbiter may track active data stream information and reassert data_start and data_info_bytes (e.g., 2456, 2460) whenever it switches streams. In this example, among other exemplary implementations, the arbiter still uses data_early_valid (e.g., 2405, 2410) as an arbitration request and data_block (e.g., 2430, 2415) to stall transmission, but reasserts data_start and data_info_byte with the original information of the stream whenever it switches between sending data from SFI agent A and SFI agent B.

[0126] It should be noted that the above apparatus, methods, and systems may be implemented in any of the electronic devices or systems described above. As a specific illustration, the figures below provide exemplary systems (e.g., SoCs, computing blocks, fabric blocks, etc.) for utilizing the solutions described herein. As the systems below are described in more detail, different interconnects, use cases, topologies, and applications are disclosed, described, and reviewed from the above description. As will be further readily apparent, the above advancements may be applied to any of those interconnects, fabrics, or architectures, and their composite components.

[0127] It should be noted that the above apparatus, methods, and systems may be implemented in any of the electronic devices or systems described above. For example, the computing platform shown in the examples of Figures 25 and 26 illustrates connections between various computing devices, at least some of which may be implemented using corresponding pass-through connector devices having socket connectors compatible with the two connected devices.

[0128] Referring to Figure 25, an embodiment of a block diagram for a computing system including a multi-core processor is depicted. Processor 2500 includes any processor or processing device, such as a microprocessor, embedded processor, digital signal processor (DSP), network processor, handheld processor, application processor, co-processor, system on chip (SOC), or other device for executing code. In one embodiment, processor 2500 includes at least two cores, namely cores 2501 and 2502, which may include asymmetric cores or symmetric cores (in the illustrated embodiment). However, processor 2500 may include any number of processing elements, which may be symmetric or asymmetric.

[0129] In one embodiment, a processing element refers to hardware or logic that supports a software thread. Examples of hardware processing elements include a thread unit, a thread slot, a thread, a process unit, a context, a context unit, a logical processor, a hardware thread, a core, and / or any other element capable of maintaining a processor state, such as an execution state or an architectural state. In other words, in one embodiment, a processing element refers to any hardware that can be associated with code, such as a software thread, an operating system, an application, or independently other code. A physical processor (or processor socket) generally refers to an integrated circuit that potentially contains any number of other processing elements, such as cores or hardware threads.

[0130] A core often refers to logic located on an integrated circuit capable of maintaining independent architectural states, where each independently maintained architectural state is associated with at least some dedicated execution resources. In contrast to a core, a hardware thread generally refers to any logic located on an integrated circuit capable of maintaining independent architectural states, where the independently maintained architectural states share access to the execution resources. As will be appreciated, the nomenclature boundaries between hardware threads and cores overlap when certain resources are shared and other resources are dedicated to architectural states. However, often cores and hardware threads are viewed by the operating system as individual logical processors, and the operating system can schedule operations on each logical processor separately.

[0131] As shown in FIG. 25, physical processor 2500 includes two cores, namely cores 2501 and 2502. Cores 2501 and 2502 are considered here to be symmetric cores, e.g., cores having the same configuration, functional units and / or logic. In another embodiment, core 2501 includes an out-of-order processor core and core 2502 includes an in-order processor core. However, cores 2501 and 2502 may be individually selected from any type of core, such as native cores, software-managed cores, cores adapted to execute a native instruction set architecture (ISA), cores adapted to execute a translated instruction set architecture (ISA), co-designed cores, or other known cores. In a heterogeneous core environment (e.g., asymmetric cores), some form of translation, such as binary translation, may be utilized to schedule or execute code on one or both cores. However, as further discussed, the functional units shown in core 2501 are described in more detail below. In the illustrated embodiment, the units in core 2502 operate similarly.

[0132] As shown, core 2501 includes two hardware threads 2501a and 2501b, which may also be referred to as hardware thread slots 2501a and 2501b. Thus, in one embodiment, a software entity such as an operating system potentially views processor 2500 as four separate processors, e.g., four logical processors or processing elements capable of simultaneously executing four software threads. As alluded to above, a first thread may be associated with architecture state register 2501a, a second thread may be associated with architecture state register 2501b, a third thread may be associated with architecture state register 2502a, and a fourth thread may be associated with architecture state register 2502b. Here, each of the architecture state registers (2501a, 2501b, 2502a, and 2502b) may also be referred to as a processing element, thread slot, or thread unit, as described above. As shown, architecture state registers 2501a are replicated in architecture state registers 2501b so that separate architecture state / context can be stored for logical processor 2501a and logical processor 2501b. In core 2501, other smaller resources such as instruction pointers and renaming logic in allocator and renamer block 2530 may also be replicated for threads 2501a and 2501b. Some resources such as reorder buffers, ILTB 2520, load / store buffers, and queues in reorder / retirement unit 2535 may be shared through partitioning. In some cases, other resources such as general purpose internal registers, page table base registers, low level data cache and data-TLB 2515, execution units 2540, and portions of out-of-order unit 2535 are fully shared.

[0133] Processor 2500 often includes other resources. These resources may be fully shared or shared through partitioning, or may be dedicated to a processing element for these resources. FIG. 25 shows only an exemplary processor embodiment with exemplary logical units / resources of the processor. Note that the processor may include or omit any of these functional units, as well as any other known functional units, logic, or firmware not shown. As shown, core 2501 includes a simplified typical out-of-order (OOO) processor core. However, different embodiments may utilize an in-order processor. The OOO core includes a branch target buffer 2520 for predicting branches to be executed / taken, and an instruction translation buffer (I-TLB) 2520 for storing address translation entries for instructions.

[0134] The core 2501 further includes a decode module 2525 coupled to the fetch unit 2520 for decoding the fetched elements. In one embodiment, the fetch logic includes separate sequencers associated with each of the thread slots 2501a, 2501b. Typically, the core 2501 is associated with a first ISA that defines / specifies instructions executable on the processor 2500. Often, the machine code instructions that are part of the first ISA include a portion of the instruction (called an opcode) that references / specifies the instruction or operation to be performed. The decode logic 2525 includes circuitry that recognizes these instructions from their opcodes and passes the decoded instructions to the pipeline for processing as defined by the first ISA. For example, as discussed in more detail below, in one embodiment, the decoder 2525 includes logic designed or adapted to recognize specific instructions, such as transactional instructions. Recognition by the decoder 2525 results in the architecture or core 2501 taking specific predefined actions to perform the tasks associated with the appropriate instruction. It is important to note that any of the tasks, blocks, operations, and methods described herein may be executed in response to a single or multiple instructions, some of which may be new or old. Note that in one embodiment, multiple decoders 2526 recognize the same ISA (or a subset thereof). Alternatively, in a heterogeneous core environment, decoder 2526 recognizes a second ISA (a subset of the first ISA, or a separate ISA).

[0135] In one example, the allocator and renamer block 2530 includes an allocator for reserving resources such as a register file for storing instruction processing results. However, in some cases, threads 2501a and 2501b are capable of out-of-order execution and the allocator and renamer block 2530 also reserves other resources such as a reorder buffer for tracking instruction results. The unit 2530 may also include a register renamer for renaming program reference registers / instruction reference registers to other registers within the processor 2500. The reorder / retirement unit 2535 includes components such as the reorder buffer, load buffer, etc., mentioned above, to support out-of-order execution and subsequently store buffers to support in-order retirement of instructions executed out-of-order.

[0136] In one embodiment, scheduler and execution units block 2540 includes a scheduler unit for scheduling instructions / operations on the execution units. For example, floating point instructions are scheduled on ports of execution units that have an available floating point execution unit. Register files associated with these execution units are also included for storing information instruction processing results. Exemplary execution units include floating point execution units, integer execution units, jump execution units, load execution units, store execution units, and other known execution units.

[0137] A low level data cache and data translation buffer (D-TLB) 2550 is coupled to the execution units 2540. The data cache stores recently used / manipulated elements, such as data operands, which are potentially kept in a memory coherent state. The D-TLB is for storing recent virtual / linear to physical address translations. As a specific example, the processor may include a page table structure for dividing physical memory into multiple virtual pages.

[0138] Here, cores 2501 and 2502 share access to a higher level or further out cache, such as a second level cache associated with on-chip interface 2510. Note that "higher level" or "further out" refers to cache levels going up from the execution units or further away from the execution units. In one embodiment, the higher level cache is the last cache in the memory hierarchy on processor 2500, such as a last level data cache, i.e., a second or third level data cache. However, the higher level cache is not so limited, as it may be associated with or include an instruction cache. Rather, a trace cache, which is a type of instruction cache, may be coupled after decoder 2525 to store traces that were decoded some time ago. Here, instruction potentially refers to a macroinstruction (e.g., a general instruction recognized by multiple decoders), which may be decoded into multiple microinstructions (multiple micro-operations).

[0139] In the illustrated configuration, the processor 2500 also includes an on-chip interface module 2510. Historically, a memory controller, described in more detail below, has been included in a computing system external to the processor 2500. In this scenario, the on-chip interface 2510 is for communicating with devices external to the processor 2500, such as a system memory 2575, a chipset (which often includes a memory controller hub for connecting to the memory 2575 and an I / O controller hub for connecting to peripheral devices), a memory controller hub, a northbridge, or other integrated circuits. Also, in this scenario, the bus 2505 can include any known interconnect, such as a multi-drop bus, a point-to-point interconnect, a serial interconnect, a parallel bus, a coherent (e.g., cache coherent) bus, a hierarchical protocol architecture, a heterogeneous bus, and a GTL bus.

[0140] Memory 2575 may be dedicated to processor 2500 or shared with other devices in the system. Common examples of types of memory 2575 include DRAM, SRAM, non-volatile memory (NV memory), and other known storage devices. It should be noted that devices 2580 may include a graphics accelerator, graphics processor, or graphics card coupled to a memory controller hub, data storage coupled to an I / O controller hub, a wireless transceiver, a flash device, an audio controller, a network controller, or other known devices.

[0141] However, nowadays, as more logic and devices are integrated on a single die, such as a SOC, each of these devices may be integrated on the processor 2500. For example, in one embodiment, a memory controller hub is provided in the same package and / or die as the processor 2500. Here, a portion of the core 2510 (the on-core portion) includes one or more controllers for interfacing with other devices, such as memory 2575 or a graphics device 2580. Configurations including interconnects and controllers for interfacing with such devices are often referred to as on-core (or un-core configurations). As an example, the on-chip interface 2510 includes a ring-type interconnect for on-chip communication and a high-speed serial point-to-point link 2505 for off-chip communication. However, in a SOC environment, much more devices can be integrated on a single die or integrated circuit, such as network interfaces, co-processors, memory 2575, graphics processor 2580, and any other known computing device / computer interface, to provide a small form factor with high functionality and low power consumption.

[0142] In one embodiment, the processor 2500 may execute compiler, optimizer, and / or translator code 2577 to compile, translate, and / or optimize application code 2576 to support or interface with the apparatus and methods described herein. A compiler often includes a program or set of programs for converting source text / code into target text / code. Typically, compiling program / application code with a compiler is done in multiple phases and passes to convert high-level programming language code into low-level machine or assembly language code. However, for simple compilation, a single pass compiler may be used. A compiler may use any known compilation technique and perform any known compiler operations such as lexical analysis, preprocessing, parsing, semantic analysis, code generation, code transformation, and code optimization.

[0143] Larger compilers often have multiple phases, but in most cases these multiple phases are contained within two general phases: (1) a front-end, e.g., typically where semantic processing and some transformations / optimizations may occur, and (2) a back-end, e.g., typically where analysis, transformations, optimizations, and code generation occur. Some compilers refer to a middle, which describes a blurring of the description between the front-end and back-end of the compiler. As a result, references to insertion, association, generation, or other operations of the compiler may occur in any of the above phases or passes, as well as any other known phases or passes of the compiler. As an illustrative example, a compiler may potentially insert operations, calls, functions, etc. during one or more phases of compilation (e.g., inserting a call / operation in the front-end phase of compilation and then translating the call / operation into lower-level code during the translation phase). Note that compiler code or dynamic optimization code may insert such operations / calls during dynamic compilation and optimize the code for execution during run-time. As a specific illustrative example, binary code (already compiled code) may be dynamically optimized during run-time, where program code may include dynamically optimized code, binary code, or a combination thereof.

[0144] Similar to a compiler, a translator, such as a binary translator, statically or dynamically translates code to optimize and / or translate the code. Thus, a reference to running code, application code, program code, or other software environment may refer to (1) dynamically or statically running a compiler program, optimizing code optimizer, or translator to compile program code, maintain software structure, perform other operations, optimize code, or translate code, (2) running main program code, such as optimized / compiled application code, that includes operations / calls, (3) running other program code, such as libraries, associated with the main program code to maintain software structure, perform other software-related operations, or optimize code, or (4) any combination of (1)-(3).

[0145] 26, a block diagram of a second system 2600 is shown in accordance with an embodiment of the present disclosure. As shown in FIG. 26, the microprocessor system 2600 is a point-to-point interconnect system and includes a first processor 2670 and a second processor 2680 coupled via a point-to-point interconnect 2650. Each of the processors 2670 and 2680 may be any version of a processor. In one embodiment, 2652 and 2654 are part of a serial, point-to-point coherent interconnect fabric, such as CXL, QPI, or UPI, among other examples.

[0146] Although shown with only two processors 2670, 2680, it should be understood that the scope of the disclosure is not so limited. In other embodiments, there may be one or more additional processors within a given processor.

[0147] Processors 2670 and 2680 are shown to include integrated memory controller units 2672 and 2682, respectively. Processor 2670 also includes point-to-point (PP) interfaces 2676 and 2678 as part of its bus controller unit, and similarly, second processor 2680 includes PP interfaces 2686 and 2688. Processors 2670, 2680 may exchange information using PP interface circuits 2678, 2688 via point-to-point (PP) interface 2650. As shown in FIG. 26, IMCs 2672 and 2682 couple the processors to respective memories, i.e., memories 2632 and 2634, which may be portions of main memory locally attached to the respective processors.

[0148] The processors 2670, 2680 exchange information with the chipset 2690 via separate PP interfaces 2652, 2654 using point-to-point interface circuits 2676, 2694, 2686, 2698, respectively. The chipset 2690 also exchanges information with the high performance graphics circuit 2638 via interface circuit 2692 through the high performance graphics interconnect 2639.

[0149] A shared cache (not shown) may be included within either processor or external to both processors, but connected to the processors via the PP interconnect, thereby allowing local cache information of either processor or both processors to be stored in shared memory when the processors are placed into a low power mode.

[0150] Chipset 2690 may be coupled to a first bus 2616 via an interface 2696. In one embodiment, the first bus 2616 may be a bus such as a Peripheral Component Interconnect (PCI) bus, or a PCIe bus or another third generation I / O interconnect bus, although the scope of the disclosure is not so limited.

[0151] As shown in FIG. 26, various I / O devices 2614 are coupled to a first bus 2616 along with a bus bridge 2618, which couples the first bus 2616 to a second bus 2620. In one embodiment, the second bus 2620 includes a low pin count (LPC) bus. In one embodiment, various devices are coupled to the second bus 2620, including, for example, a keyboard and / or mouse 2622, a communication device 2627, and a storage unit 2628, such as a disk drive or other mass storage device, which often includes instructions / code and data 2630. Additionally, an audio I / O 2624 is shown coupled to the second bus 2620. It should be noted that other architectures are possible. The included component architectures and interconnect architectures are various. For example, instead of the point-to-point architecture of FIG. 26, a system may implement a multi-drop bus or other architecture.

[0152] A computing system may include various combinations of components. These components may be implemented as ICs, portions thereof, discrete electronic devices, or other modules, logic, hardware, software, firmware, or combinations thereof that are adapted in a computer system, or as components that are otherwise integrated into the chassis of a computer system. However, it should be understood that in other implementations, some of the components shown may be omitted, additional components may be present, and different arrangements of the components shown may be made. As a result, the solutions described above may be implemented in any portion of one or more of the interconnects shown or described herein.

[0153] In one embodiment, the processor includes a microprocessor, a multi-core processor, a multi-threaded processor, an ultra-low voltage processor, an embedded processor, or other known processing elements. In the implementation shown, the processor acts as a main processing unit and a central hub for communicating with many of the various components of the system. As an example, the processor is implemented as a system on a chip (SoC). As a specific illustrative example, the processor includes an Intel® Architecture Core®-based processor, such as an i3, i5, i7, or another such processor available from Intel Corporation. However, it should be understood that other low power processors, such as those available from Advanced Micro Devices, Inc. (AMD) of Sunnyvale, Calif., MIPS-based designs from MIPS Technologies, Inc. of Sunnyvale, Calif., ARM-based designs licensed from ARM Holdings, Ltd. or its customers or its licensees or adopters, may alternatively be present in other embodiments, such as the Apple A5 / A6 processor, the Qualcomm Snapdragon processor, or the TI OMAP processor. It should be noted that many customer versions of such processors are modified and varied, however, they may support or recognize a particular instruction set that executes a defined algorithm described by the processor licensor, where the microarchitecture implementation may vary, but the architectural features of the processor are typically consistent. To provide an illustrative example, specific details regarding the architecture and operation of the processor in one implementation are discussed further below.

[0154] In one embodiment, the processor communicates with the system memory. As an illustrative example, the embodiment may be implemented via multiple memory devices to provide a given amount of system memory. By way of example, the memory may be compliant with a JEDEC low power double data rate (LPDDR) based design, such as the current LPDDR2 standard compliant with Joint Electronic Equipment Engineering Council (JEDEC) JESD 209-2E (published in April 2009), or the next generation LPDDR standard called LPDDR3 or LPDDR4, which will provide extensions to LPDDR2 to increase bandwidth. In various implementations, the individual memory devices may be in different package types, such as single die package (SDP), dual die package (DDP), or quad die package (13P). In some embodiments, these devices are soldered directly onto the motherboard to provide a low profile solution, while in other embodiments, these devices are configured as one or more memory modules that will be coupled to the motherboard with a given connector. Of course, other memory implementations are possible, such as other types of memory modules, such as different kinds of dual in-line memory modules (DIMMs), including, but not limited to, microDIMMs, MiniDIMMs, etc. In certain exemplary embodiments, the memory may be configured as a DDR3LM package or LPDDR2 or LPDDR3 memory sized between 2 GB and 16 GB and soldered onto a motherboard via a ball grid array (BGA).

[0155] Mass storage may also be coupled to the processor to provide persistent storage of information such as data, applications, one or more operating systems, and the like. In various embodiments, this mass storage may be implemented via SSDs to enable thinner and lighter system designs and improve system responsiveness. However, in other embodiments, mass storage may be implemented primarily using hard disk drives (HDDs), with a smaller amount of SSD storage acting as an SSD cache, allowing non-volatile storage of context state and other such information during a power down event, resulting in faster power up upon resumption of system activity. A flash device may be coupled to the processor, for example, via a serial peripheral interface (SPI). This flash device may provide non-volatile storage of system software, including basic input / output software (BIOS) and other firmware of the system.

[0156] In various embodiments, the system's mass storage is implemented exclusively by SSDs or as disk drives, optical drives, or other drives with SSD cache. In some embodiments, the mass storage is implemented as SSDs or as HDDs in conjunction with a restore (RST) cache module. In various implementations, the HDDs provide between 320 GB and 4 terabytes (TB) of storage and more, while the RST cache is implemented using SSDs having capacities between 24 GB and 256 GB. However, such SSD caches may be configured as single level cache (SLC) or multi-level cache (MLC) options to provide an appropriate level of responsiveness. In the SSD-only option, the modules may be housed in various locations, for example, in an mSATA or NGFF slot. By way of example, the SSDs have capacities ranging from 120 GB to 1 TB.

[0157] While the present disclosure has been described with respect to a limited number of embodiments, those skilled in the art will recognize numerous modifications and variations therefrom, and it is intended that the appended claims cover all such modifications and variations as fall within the true spirit and scope of the present disclosure.

[0158] A design may go through various stages from creation to simulation to fabrication. Data representing the design may represent the design in multiple ways. First, the hardware may be represented using a hardware description language or another functional description language, as is useful in simulation. In addition, a circuit level model with logic and / or transistor gates may be generated at some stages of the design process. Furthermore, most designs reach a data level at some stage that represents the physical placement of various devices in the hardware model. If conventional semiconductor fabrication techniques are used, the data representing the hardware model may be data specifying the presence or absence of various features on different mask layers of the mask used to fabricate the integrated circuit. In any representation of the design, the data may be stored in the form of any machine-readable medium. Memory, or magnetic or optical storage such as a disk, may be machine-readable media that store such information transmitted via light or electric waves that are modulated or otherwise generated to transmit the information. When an electrical carrier wave indicating or carrying the code or design is transmitted, a new copy is made to the extent that copying, buffering or transmission of the electrical signal is performed. Thus, a communications provider or network provider may store, at least temporarily, items such as information encoded in a carrier wave on a tangible, machine-readable medium to embody the techniques of embodiments of the present disclosure.

[0159] A module as used herein refers to any combination of hardware, software, and / or firmware. As an example, a module includes hardware such as a microcontroller associated with a non-transitory medium that stores code adapted to be executed by the microcontroller. Thus, in one embodiment, reference to a module refers to hardware specifically configured to recognize and / or execute code held on the non-transitory medium. Furthermore, in another embodiment, the use of a module refers to a non-transitory medium that includes code that is specifically adapted to be executed by a microcontroller to perform a number of predefined operations. As may be expected, in yet another embodiment, the term module (in this example) may refer to a combination of a microcontroller and a non-transitory medium. Often, the boundaries of multiple modules that are shown as separate typically vary and potentially overlap. For example, a first module and a second module may share hardware, software, firmware, or a combination thereof, while potentially maintaining some independent hardware, software, or firmware. In one embodiment, the use of the word "logic" includes hardware such as transistors, registers, or other hardware such as programmable logic devices.

[0160] In one embodiment, the use of the terms "in" or "configured" refers to constructing, combining, manufacturing, offering for sale, importing, and / or designing an apparatus, hardware, logic, or element to perform a specified or determined task. In this example, an apparatus or element thereof that is not in operation is still "configured" to perform said specified task if it is designed, coupled, and / or interconnected to perform said specified task. As a purely illustrative example, a logic gate may provide a 0 or a 1 during operation. However, a logic gate that is "configured" to provide an enable signal to a clock does not include any underlying logic gate that may provide a 1 or a 0. Instead, the logic gate is coupled in some manner such that a 1 or 0 output enables the clock during operation. Again, note that the use of the term "configured" does not require operation, but instead focuses on the hidden state of the apparatus, hardware, and / or element. In the hidden state, the apparatus, hardware, and / or element is designed to perform a particular task when the apparatus, hardware, and / or element is in operation.

[0161] Additionally, in one embodiment, use of the terms "enable" or "operable" refers to any device, logic, hardware, and / or element designed to enable the device, logic, hardware, and / or element to be used in a specified manner. It should also be noted that in one embodiment, use of the terms "enable" or "operable" refers to the hidden state of the device, logic, hardware, and / or element, where the device, logic, hardware, and / or element is not operational but is designed to enable the device to be used in a specified manner.

[0162] A value, as used herein, includes any known representation of a number, state, logic state, or binary logic state. Often the use of logic levels, logic values, or logic values, also referred to as "ones" and "zeros," simply represents a binary logic state. For example, a one refers to a high logic level and a zero refers to a low logic level. In one embodiment, a storage cell, such as a transistor cell or a flash cell, may be capable of holding a single logic value or multiple logic values. However, other representations of values ​​in computer systems are used. For example, the decimal number 10 may be represented as the binary value 2510, or in hexadecimal as the letter A. Thus, a value includes any representation of information that can be held in a computer system.

[0163] Further, a state may be represented by a value or a portion of a value. As an example, a first value, such as a logical one, may represent a default or initial state, while a second value, such as a logical zero, may represent a non-default state. Additionally, in one embodiment, the terms reset and set refer to default and updated values ​​or states, respectively. For example, a default value may include a potential high logical value, e.g., reset, whereas an updated value may include a potential low logical value, e.g., set. It should be noted that any combination of multiple values ​​may be utilized to represent any number of states.

[0164] The above-described methods, hardware, software, firmware, or code embodiments may be implemented via instructions or codes stored on a machine-accessible, machine-readable, computer-accessible, or computer-readable medium executable by a processing element. A non-transitory machine-accessible / readable medium includes any mechanism that provides (e.g., stores and / or transmits) information in a form readable by a machine, such as a computer or electronic system. For example, non-transitory machine-accessible media include random access memory (RAM), such as static RAM (SRAM) or dynamic RAM (DRAM), ROM, magnetic or optical storage media, flash memory devices, electrical storage devices, optical storage devices, acoustic storage devices, other types of storage devices for holding information received from a transitory (propagated) signal (e.g., carrier wave, infrared signal, digital signal), and the like, and these non-transitory machine-accessible media should be distinguished from non-transitory media that can receive information from these media.

[0165] The instructions used to program the logic to execute the embodiments of the present disclosure may be stored in a memory in the system, such as a DRAM, cache, flash memory, or other storage. Additionally, the instructions may be distributed over a network or using other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to tangible machine-readable storage such as floppy disks, optical disks, compact disks, read-only memories (CD-ROMs), and magnetic-optical disk read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memories, or the like used in transmitting information over the Internet via electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Thus, a computer-readable medium includes any type of tangible machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0166] The following examples relate to embodiments according to the present specification. Example 1 is a protocol circuit for implementing an input / output (I / O) interconnect protocol, wherein the I / O interconnect protocol includes a flit mode and a non-flit mode, wherein a set of flit mode header formats is used when in the flit mode, and a set of non-flit mode header formats is used when in the non-flit mode, the set of non-flit mode header formats including one or more non-flit mode fields; and An interface circuit for implementing an interface for coupling to a fabric, said interface circuit comprising: determining that a link is to be trained for the non-flit mode; generating a header according to the set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, and where one or more fields of the set of flit mode header formats are reused in the header to carry the one or more non-flit mode fields; and transmitting the header over the interface; The device comprises:

[0167] Example 2 includes the subject matter of example 1, where one or more non-flit mode fields are not included in the set of flit mode header formats.

[0168] Example 3 includes the subject matter of any one of Examples 1-2, where the I / O interconnect protocol includes a load / store interconnect protocol.

[0169] Example 4 includes the subject matter of example 3, wherein the I / O interconnect protocol includes one of a Peripheral Component Interconnect Express (PCIe) based protocol or a Compute Express Link (CXL) based protocol.

[0170] Example 5 includes the subject matter of any one of Examples 1-4, wherein the interface: a header channel implemented on a first subset of a plurality of physical lanes, the first subset of lanes including a first lane for carrying packet headers based on the interconnect protocol and a second lane for carrying metadata about the packet headers; and a DATA channel implemented on a separate second subset of the plurality of physical lanes, the second subset of lanes including a third lane for carrying packet payloads and a fourth lane for carrying metadata about the packet payloads; Including, Here, the header is transmitted over the header channel.

[0171] Example 6 includes the subject matter of any one of examples 1-5, wherein the flit mode and the non-flit mode are based on a PCIe-based protocol.

[0172] Example 7 includes the subject matter of example 6, wherein the one or more non-flit mode fields are conveyed in the one or more fields of the set of flit mode header formats based on a mapping.

[0173] Example 8 includes the subject matter of any one of Examples 6-7, wherein the set of flit mode header formats includes one or more orthogonal content headers, and a particular field in a particular one of the one or more orthogonal content headers holds a particular field in the one or more non-flit mode fields.

[0174] Example 9 includes the subject matter of any one of Examples 6-8, where the set of flit mode header formats includes one or more prefixes, and a particular field in a particular one of the one or more prefixes holds a particular field in the one or more non-flit mode fields.

[0175] Example 10 includes the subject matter of example 9, where the particular prefix includes a mode field to indicate that the corresponding packet originated as a non-flit mode packet.

[0176] Example 11 includes the subject matter of any one of Examples 1-10, where end-to-end encryption is provided on the link based on a flit mode.

[0177] Example 12 includes the subject matter of any one of Examples 1-11, where the interface is based on the Streaming Fabric Interface (SFI) specification.

[0178] Example 13 includes identifying a header of a packet, where the header of the packet is based on a non-flit mode format of a load / store interconnect protocol, the load / store interconnect protocol further defining a flit mode; generating a flit mode version of the header of the packet, where the flit mode version of the header of the packet is based on a flit mode format, where a first subset of fields in the non-flit mode format are also provided in the flit mode format and a second subset of fields in the non-flit mode format are excluded in the flit mode format, where generating the flit mode version of the header of the packet includes carrying one or more fields of the second subset of fields in reuse fields defined in the flit mode format; transmitting the flit mode version of the header of the packet over an interface to a fabric, where the flit mode version of the header of the packet is transmitted on a header channel implemented on a first plurality of physical lanes; and transmitting payload data of the packets to the fabric through the interface, wherein the payload data of the packets is transmitted on a DATA channel implemented on a separate second plurality of physical lanes. This is a method for providing the above.

[0179] Example 14 includes the subject matter of Example 13, where the interface is defined according to the SFI specification and the load / store protocol includes one of PCIe or CXL.io.

[0180] Example 15 includes the subject matter of any one of Examples 13-14, wherein the flit mode version of the header of the packet is transmitted on a first subset of the first plurality of physical lanes, and the method includes transmitting header metadata over the interface using a second subset of the second plurality of physical lanes of the header channel.

[0181] Example 16 includes the subject matter of any one of Examples 13-15, wherein the interface: a header channel implemented on a first subset of a plurality of physical lanes, the first subset of lanes including a first lane for carrying packet headers based on the interconnect protocol and a second lane for carrying metadata about the packet headers; and a DATA channel implemented on a separate second subset of the plurality of physical lanes, the second subset of lanes including a third lane for carrying packet payloads and a fourth lane for carrying metadata about the packet payloads; Including, Here, the header is transmitted over the header channel.

[0182] Example 17 includes the subject matter of any one of Examples 13-16, where the flit mode and the non-flit mode are based on a PCIe-based protocol.

[0183] Example 18 includes the subject matter of example 17, wherein the second subset of fields is retained in one or more fields of the set of flit mode header formats based on the mapping.

[0184] Example 19 includes the subject matter of any one of Examples 17-18, where the set of flit mode header formats includes one or more orthogonal content headers, and a particular field in a particular one of the one or more orthogonal content headers holds a particular field in a second subset of fields.

[0185] Example 20 includes the subject matter of any one of Examples 17-19, where the set of flit mode header formats includes one or more prefixes, and a particular field in a particular one of the one or more prefixes holds a particular field in the second subset of fields.

[0186] Example 21 includes the subject matter of example 20, wherein the particular prefix includes a mode field to indicate that the corresponding packet originated as a non-flit mode packet.

[0187] Example 22 includes the subject matter of any one of examples 13-21, further including providing end-to-end encryption based on a flit mode.

[0188] Example 23 is a system including means for carrying out the method according to any one of Examples 13-22.

[0189] Example 24 is a system comprising a fabric; and a plurality of compute blocks communicatively coupled through the fabric, where a particular compute block in the plurality of compute blocks comprises agent circuitry for supporting a load / store interconnect protocol; and interface circuitry for implementing an interface for coupling to the fabric, where the interface circuitry determines that a link is trained for non-flit mode; generates a header according to a set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, one or more fields of the set of flit mode header formats being reused in the header to carry one or more non-flit mode fields; and transmits the header over the interface.

[0190] Example 25 includes the subject matter of example 24, further comprising a bufferless arbiter for facilitating one-to-many connections on the interface.

[0191] Example 26 includes the subject matter of any one of Examples 24-25, further comprising a credit gasket for converting dedicated credits used by a transmitter in a first one of the compute blocks to shared credits used by a receiver in a second one of the compute blocks.

[0192] Example 27 includes the subject matter of any one of Examples 24-26, where the fabric includes an interconnect fabric of a system-on-chip (SoC) device, the SoC device including a plurality of compute blocks.

[0193] Example 28 includes the subject matter of any one of Examples 24-27, where the interface includes a header channel including a set of dedicated physical lanes for communicating packet headers, and the flit mode is used for headers communicated on the header channel.

[0194] Example 29 is an apparatus including fabric circuitry for implementing a fabric, the fabric supporting communication according to an input / output (I / O) interconnect protocol, the I / O interconnect protocol including a flit mode and a non-flit mode, wherein a set of flit mode header formats is used when in the flit mode, and a set of non-flit mode header formats is used when in the non-flit mode, the set of non-flit mode header formats including one or more non-flit mode fields; and an interface circuit for implementing an interface for coupling to an agent, said interface circuit comprising: determining that a link is to be trained for the non-flit mode; generating a header according to the set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, and where one or more fields of the set of flit mode header formats are reused in the header to carry the one or more non-flit mode fields; and The header is transmitted over the interface.

[0195] Example 30 includes the subject matter of example 29, wherein the one or more non-flit mode fields are not included in the set of flit mode header formats.

[0196] Example 31 includes the subject matter of any one of Examples 29-30, where the I / O interconnect protocol includes a load / store interconnect protocol.

[0197] Example 32 includes the subject matter of Example 31, wherein the I / O interconnect protocol includes one of a Peripheral Component Interconnect Express (PCIe) based protocol or a Compute Express Link (CXL) based protocol.

[0198] Example 33 includes the subject matter of any one of Examples 29-32, wherein the interface: a header channel implemented on a first subset of a plurality of physical lanes, the first subset of lanes including a first lane for carrying packet headers based on the interconnect protocol and a second lane for carrying metadata about the packet headers; and a DATA channel implemented on a separate second subset of the plurality of physical lanes, the second subset of lanes including a third lane for carrying packet payloads and a fourth lane for carrying metadata about the packet payloads; Including, Here, the header is transmitted over the header channel.

[0199] Example 34 includes the subject matter of any one of Examples 29-33, where the flit mode and the non-flit mode are based on a PCIe-based protocol.

[0200] Example 35 includes the subject matter of example 34, wherein the one or more non-flit mode fields are conveyed in the one or more fields of the set of flit mode header formats based on a mapping.

[0201] Example 36 includes the subject matter of any one of Examples 34-35, where the set of flit mode header formats includes one or more orthogonal content headers, and a particular field in a particular one of the one or more orthogonal content headers holds a particular field in the one or more non-flit mode fields.

[0202] Example 37 includes the subject matter of any one of Examples 34-36, wherein the set of flit mode header formats includes one or more prefixes, and a particular field in a particular one of the one or more prefixes holds a particular field in the one or more non-flit mode fields.

[0203] Example 38 includes the subject matter of example 37, wherein the particular prefix includes a mode field to indicate that the corresponding packet originated as a non-flit mode packet.

[0204] Example 39 includes the subject matter of any one of Examples 34-38, wherein end-to-end encryption is provided on the link based on a flit mode.

[0205] Example 40 includes the subject matter of any one of Examples 34-39, where the interface is based on the Streaming Fabric Interface (SFI) specification.

[0206] Throughout this specification, reference to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the disclosure. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0207] In the foregoing specification, a detailed description has been given with reference to specific exemplary embodiments. It will be evident, however, that various modifications and changes may be made thereto without departing from the broader spirit and scope as set forth in the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. Moreover, the above use of embodiment and other exemplary language does not necessarily refer to the same embodiment or the same example, but may refer to different individual embodiments and potentially the same embodiment. Other possible items (Item 1) a protocol circuit for implementing an input / output (I / O) interconnect protocol, the I / O interconnect protocol including a flit mode and a non-flit mode, wherein a set of flit mode header formats is used when in the flit mode, and a set of non-flit mode header formats is used when in the non-flit mode, the set of non-flit mode header formats including one or more non-flit mode fields; and An interface circuit for implementing an interface for coupling to a fabric, said interface circuit comprising: determining that a link is to be trained for the non-flit mode; generating a header according to the set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, and where one or more fields of the set of flit mode header formats are reused in the header to carry the one or more non-flit mode fields; and transmitting the header over the interface; An apparatus comprising: (Item 2) 2. The apparatus of claim 1, wherein the one or more non-flit mode fields are not included in the set of flit mode header formats. (Item 3) 2. The apparatus of claim 1, wherein the I / O interconnect protocol includes a load / store interconnect protocol. (Item 4) 4. The apparatus of claim 3, wherein the I / O interconnect protocol includes one of a Peripheral Component Interconnect Express (PCIe) based protocol or a Compute Express Link (CXL) based protocol. (Item 5) The interface is: a header channel implemented on a first subset of a plurality of physical lanes, the first subset of lanes including a first lane for carrying packet headers based on the interconnect protocol and a second lane for carrying metadata about the packet headers; and a data channel implemented on a distinct second subset of the plurality of physical lanes, the second subset of lanes including a third lane for carrying packet payloads and a fourth lane for carrying metadata about the packet payloads. Including, 2. The apparatus of claim 1, wherein the header is transmitted through the header channel. (Item 6) 2. The apparatus of claim 1, wherein the flit mode and the non-flit mode are based on a PCIe-based protocol. (Item 7) 7. The apparatus of claim 6, wherein the one or more non-flit mode fields are carried in the one or more fields of the set of flit mode header formats based on a mapping. (Item 8) 7. The apparatus of claim 6, wherein the set of flit mode header formats includes one or more orthogonal content headers, and a particular field in a particular one of the one or more orthogonal content headers holds a particular field in the one or more non-flit mode fields. (Item 9) 7. The apparatus of claim 6, wherein the set of flit mode header formats includes one or more prefixes, and a particular field in a particular one of the one or more prefixes holds a particular field in the one or more non-flit mode fields. (Item 10) 10. The apparatus of claim 9, wherein the particular prefix includes a mode field to indicate that the corresponding packet originated as a non-flit mode packet. (Item 11) 2. The apparatus of claim 1, wherein end-to-end encryption is provided over the link based on the flit mode. (Item 12) 2. The device of claim 1, wherein the interface is based on the Streaming Fabric Interface (SFI) specification. (Item 13) identifying a header of a packet, where the header of the packet is based on a non-flit mode format of a load / store interconnect protocol, the load / store interconnect protocol further defining a flit mode; generating a flit mode version of the header of the packet, where the flit mode version of the header of the packet is based on a flit mode format, where a first subset of fields in the non-flit mode format are also provided in the flit mode format and a second subset of fields in the non-flit mode format are excluded in the flit mode format, where generating the flit mode version of the header of the packet includes carrying one or more fields of the second subset of fields in reuse fields defined in the flit mode format; transmitting the flit mode version of the header of the packet over an interface to a fabric, where the flit mode version of the header of the packet is transmitted on a header channel implemented on a first plurality of physical lanes; and transmitting payload data of the packets over the interface to the fabric, where the payload data of the packets is transmitted on a data channel implemented on a separate second plurality of physical lanes. A method for providing the above. (Item 14) Item 14. The method according to item 13, wherein the interface is defined according to the SFI specification and the load / store protocol includes one of PCIe or CXL.io. (Item 15) The flit mode version of the header of the packet is transmitted on a first subset of the first plurality of physical lanes, and the method further comprises: transmitting header metadata over the interface using a second subset of the second plurality of physical lanes of the header channel. Item 14. The method of item 13, comprising: (Item 16) Fabric; and a plurality of compute blocks communicatively coupled through the fabric, where a particular compute block in the plurality of compute blocks: an agent circuit supporting a load / store interconnect protocol, the load / store protocol supporting a flit mode and a non-flit mode, wherein in the flit mode a set of flit mode header formats is used, and in the non-flit mode a set of non-flit mode header formats is used, the set of non-flit mode header formats including one or more non-flit mode fields; and an interface circuit for implementing an interface for coupling to said fabric, said interface circuit comprising: determining that the link is to be trained for said non-flit mode; generating a header according to the set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, and where one or more fields of the set of flit mode header formats are reused in the header to carry the one or more non-flit mode fields; and Sending the header over the interface Including, A system comprising: (Item 17) Item 17. The system of item 16, further comprising a bufferless arbiter for facilitating one-to-many connections on the interface. (Item 18) Item 17. The system of item 16, further comprising a credit gasket for converting dedicated credits used by a transmitter in a first one of the compute blocks to shared credits used by a receiver in a second one of the compute blocks. (Item 19) Item 17. The system of item 16, wherein the fabric comprises an interconnect fabric of a system-on-chip (SoC) device, the SoC device comprising the plurality of compute blocks. (Item 20) 20. The system of claim 16, wherein the interface includes a header channel including a set of dedicated physical lanes for communicating packet headers, and the flit mode is used for headers communicated on the header channel.

Claims

1. a protocol circuit for implementing an input / output (I / O) interconnect protocol, the I / O interconnect protocol including a flit mode and a non-flit mode, wherein a set of flit mode header formats is used when in the flit mode, and a set of non-flit mode header formats is used when in the non-flit mode, the set of non-flit mode header formats including one or more non-flit mode fields; and An interface circuit for implementing an interface for coupling to a fabric, said interface circuit comprising: determining that a link is to be trained for the non-flit mode; generating a header according to the set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, and where one or more fields of the set of flit mode header formats are reused in the header to carry the one or more non-flit mode fields; and transmitting the header over the interface; An apparatus comprising:

2. The apparatus of claim 1 , wherein the one or more non-flit mode fields are not included in the set of flit mode header formats.

3. The apparatus of claim 1 , wherein the I / O interconnect protocol comprises a load / store interconnect protocol.

4. The interface: a header channel implemented on a first subset of a plurality of physical lanes, the first subset of lanes including a first lane for carrying packet headers based on the interconnect protocol and a second lane for carrying metadata about the packet headers; and a data channel implemented on a distinct second subset of the plurality of physical lanes, the second subset of lanes including a third lane for carrying packet payloads and a fourth lane for carrying metadata about the packet payloads. Including, The apparatus of claim 1 , wherein the header is transmitted over the header channel.

5. The apparatus of claim 1 , wherein the flit mode and the non-flit mode are based on a PCIe-based protocol.

6. The apparatus of claim 5 , wherein the one or more non-flit mode fields are carried in the one or more fields of the set of flit mode header formats based on a mapping.

7. 6. The apparatus of claim 5, wherein the set of flit mode header formats includes one or more orthogonal content headers, and a particular field in a particular one of the one or more orthogonal content headers holds a particular field in the one or more non-flit mode fields.

8. 6. The apparatus of claim 5, wherein the set of flit mode header formats includes one or more prefixes, and a particular field in a particular one of the one or more prefixes holds a particular field in the one or more non-flit mode fields.

9. 10. The apparatus of claim 8, wherein the particular prefix includes a mode field to indicate that the corresponding packet originated as a non-flit mode packet.

10. The apparatus of claim 1 , wherein end-to-end encryption is provided over the link based on the flit mode.

11. The apparatus of claim 1 , wherein the interface is based on the Streaming Fabric Interface (SFI) specification.

12. identifying a header of a packet, where the header of the packet is based on a non-flit mode format of a load / store interconnect protocol, the load / store interconnect protocol further defining a flit mode; generating a flit mode version of the header of the packet, where the flit mode version of the header of the packet is based on a flit mode format, where a first subset of fields in the non-flit mode format are also provided in the flit mode format and a second subset of fields in the non-flit mode format are excluded in the flit mode format, where generating the flit mode version of the header of the packet includes carrying one or more fields of the second subset of fields in reuse fields defined in the flit mode format; transmitting the flit mode version of the header of the packet over an interface to a fabric, where the flit mode version of the header of the packet is transmitted on a header channel implemented on a first plurality of physical lanes; and transmitting payload data of the packets over the interface to the fabric, where the payload data of the packets is transmitted on a data channel implemented on a separate second plurality of physical lanes. A method for providing the above.

13. 13. The method of claim 12, wherein the interface is defined according to an SFI specification and the load / store interconnect protocol includes one of PCIe or CXL.io.

14. The flit mode version of the header of the packet is transmitted on a first subset of the first plurality of physical lanes, the method comprising: transmitting header metadata over the interface using a second subset of the second plurality of physical lanes of the header channel. The method of claim 12 comprising:

15. The interface: a header channel implemented on a first subset of a plurality of physical lanes, the first subset of lanes including a first lane for carrying packet headers based on the load / store interconnect protocol and a second lane for carrying metadata about the packet headers; and a data channel implemented on a distinct second subset of the plurality of physical lanes, the second subset of lanes including a third lane for carrying packet payloads and a fourth lane for carrying metadata about the packet payloads. Including, The method of claim 12 , wherein the header is transmitted over the header channel.

16. A system comprising means for carrying out the method according to any one of claims 12 to 15.

17. Fabric; and a plurality of compute blocks communicatively coupled through the fabric, where a particular compute block in the plurality of compute blocks: an agent circuit supporting a load / store interconnect protocol, the load / store interconnect protocol supporting a flit mode and a non-flit mode, wherein in the flit mode a set of flit mode header formats is used, and in the non-flit mode a set of non-flit mode header formats is used, the set of non-flit mode header formats including one or more non-flit mode fields; and an interface circuit for implementing an interface for coupling to said fabric, said interface circuit comprising: determining that the link is to be trained for the non-flit mode; generating a header according to the set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, and where one or more fields of the set of flit mode header formats are reused in the header to carry the one or more non-flit mode fields; and Sending the header over the interface Including, A system comprising:

18. 20. The system of claim 17, further comprising a bufferless arbiter for facilitating one-to-many connections on the interface.

19. 20. The system of claim 17, further comprising a credit gasket for converting dedicated credits used by a transmitter in a first one of the compute blocks to shared credits used by a receiver in a second one of the compute blocks.

20. 20. The system of claim 17, wherein the fabric comprises an interconnect fabric of a system-on-chip (SoC) device, the SoC device comprising the plurality of compute blocks.

21. 21. The system of claim 17, wherein the interface includes a header channel including a set of dedicated physical lanes for communicating packet headers, and the flit mode is used for headers communicated on the header channel.

22. fabric circuitry for implementing a fabric, the fabric supporting communications according to an input / output (I / O) interconnect protocol, the I / O interconnect protocol including a flit mode and a non-flit mode, wherein a set of flit mode header formats is used when in the flit mode, and a set of non-flit mode header formats is used when in the non-flit mode, the set of non-flit mode header formats including one or more non-flit mode fields; and an interface circuit for implementing an interface for coupling to an agent, said interface circuit comprising: determining that a link is to be trained for the non-flit mode; generating a header according to the set of flit mode header formats, where the header includes a field for indicating that a corresponding packet originated as a non-flit mode packet, and where one or more fields of the set of flit mode header formats are reused in the header to carry the one or more non-flit mode fields; and transmitting the header over the interface; An apparatus comprising:

23. The apparatus of claim 22 , wherein the one or more non-flit mode fields are not included in the set of flit mode header formats.

24. 23. The apparatus of claim 22, wherein the I / O interconnect protocol comprises one of a Peripheral Component Interconnect Express (PCIe) based protocol or a Compute Express Link (CXL) based protocol.

25. The interface: a header channel implemented on a first subset of a plurality of physical lanes, the first subset of lanes including a first lane for carrying packet headers based on the interconnect protocol and a second lane for carrying metadata about the packet headers; and a data channel implemented on a distinct second subset of the plurality of physical lanes, the second subset of lanes including a third lane for carrying packet payloads and a fourth lane for carrying metadata about the packet payloads. Including, 25. The apparatus of claim 22, wherein the header is transmitted over the header channel.

Images