Message protocol for data processing systems

Abstract

The present invention is entitled "MESSAGE PROTOCOL FOR DATA PROCESSING SYSTEMS". The present disclosure advantageously provides methods and systems for transmitting data through at least one interconnect. A requesting node coupled to the interconnect receives a first write burst from a first device via a first connection, divides the first write burst into an ordered sequence of smaller write requests based on the size of the first write burst, and sends the ordered sequence of write requests to a master node coupled to the interconnect. The master node generates an ordered sequence of write transactions based on the ordered sequence of write requests, and sends the ordered sequence of write transactions to a write combiner coupled to the master node. The write combiner combines the ordered sequence of write transactions into a second write burst of the same size as the first write burst, and sends the second write burst to a second device via a second connection.

Description

Background Art

[0001] The present disclosure relates to data processing systems. More particularly, the present disclosure relates to a message protocol for a data processing system.

[0002] A data processing system can be arranged as a system-on-chip (SoC) comprising various components connected via interconnects. Generally speaking, certain components, such as processors, processor clusters, and graphics processors, process data, while other components, such as memory controllers and input / output (I / O) controllers, send and receive data to and from the data processing components. Data configuration components can be connected to on-chip or off-chip memory, data buses, wired or wireless networks, and the like. The interconnect provides the network infrastructure over which data transactions between components are transmitted.

[0003] The interconnect can implement packet-based communication using a request-response-retry messaging protocol. Generally speaking, a requester is a component that initiates a transaction or request through the interconnect, and a completer is a component that responds to the request through the interconnect. Additionally, a requester can receive requests from a master device connected to the requester, and a completer can send requests to a slave device connected to the completer.

[0004] In this interconnect, if the transaction size of a request from a master to a requester is larger than the maximum transaction size supported by the interconnect, the requester divides the request into several smaller requests that are transmitted to the completer through the interconnect. The completer then issues a separate request to the slave for each request received from the requester through the interconnect. Unfortunately, this limitation significantly reduces interconnect efficiency, performance, and throughput, and increases message header overhead for requesters, completers, and slaves. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1A Depicted is a block diagram of a SoC according to an embodiment of the present disclosure.

[0006] Figure 1B Depicts using protocol node naming according to an embodiment of the present disclosure Figure 1A Block diagram of the SoC depicted in .

[0007] Figure 2A 、 Figure 2B and Figure 2C Different topologies of interconnects according to embodiments of the present disclosure are depicted.

[0008] Figure 3A and Figure 3B Depicts Figure 1A and Figure 1B The protocol flow for the ordered write flow of the SoC is depicted in FIG.

[0009] Figure 4A Depicted is a block diagram of a system with two interconnected SoCs, according to an embodiment of the present disclosure.

[0010] Figure 4B Depicts using protocol node naming according to an embodiment of the present disclosure Figure 4A A block diagram of the system depicted in FIG.

[0011] Figure 5A and Figure 5B Depicts Figure 4A and Figure 4B The protocol flow of the ordered write stream of the system 20 depicted in FIG.

[0012] Figure 6A Depicted is a block diagram of another SoC according to an embodiment of the present disclosure.

[0013] Figure 6B Depicts using protocol node naming according to an embodiment of the present disclosure Figure 6A Block diagram of the SoC depicted in .

[0014] Figure 7A and Figure 7B Depicted is an embodiment of the present disclosure Figure 6A and Figure 6B The protocol flow for the ordered write flow of the SoC is depicted in FIG.

[0015] Figure 8 Depicted is an embodiment of the present disclosure Figure 6B A portion of the block diagram depicted in .

[0016] Figure 9A Depicted is a block diagram of another system having two interconnected SoCs according to an embodiment of the present disclosure.

[0017] Figure 9B Depicts using protocol node naming according to an embodiment of the present disclosure Figure 9A A block diagram of the system depicted in FIG.

[0018] Figure 10A and Figure 10B Depicted is an embodiment of the present disclosure Figure 9A and Figure 9B The protocol flow for the ordered write stream of the system depicted in FIG.

[0019] Figure 11 Depicted is a flowchart representing functionality associated with a messaging protocol for a data processing network, according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure will now be described with reference to the drawings, wherein like reference numerals refer to like parts throughout.

[0021] The disclosed embodiments advantageously provide an efficient message protocol for a data processing system including one or more interconnects. While applicable to many message protocols that may pass through an interconnect, the disclosed embodiments advantageously improve the efficiency of PCIe peer-to-peer transactions, as discussed in detail below.

[0022] In one embodiment, a computer-based method for transmitting data over at least one interconnect is provided. At a requesting node coupled to the interconnect: receiving a first write burst from a first device via a first connection; dividing the first write burst into an ordered sequence of write requests based on the first write burst data size, the ordered sequence of write requests including at least a first write request and a subsequent write request, each write request having a data size less than the first write burst data size; and sending the ordered sequence of write requests to a master node coupled to the interconnect, the subsequent write request including a subsequent write identifier. At the master node: generating an ordered sequence of write transactions based on the ordered sequence of write requests, the ordered sequence of write transactions including at least a first write transaction and a subsequent write transaction, each write transaction having a data size equal to the data size of each write request; and sending the ordered sequence of write transactions to a write combiner coupled to the master node, the subsequent write transaction including a subsequent write identifier. At the write combiner: combining the ordered sequence of write transactions into a second write burst based at least on the subsequent write identifier, the second write burst having a data size equal to the first write burst data size; and sending the second write burst to a second device via a second connection.

[0023] Figure 1A A block diagram of a SoC 10 is depicted, according to an embodiment of the present disclosure.

[0024] In this embodiment, the SoC 10 includes an interconnect 100 coupled to one or more processors 110, one or more accelerators or specialized processors 120, a system memory management unit (SMMU) 130, one or more memory controllers 140-1, 140-2, etc., and a network interface controller (NIC) device 150. Each memory controller 140-1, 140-2 is coupled to a respective memory 142-1, 142-2, which may be integrated on the SoC 10 or connected externally. Peripheral Component Interconnect Express (PCIe) devices 152-1, 152-2 are connected to both the SMMU 130 and the NIC 150.

[0025] The interconnect 100 is a communication system that transmits data between the processor 110, the accelerator or special purpose processor 120, the SMMU 130, the memory controllers 140-1, 140-2 and the memories 142-1, 142-2, the NIC device 150 and the PCIe devices 152-1, 152-2, and other components. Certain components of the SoC 10 can be categorized as nodes of a specific type of interconnect protocol, such as with respect to Figure 1B Discuss in more detail.

[0026] Generally speaking, the interconnect 100 may include, among other things, a shared or hierarchical bus, a crossbar, a packet-based serialized network-on-chip (NoC), or the like. Figure 2A The interconnect 101 depicted in FIG. 1 has a crossbar topology that provides an ordered network with low latency and may be particularly suitable for small interconnects with a small number of protocol nodes, switches, and wire counts. Figure 2B The interconnect 102 depicted in FIG. 1 has a ring topology that balances wiring efficiency with delay, scales linearly with the number of protocol nodes, and may be particularly suitable for medium-sized interconnects. Figure 2C The interconnect 103 depicted in FIG has a mesh topology that has more wires to provide greater bandwidth, is modular and easily scalable by adding more rows and columns of switches or routers, and may be particularly suitable for large interconnects.

[0027] In many embodiments, the interconnect 100 is a coherent mesh network that includes multiple switch or router logic modules (routers) arranged in a two-dimensional rectangular mesh topology, such as an Arm CoreLink coherent mesh network. In this example, the switch or router is a crosspoint (i.e., XP). Each XP can be connected to up to four adjacent XPs using a mesh port and can be connected to one or two components (devices) using a device port. In addition, each XP can support four coherent hub interface (CHI) channels to transfer data from a source device to a destination or target device, as described, for example, in the ARM Advanced Microcontroller Bus Architecture (AMBA) CHI specification.

[0028] In these embodiments, the interconnect 100 may have an architecture comprising three layers (i.e., an upper protocol layer, a middle network layer, and a lower link layer). The protocol layer generates and processes requests and responses at protocol nodes, defines allowed cache state transitions at protocol nodes including caches, defines transaction flows for each request type, and manages protocol-level flow control. The network layer packets protocol messages, determines the source and destination node IDs required to route the packets to the desired destination through the interconnect 100, and adds them to the packets. The link layer provides flow control between components and manages link channels to provide deadlock-free switching across the interconnect 100.

[0029] The processor 110 is a general-purpose central processing unit (CPU) that executes instructions to perform various functions of the SoC 10, such as control, computation, input / output, etc. More specifically, the processor 110 may include a single processor core or multiple processor cores, which may be arranged in a processor cluster, such as the Arm Cortex A, R, and M series processors. Generally speaking, the processor 110 may execute computer programs or modules stored in a memory (such as memory 142-1, memory 142-2, etc.), such as an operating system, application software, other software modules, etc. The processor 110 may also include a local cache memory.

[0030] The accelerator or dedicated processor 120 is a specialized processor optimized to perform specific functions, such as processing graphics, images, and / or multimedia data, processing digital signal data, processing artificial neural network data, etc. For example, the accelerator or dedicated processor 120 may be a graphics processing unit (GPU), a neural processing unit (NPU), a digital signal processor (DSP), etc. More specifically, the accelerator or dedicated processor 120 may include a single processor core or multiple processor cores, such as an Arm Mali series GPU, a display processor and video processor, an Arm machine learning processor, etc. The accelerator or dedicated processor 120 may also include local cache memory.

[0031] The SMMU 130 includes a microprocessor, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc., and is configured to transmit data to various devices, such as PCIe devices, serial advanced technology attachment (SATA) devices, etc. Generally speaking, the SMMU 130 converts an input address into an output address by performing one or more translation table traversals.

[0032] In certain embodiments, SMMU 130 is a memory management unit (MMU), such as the ARM CoreLink MMU family, which includes a translation buffer unit (TBU) and a translation cache unit (TCU), and uses an AMBA connection with an Advanced eXtensible Interface (AXI) and / or AXI Coherence Extension (ACE) Lite protocol to communicate with various devices. SMMU 130 implements a master interface (i.e., TBU) for each connected master device (e.g., TBU0 for the master interface of PCIe device 152-1, TBU 1 for the master interface of PCIe device 152-2, etc.), and each TBU contains a translation lookaside buffer (TLB) for caching page tables. The TCU controls and manages address translation. TBU 0 and TBU 1 are independently connected to the interconnect 100 and communicate with the interconnect 100 using an AMBA connection with an AXI and / or ACE Lite protocol.

[0033] The memory controllers 140-1 and 140-2 include a microprocessor, a microcontroller, an ASIC, an FPGA, etc., and are configured to provide access to the memories 142-1 and 142-2 through the interconnect 100. The memories 142-1 and 142-2 may include various non-transitory computer-readable media that can be accessed by other components of the SoC 10, such as the processor 110, the accelerator or special-purpose processor 120, the SMMU 130, etc. For example, the memory 142-1 may store data and instructions for execution by the processor 110, the accelerator or special-purpose processor 120, etc.

[0034] In various embodiments, the memories 142-1, 142-2 may include volatile and non-volatile media, non-removable media, and / or removable media. For example, the memories 142-1, 142-2 may include any combination of random access memory (RAM), dynamic RAM (DRAM), double data rate (DDR) DRAM or synchronous DRAM (SDRAM), static RAM (SRAM), read-only memory (ROM), flash memory, cache memory, and / or any other type of non-transitory computer-readable media. In certain embodiments, the memory controllers 140-1, 140-2 are dynamic memory controllers that provide data transfer to and from high-density DDR3 or DDR4 DRAM memory (such as the Arm CoreLink dynamic memory controller (DMC) family), each dynamic memory controller including a fast single-port CHI channel interface for connecting to the interconnect 100.

[0035] NIC 150 includes a microprocessor, microcontroller, ASIC, FPGA, etc., and is configured to connect various devices, such as PCIe devices 152-1, 152-2, etc., using AMBA connections with AXI and / or ACE Lite protocols. In some embodiments, NIC 150 is a network interconnect, such as the Arm CoreLink NIC series, which includes an AMBA interface with AXI and / or ACE Lite protocols. NIC 150 is coupled to interconnect 100 using, for example, one or more AMBA connections with AXI and / or ACE Lite protocols.

[0036] PCIe device 152-1 includes a PCIe root complex coupled to PCIe endpoint A. Similarly, PCIe device 152-2 includes a PCIe root complex coupled to PCIe endpoint B. In this embodiment, each root complex includes an AXI master interface for communicating with SMMU 130 and an AXI slave interface for communicating with NIC 150. The AXI master interface of PCIe device 152-1 is coupled to TBU 0 of SMMU 130, while the AXI master interface of PCIe device 152-2 is coupled to TBU 1 of SMMU 130.

[0037] Generally speaking, PCIe endpoint A on PCIe device 152 - 1 and PCIe endpoint B on PCIe device 152 - 2 exchange PCIe peer-to-peer traffic through SMMU 130 , interconnect 100 , and NIC 150 .

[0038] Figure 1B Depicted is a block diagram of a SoC 10 using protocol node naming, according to an embodiment of the present disclosure.

[0039] Requesters are represented by requesting nodes (RNs), which are protocol nodes that generate protocol transactions for the interconnect 100, such as memory reads and writes, I / O data transfers, etc. RN-F protocol nodes represent "full" coherent requesters, and RN-I protocol nodes represent "I / O" coherent requesters. The processor 110 and the accelerator or dedicated processor 120 are fully coherent requesters, so RN-F 210 represents the processor 110, and RN-F 220 represents the accelerator or dedicated processor 120. Each RN-I represents a master device connected to the SMMU 130 and includes a microprocessor, microcontroller, ASIC, FPGA, logic circuitry, etc. to provide the relevant functionality. In some embodiments, the RN-Is may use the same hardware. PCIe devices 152-1, 152-2 are I / O coherent requesters, so RN-I 206-1 represents PCIe device 152-1, and RN-I 206-2 represents PCIe device 152-2. In this embodiment, the SMMU 130 acts as a bridge from the RN-Is 206-1, 206-2 to the host interfaces of the PCIe devices 152-1, 152-2, respectively.

[0040] In this embodiment, the interconnect 100 includes several completers, each of which includes a microprocessor, microcontroller, ASIC, FPGA, logic circuit, etc. to provide related functions. Each completer is represented by a master node (HN), which is a protocol node that receives protocol transactions from the RN and can receive protocol transactions from the slave node (SN), as described below. Each HN is responsible for managing a specific portion of the total address space of the SoC 10. Similar to the RN, the HN-F protocol node represents a fully coherent completer, and the HN-I protocol node represents an I / O coherent completer.

[0041] In many embodiments, the entire address space of memories 142-1, 142-2 can be managed by HN-Fs 202-1, 202-2 in SoC 10. Each HN-F can include a system-level cache and a snoop traffic filter, and act as a consistency point (VP) and serialization point (PoS) for memory requests sent to that HN-F. To avoid conflicts when multiple RNs attempt to access the same memory address within memories 142-1, 142-2 in a serial manner, HN-Fs 202-1, 202-2 act as PoS to handle read requests, write requests, etc. A fully coherent slave device can be represented by an SN-F node, which is a protocol node that receives and completes requests from the HN-F. Memory controllers 140-1, 140-2 are fully coherent slave devices and are represented as SN-Fs 240-1, 240-2. Each HN-F 202-1, 202-2 is associated with a corresponding SN-F 240-1, 240-2.

[0042] Each HN-I is responsible for managing all transactions targeting the address space of a specific memory-mapped I / O slave device or subsystem and serves as the consistency point (VP) and serialization point (PoS) for requests sent to that slave device. PCIe devices 152-1 and 152-2 are also memory-mapped I / O slave devices, so HN-I 204 manages the address space of PCIe devices 152-1 and 152-1 (slave interfaces). In this embodiment, NIC 150 acts as a bridge to the slave interfaces of PCIe devices 152-1 and 152-2.

[0043] PCIe peer-to-peer traffic originating from PCIe endpoint A on PCIe device 152-1 and proceeding to PCIe endpoint B on PCIe device 152-2 enters through RN-I 206-1, traverses interconnect 100, and exits through HN-I 204. Similarly, PCIe peer-to-peer traffic originating from PCIe endpoint B on PCIe device 152-2 and proceeding to PCIe endpoint A on PCIe device 152-1 enters through RN-I 206-2, traverses interconnect 100, and exits through HN-I 204. Posted write transactions adhere to ordered write observation (OWO) to maintain PCIe ordering rules.

[0044] As mentioned above, Figure 2A 、 Figure 2B and Figure 2C Different topologies of the interconnect 100 according to embodiments of the present disclosure are depicted.

[0045] Figure 2A An interconnector 101 is depicted having a crossbar topology configured to directly connect a pair of protocol nodes to exchange requests and responses. The interconnector 101 includes two crossbar matrices, CSM1 and CSM2. CSM1 connects RN-F 210, RN-F 220, RN-I 206-1, and RN-I 206-2 to HN-F 202-1, HN-F 202-2, and HN-I 204. CSM2 connects HN-F 202-1 and HN-F 202-2 to SN-F 240-1 and SN-F 240-2. CSM1 includes twelve switches 104 (S1 to S2). 12 ) and connections 105 between adjacent switches 104. CSM2 includes four switches 104 (S 13 To S 16 ) and connections 105 between adjacent switches 104. Each connection 105 includes a sufficient number of conductors to support the channels through which requests and responses are transmitted. RN-F 210, RN-F 220, RN-I 206-1, RN-I 206-2, HN-F 202-1, HN-F 202-2, and HN-I 204 are directly connected to CSM1. HN-F 202-1, HN-F 202-2, SN-F 240-1, and SN-F 240-2 are directly connected to CSM1. Figure 2A As shown, HN-F 202-1 and HN-F 202-2 are directly connected to both CSM1 and CSM2.

[0046] For example, to exchange requests and responses between RN-F 210 and HN-F 202-1, switch S1 is closed, and to exchange requests and responses between HN-F 202-1 and SN-F 240-1, switch S2 is closed. 15 Only a single path between a pair of protocol nodes is configured at a time in each CSM. The relevant elements and paths are in Figure 2A Highlighted in.

[0047] Figure 2B An interconnect 102 is depicted having a ring topology configured to connect all protocol nodes. The ring has nine routers 106 (R1 through R9) and connections 107 between adjacent routers 106. Each connection 107 includes a sufficient number of conductors to support a channel over which requests and responses are transmitted. RN-F 210 is connected to router R1, RN-F 220 is connected to router R2, HN-F 202-1 is connected to router R3, HN-F 202-2 is connected to router R4, SN-F 240-1 is connected to router R5, SN-F 240-2 is connected to router R6, HN-I 204 is connected to router R7, RN-I 206-2 is connected to router R8, and RN-I 206-1 is connected to router R9.

[0048] For example, to exchange requests and responses between RN-F 210 and HN-F 202-1, messages pass through routers R1, R2, and R3, and to exchange requests and responses between HN-F 202-1 and SN-F 240-1, messages pass between routers R3, R4, and R5. In a ring topology, multiple requests and responses can flow around the ring network simultaneously. The relevant elements and paths are shown in Figure 1. Figure 2B Highlighted in.

[0049] Figure 2C An interconnect 102 is depicted having a mesh topology configured to connect all protocol nodes. The mesh has nine routers 108 (R1 through R9) and connections 109 between adjacent routers 108. Each connection 109 includes a sufficient number of conductors to support a channel over which requests and responses are transmitted. RN-F 210 is connected to router R1, RN-F 220 is connected to router R2, RN-I 206-1 is connected to router R3, HN-F 202-1 is connected to router R4, HN-F 202-2 is connected to router R5, RN-I 206-2 is connected to router R6, SN-F 240-1 is connected to router R7, SN-F 240-2 is connected to router R8, and HN-I 204 is connected to router R9.

[0050] For example, to exchange requests and responses between RN-F 210 and HN-F 202-1, messages pass through routers R1 and R4, and to exchange requests and responses between HN-F 202-1 and SN-F 240-1, messages pass between routers R4 and R7. In a mesh topology, multiple requests and responses can flow through the mesh network simultaneously. The relevant elements and paths are shown in Figure 2. Figure 2C Highlighted in.

[0051] Figure 3A Describes the use Figure 1A and Figure 1B The protocol flow 300 of the ordered write flow of the SoC 10 is depicted in FIG.

[0052] 2. The protocol flow 300 shows an orderly write flow of PCIe peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer traffic flows between PCIe device 152-1 (master interface) and RN-I 206-1 by AXI connection, flows between RN-I 206-1 and HN-I 204 by CHI connection, and flows between HN-I 204 and PCIe device 152-2 (slave interface) by AXI connection. Protocol nodes are positioned along the horizontal axis, and time is vertically indicated from top to bottom. The lifespan of write transactions at each protocol node is illustrated by the elongated shaded rectangle along the time axis, which begins with a square and ends with a circle. Other connection protocols can also be expected.

[0053] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written to the RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. Because the size of the data to be written is 128 bytes (128B), the RN-I 206-1 breaks or divides the write request into two separate 64-byte (64B) write transactions for transmission via the AMBA CHI connection, namely, a WriteA request message and a WriteB request message.

[0054] RN-I 206-1 sequentially sends WriteA and WriteB request messages to HN-I 204 via the request channel of interconnect 100. These request messages include the addresses to be written. In response to receiving the WriteA and WriteB request messages, HN-I 204 sequentially sends DBID_RspA and DBID_RspB response messages to RN-I 206-1 via the response channel of interconnect 100. These response messages provide the data buffer identifier for each write transaction. In response to receiving the DBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to HN-I 204 via the data channel of interconnect 100. This data message includes the first 64B of the data to be written. Similarly, in response to receiving the DBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to HN-I 204 via the data channel of interconnect 100. This data message includes the second 64B of the data to be written.

[0055] In response to receiving the Wr_DataA data message, HN-I 204 sends the address to be written and the first 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. Similarly, in response to receiving the Wr_DataB data message, HN-I 204 sends the address to be written and the second 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively.

[0056] After the first 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespA write response to HN-I 204 via the write response channel, thereby indicating the status of the first write transaction. Similarly, after the second 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespB write response to HN-I 204 via the write response channel, thereby indicating the status of the second write transaction.

[0057] In response to receiving the BRespA write response, HN-I 204 sends a CompA write complete response message to RN-I 206-1 via the response channel. Before sending the CompA write complete response message, HN-I 204 must wait for the BRespA write response from PCIe device 152-2 to arrive, because in this embodiment, PCIe device 152-2 is the PoS. Similarly, in response to receiving the BRespB write response, HN-I 204 sends a CompB write complete response message to RN-I 206-1 via the response channel. Before sending the CompB write complete response message, HN-I 204 must wait for the BRespB write response from PCIe device 152-2 to arrive, because in this embodiment, PCIe device 152-2 is the PoS.

[0058] Finally, in response to receiving the CompB write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 3A The lifetime L of a WriteA write transaction is depicted in A and the lifetime L of the WriteB write transaction B .

[0059] Figure 3B Describes the use Figure 1A and Figure 1B 3. The protocol flow 302 of another ordered write flow of the SoC depicted in FIG.

[0060] Protocol flow 302 illustrates an ordered write flow of PCIe peer-to-peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer-to-peer traffic flows between PCIe device 152-1 (master interface) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and HN-I 204 over a CHI connection, and between HN-I 204 and PCIe device 152-2 (slave interface) over an AXI connection.

[0061] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written to the RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. Because the size of the data to be written is 256 bytes (256B), the RN-I 206-1 breaks or divides the write request into four separate 64B write transactions (i.e., WriteA, WriteB, WriteC, and WriteD request messages) for transmission via the AMBA CHI connection.

[0062] RN-I 206-1 sequentially sends WriteA, WriteB, WriteC, and WriteD request messages to HN-I 204 via the request channel of interconnect 100. These request messages include the addresses to be written. In response to receiving the WriteA, WriteB, WriteC, and WriteD request messages, HN-I 204 sequentially sends DBID_RspA, DBID_RspB, DBID_RspC, and DBID_RspD response messages to RN-I 206-1 via the response channel of interconnect 100. These response messages provide the data buffer identifier for each write transaction. In response to receiving the DBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to HN-I 204 via the data channel of interconnect 100. This data message includes the first 64B of the data to be written. Similarly, in response to receiving the DBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to HN-I 204 via the data channel of interconnect 100. This data message includes the second 64B of the data to be written. In response to receiving the DBID_RspC response message, RN-I 206-1 sends a Wr_DataC data message to HN-I 204 via the data channel of interconnect 100. This data message includes the third 64B of the data to be written. And, in response to receiving the DBID_RspD response message, RN-I 206-1 sends a Wr_DataD data message to HN-I 204 via the data channel of interconnect 100. This data message includes the fourth 64B of the data to be written.

[0063] In response to receiving the Wr_DataA data message, HN-I 204 sends the address to be written and the first 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. Similarly, in response to receiving the Wr_DataB data message, HN-I 204 sends the address to be written and the second 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. In response to receiving the Wr_DataC data message, HN-I 204 sends the address to be written and the third 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. And, in response to receiving the Wr_DataD data message, HN-I 204 sends the fourth 64B of the address to be written and the data to be written to PCIe device 152-2 (slave interface) through the AXI write address channel (AW) and the write data channel (W), respectively.

[0064] After the first 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespA write response to HN-I 204 via the write response channel, thereby indicating the status of the first write transaction. Similarly, after the second 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespB write response to HN-I 204 via the write response channel, thereby indicating the status of the second write transaction. After the third 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespC write response to HN-I 204 via the write response channel, thereby indicating the status of the third write transaction. Furthermore, after the fourth 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespD write response to HN-I 204 via the write response channel, thereby indicating the status of the fourth write transaction.

[0065] In response to receiving the BRespA write response, HN-I 204 sends a CompA write complete response message to RN-I 206-1 via the response channel. Before sending the CompA write complete response message, HN-I 204 must wait for the BRespA write response from PCIe device 152-2 to arrive, because in this embodiment, PCIe device 152-2 is the PoS. Similarly, in response to receiving the BRespB write response, HN-I 204 sends a CompB write complete response message to RN-I 206-1 via the response channel. Before sending the CompB write complete response message, HN-I 204 must wait for the BRespB write response from PCIe device 152-2 to arrive, because in this embodiment, PCIe device 152-2 is the PoS. In response to receiving the BRespC write response, HN-I 204 sends a CompC write complete response message to RN-I 206-1 via the response channel. Before sending the CompC write complete response message, HN-I 204 must wait for the BRespC write response from PCIe device 152-2 to arrive, because PCIe device 152-2 is the PoS in this embodiment. Furthermore, in response to receiving the BRespD write response, HN-I 204 sends a CompD write complete response message to RN-I 206-1 via the response channel. Before sending the CompD write complete response message, HN-I 204 must wait for the BRespD write response from PCIe device 152-2 to arrive, because PCIe device 152-2 is the PoS in this embodiment.

[0066] Finally, in response to receiving the CompD write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 3B The lifetime L of the write transactions WriteA, WriteB, WriteC, and WriteD is depicted in A 、L B 、L C and L D .

[0067] Figure 4A Depicted is a block diagram of a system 20 having two interconnected SoCs according to an embodiment of the present disclosure.

[0068] SoC 10-1 and SoC 10-2 are depicted as including a baseline number of components; in other embodiments, SoC 10-1 and SoC 10-2 may include additional components, such as accelerators of specialized processors 120, etc. Additional SoCs 10 may be readily added to system 20 as described below.

[0069] In this embodiment, SoC 10-1 includes an interconnect 100-1 coupled to one or more processors 110-1, a memory controller 140-1, and a NIC device 150-1. Memory controller 140-1 is coupled to memory 142-1, which may be integrated on SoC 10-1 or connected externally. PCIe device 152-1 (slave interface) is connected to NIC 150-1. In some embodiments, SoC 10-1 may include an SMMU 130-1 (depicted in dashed outline) that connects interconnect 100-1 to PCIe device 152-1 (master interface); in other embodiments, interconnect 100-1 may be directly connected to PCIe device 152-1 (master interface).

[0070] Similarly, SoC 10-2 includes an interconnect 100-2 coupled to one or more processors 110-2, a memory controller 140-2, and a NIC device 150-2. Memory controller 140-2 is coupled to memory 142-2, which may be integrated on SoC 10-2 or connected externally. PCIe device 152-2 (slave interface) is connected to NIC 150-2. In some embodiments, SoC 10-2 may include an SMMU 130-2 (depicted in dashed outline) that connects interconnect 100-2 to PCIe device 152-2 (master interface); in other embodiments, interconnect 100-2 may be directly connected to PCIe device 152-2 (master interface).

[0071] Interconnect 100-1 may be connected to interconnect 100-2 using two or more cache coherent interconnect (CCIX) connections 112-1, 112-2; other types of connections are also contemplated. CCIX is a chip-to-chip interconnect that enables two or more devices to share data in a cache coherent manner. Figure 3A In the depicted embodiment, two CCIX connections 112-1 and 112-2 are depicted, one for each traffic direction. In other embodiments, a single CCIX connection 112 may be shared by all traffic. Generally, one or more CCIX connections 112 form a CCIX interconnect. The CCIX protocol advantageously integrates with the PCIe protocol as well as the AMBA CHI, AXI, and / or ACE Lite protocols. Additional SoCs 10 can be added to the system 20 simply by adding additional CCIX connections 112 according to a specific connection topology (such as direct, switched, daisy-chain, mesh, etc.).

[0072] Generally speaking, PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2 exchange PCIe peer traffic through (optional) SMMUs 130-1, 130-2, interconnects 100-1, 100-2, CCIX connections 112-1, 112-2, and NICs 150-1, 150-2.

[0073] Figure 4B Depicts using protocol node naming according to an embodiment of the present disclosure Figure 3A A block diagram of the system 20 depicted in FIG.

[0074] SoC 10-1 and SoC 10-2 include Figure 3B , and a subset of the protocol nodes depicted in and discussed above, as well as additional protocol nodes associated with CCIX connections 112 - 1 and 112 - 2 .

[0075] With respect to SoC 10-1, RN-F 210-1 represents processor 110-1, RN-I 206-1 represents PCIe device 152-1 (master interface), HN-F 202-1 manages the address space of memory 142-1 and is associated with SN-F 240-1, which represents memory controller 140-1, and HN-I 204 manages the address space of PCIe device 152-1 (slave interface). In this embodiment, NIC 150-1 acts as a bridge to the slave interface of PCIe device 152-1. If present, SMMU 130-1 can act as a bridge from RN-I 206-1 to the master interface of PCIe device 152-1.

[0076] CCIX Request Agent (CXRA) 208-1 performs read and write transactions over CCIX connection 112-1 and includes a microprocessor, microcontroller, ASIC, FPGA, logic circuitry, etc. to provide related functionality. CCIX Host Agent (CXHA) 209-1 is responsible for managing the consistency of CCIX connection 112-2 and includes a microprocessor, microcontroller, ASIC, FPGA, logic circuitry, etc. to provide related functionality. In some embodiments, CXRA 208-1 and CXHA 209-1 may utilize the same hardware.

[0077] With respect to SoC 10-2, RN-F 210-2 represents processor 110, RN-I 206-2 represents PCIe device 152-2 (master interface), HN-F 202-2 manages the address space of memory 142-2 and is associated with SN-F 240-2, which represents memory controller 140-2, and HN-I 204 manages the address space of PCIe device 152-2 (slave interface). In this embodiment, NIC 150-2 acts as a bridge to the slave interface of PCIe device 152-2. If present, SMMU 130-2 can act as a bridge from RN-I 206-2 to the master interface of PCIe device 152-2.

[0078] CCIX Request Agent (CXRA) 208-2 performs read and write transactions over CCIX connection 112-2 and includes a microprocessor, microcontroller, ASIC, FPGA, logic circuitry, etc. to provide related functionality. CCIX Host Agent (CXHA) 209-2 is responsible for managing the consistency of CCIX connection 112-1 and includes a microprocessor, microcontroller, ASIC, FPGA, logic circuitry, etc. to provide related functionality. In some embodiments, CXRA 208-2 and CXHA 209-2 may utilize the same hardware.

[0079] PCIe peer-to-peer traffic originating from PCIe endpoint A on PCIe device 152-1 and proceeding to PCIe endpoint B on PCIe device 152-2 enters through RN-I 206-1, passes through interconnect 100-1, CCIX connection 112-1, and interconnect 100-2, and exits through HN-I 204-2. Similarly, PCIe peer-to-peer traffic originating from PCIe endpoint B on PCIe device 152-2 and proceeding to PCIe endpoint A on PCIe device 152-1 enters through RN-I 206-2, passes through interconnect 100-2, CCIX connection 112-2, and interconnect 100-1, and exits through HN-I 204-1. Issued write transactions adhere to OWO to maintain PCIe ordering rules.

[0080] Figure 5A Describes the use Figure 4A and Figure 4B The protocol flow 310 of the ordered write stream of the system 20 depicted in FIG.

[0081] Protocol flow 310 illustrates an ordered write flow of PCIe peer-to-peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer-to-peer traffic flows between PCIe device 152-1 (master) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and CXRA 208-1 over a CHI connection on interconnect 100-1, between CXRA 208-1 and CXHA 290-2 over a CCIX connection, between CXHA 290-2 and HN-I 204-2 over a CHI connection on interconnect 100-2, and between HN-I 204-2 and PCIe device 152-2 (slave) over an AXI connection.

[0082] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written to the RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. Because the size of the data to be written is 128 bytes (128B), the RN-I 206-1 breaks or divides the write request into two separate 64-byte (64B) write transactions for transmission via the AMBA CHI connection, namely, a WriteA request message and a WriteB request message.

[0083] RN-I 206-1 sends WriteA and WriteB request messages, including the addresses to be written, to CXRA 208-1 via the request channel of interconnector 100-1. CXRA 208-1 forwards the WriteA and WriteB request messages to CXHA 209-2 via the CCIX connection. CXHA then forwards the WriteA and WriteB request messages to HN-I 204-2 via the request channel of interconnector 100-2.

[0084] In response to receiving the WriteA and WriteB request messages, HN-I 204-2 sequentially sends DBID_RspA and DBID_RspB response messages to CXHA 209-2 via the response channel of interconnect 100-2. These response messages provide the data buffer identifier for each write transaction. CXHA 209-2 sequentially forwards the DBID_RspA and DBID_RspB response messages to CXRA 208-1 via the CCIX connection. CXRA then sequentially forwards the DBID_RspA and DBID_RspB response messages to RN-I 206-1 via the response channel of interconnect 100-1.

[0085] In response to receiving the DBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to CXRA 208-1 via the data channel of interconnect 100-1. This data message includes the first 64B of the data to be written. Similarly, in response to receiving the DBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to CXRA 208-1 via the data channel of interconnect 100-1. This data message includes the second 64B of the data to be written. CXRA 208-1 forwards the Wr_DataA and Wr_DataB data messages to CXHA 209-2 via the CCIX connection. CXHA then forwards the Wr_DataA and Wr_DataB data messages to HN-I 204-2 via the data channel of interconnect 100-2.

[0086] In response to receiving the Wr_DataA data message, HN-I 204-2 sends the address to be written and the first 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. Similarly, in response to receiving the Wr_DataB data message, HN-I 204-2 sends the address to be written and the second 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively.

[0087] After the first 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespA write response to HN-I 204-2 via the write response channel, thereby indicating the status of the first write transaction. Similarly, after the second 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespB write response to HN-I 204-2 via the write response channel, thereby indicating the status of the second write transaction.

[0088] In response to receiving the BRespA Write Response, HN-I 204-2 sends a CompA Write Complete Response message to CXHA 209-2 via the response channel of interconnect 100-2. The CXHA, in turn, forwards the CompA Write Complete Response message to CXRA 208-1 via the CCIX connection. The CXRA, in turn, forwards the CompA Write Complete Response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompA Write Complete Response message, HN-I 204-2 must wait for the BRespA Write Response to arrive from PCIe device 152-2, as PCIe device 152-2 is a PoS in this embodiment.

[0089] Similarly, in response to receiving the BRespB Write Response, HN-I 204-2 sends a CompB Write Complete Response message to CXHA 209-2 via the response channel of interconnect 100-2. CXHA, in turn, forwards the CompB Write Complete Response message to CXRA 208-1 via the CCIX connection. CXRA, in turn, forwards the CompB Write Complete Response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompB Write Complete Response message, HN-I 204-2 must wait for the BRespB Write Response from PCIe device 152-2 to arrive, since PCIe device 152-2 is a PoS in this embodiment.

[0090] Finally, in response to receiving the CompB write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 5A The lifetime L of a WriteA write transaction is depicted in A and the lifetime L of the WriteB write transaction B .

[0091] Figure 5B Describes the use Figure 4A and Figure 4B The protocol flow 312 of the ordered write stream of the system 20 depicted in FIG.

[0092] Protocol flow 312 illustrates an ordered write flow of PCIe peer-to-peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer-to-peer traffic flows between PCIe device 152-1 (master) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and CXRA 208-1 over a CHI connection on interconnect 100-1, between CXRA 208-1 and CXHA 290-2 over a CCIX connection, between CXHA 290-2 and HN-I 204-2 over a CHI connection on interconnect 100-2, and between HN-I 204-2 and PCIe device 152-2 (slave) over an AXI connection.

[0093] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written to the RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. Because the size of the data to be written is 256 bytes (256B), the RN-I 206-1 breaks or divides the write request into four separate 64B write transactions (i.e., WriteA, WriteB, WriteC, and WriteD request messages) for transmission via the AMBA CHI connection.

[0094] RN-I 206-1 sequentially sends WriteA, WriteB, WriteC, and WriteD request messages to CXRA 208-1 via the request channel of interconnector 100-1. These request messages include the addresses to be written. CXRA 208-1 sequentially forwards these WriteA, WriteB, WriteC, and WriteD request messages to CXHA 209-2 via the CCIX connection. CXHA then sequentially forwards these WriteA, WriteB, WriteC, and WriteD request messages to HN-I 204-2 via the request channel of interconnector 100-2.

[0095] In response to receiving the WriteA, WriteB, WriteC, and WriteD request messages, HN-I 204-2 sequentially sends DBID_RspA, DBID_RspB, DBID_RspC, and DBID_RspD response messages to CXHA 209-2 via the response channel of interconnect 100-2. These response messages provide the data buffer identifier for each write transaction. CXHA 209-2 sequentially forwards the DBID_RspA, DBID_RspB, DBID_RspC, and DBID_RspD response messages to CXRA 208-1 via the CCIX connection. The CXRA then sequentially forwards the DBID_RspA, DBID_RspB, DBID_RspC, and DBID_RspD response messages to RN-I 206-1 via the response channel of interconnect 100-1.

[0096] In response to receiving the DBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to CXRA 208-1 via the data channel of interconnect 100-1. This data message includes the first 64B of the data to be written. Similarly, in response to receiving the DBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to CXRA 208-1 via the data channel of interconnect 100-1. This data message includes the second 64B of the data to be written. In response to receiving the DBID_RspC response message, RN-I 206-1 sends a Wr_DataC data message to CXRA 208-1 via the data channel of interconnect 100-1. This data message includes the third 64B of the data to be written. In response to receiving the DBID_RspD response message, RN-I 206-1 sends a Wr_DataD data message to CXRA 208-1 via the data channel of interconnect 100-1. The data message includes the fourth 64B of the data to be written. CXRA 208-1 forwards the Wr_DataA, Wr_DataB, Wr_DataC, and Wr_DataD data messages to CXHA 209-2 via the CCIX connection. CXHA then forwards the Wr_DataA, Wr_DataB, Wr_DataC, and Wr_DataD data messages to HN-I 204-2 via the data channel of interconnect 100-2.

[0097] In response to receiving the Wr_DataA data message, HN-I 204-2 sends the address to be written and the first 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. Similarly, in response to receiving the Wr_DataB data message, HN-I 204-2 sends the address to be written and the second 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. In response to receiving the Wr_DataC data message, HN-I 204-2 sends the address to be written and the third 64B of the data to be written to PCIe device 152-2 (slave interface) via the AXI write address channel (AW) and the write data channel (W), respectively. And, in response to receiving the Wr_DataD data message, HN-I 204-2 sends the fourth 64B of the address to be written and the data to be written to PCIe device 152-2 (slave interface) through the AXI write address channel (AW) and the write data channel (W), respectively.

[0098] After the first 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespA write response to HN-I 204-2 via the write response channel, thereby indicating the status of the first write transaction. Similarly, after the second 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespB write response to HN-I 204-2 via the write response channel, thereby indicating the status of the second write transaction. After the third 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespC write response to HN-I 204-2 via the write response channel, thereby indicating the status of the third write transaction. Furthermore, after the fourth 64B write transaction is completed, PCIe device 152-2 (slave interface) sends a BRespD write response to HN-I 204-2 via the write response channel, thereby indicating the status of the fourth write transaction.

[0099] In response to receiving the BRespA Write Response, HN-I 204-2 sends a CompA Write Complete Response message to CXHA 209-2 via the response channel of interconnect 100-2. The CXHA, in turn, forwards the CompA Write Complete Response message to CXRA 208-1 via the CCIX connection. The CXRA, in turn, forwards the CompA Write Complete Response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompA Write Complete Response message, HN-I 204-2 must wait for the BRespA Write Response to arrive from PCIe device 152-2, as PCIe device 152-2 is a PoS in this embodiment.

[0100] Similarly, in response to receiving the BRespB Write Response, HN-I 204-2 sends a CompB Write Complete Response message to CXHA 209-2 via the response channel of interconnect 100-2. CXHA, in turn, forwards the CompB Write Complete Response message to CXRA 208-1 via the CCIX connection. CXRA, in turn, forwards the CompB Write Complete Response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompB Write Complete Response message, HN-I 204-2 must wait for the BRespB Write Response from PCIe device 152-2 to arrive, since PCIe device 152-2 is a PoS in this embodiment.

[0101] In response to receiving the BRespC Write Response, HN-I 204-2 sends a CompC Write Complete Response message to CXHA 209-2 via the response channel of interconnect 100-2. The CXHA, in turn, forwards the CompC Write Complete Response message to CXRA 208-1 via the CCIX connection. The CXRA, in turn, forwards the CompC Write Complete Response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompC Write Complete Response message, HN-I 204-2 must wait for the BRespC Write Response to arrive from PCIe device 152-2, since PCIe device 152-2 is a PoS in this embodiment.

[0102] Furthermore, in response to receiving the BRespD write response, HN-I 204-2 sends a CompD write complete response message to CXHA 209-2 via the response channel of interconnect 100-2. CXHA, in turn, forwards the CompD write complete response message to CXRA 208-1 via the CCIX connection. CXRA, in turn, forwards the CompD write complete response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompD write complete response message, HN-I 204-2 must wait for the BRespD write response from PCIe device 152-2 to arrive, because in this embodiment, PCIe device 152-2 is a PoS.

[0103] Finally, in response to receiving the CompD write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 5B The lifetime L of the write transactions WriteA, WriteB, WriteC, and WriteD is depicted in A 、L B 、L C and L D .

[0104] As described above, PCIe peer traffic can be a single, large AXI write burst (e.g., 128B, 256B, etc.) originating from one of the PCIe devices 152-1, 152-2. Due to the smaller write transaction size limit (e.g., 64B) of the interconnects 100, 100-1, 100-2, the large write burst must be broken or divided into several smaller write transactions (e.g., 64B) for transmission across the interconnects 100, 100-1, 100-2. After transmission across the interconnects 100, 100-1, 100-2, the smaller write transactions (e.g., 64B) are converted into several corresponding AXI write transactions (e.g., 64B) for transmission to the other PCIe device 152-1, 152-1. Importantly, because the original PCIe peer-to-peer write burst has been divided into several write transactions, and each write transaction includes a message header and a message body, the number of message headers required to transmit the original data through the interconnect 100, 100-1, 100-2 and the final AXI connection to another PCIe device 152-1, 152-1 will be doubled (e.g., 128B AXI write burst), quadrupled (e.g., 256B AXI write burst), etc. As described above, each write transaction through the interconnect requires two response messages, namely DBID_Rsp and Comp.

[0105] Fragmenting or dividing a single, larger PCIe peer-to-peer write burst into multiple, smaller write transactions creates several problems, including, among other things, increased message traffic and message header overhead due to the additional messages, which reduces the available data rate or effective bandwidth, increases the processing load on various components of the system, etc. Additionally, due to the out-of-order nature of the interconnect protocol, multiple smaller write transactions may be reordered for transmission across the interconnects 100, 100-1, 100-2, which is generally incompatible with the AXI protocol. Furthermore, because the PCIe device 152-1 (or 152-2) receiving the PCIe peer-to-peer write burst is the POS of the write transaction, the interconnect protocol cannot issue early completions for any of the smaller write transactions, which prolongs the latency of these write transactions (e.g., Figure 3A 、 Figure 3B 、 Figure 5A 、 Figure 5B L depicted in A 、L B etc.) lifespan.

[0106] In certain embodiments, PCIe peer-to-peer write bursts targeting the same destination PCIe device but originating from different PCIe devices are received by different RN-Is. Each RN-I breaks the received PCIe peer-to-peer write burst into smaller write transactions, which are then sent to the HN-I servicing the destination PCIe device. However, the HN-I cannot distinguish between the smaller write transactions generated by each RN-I, so the HN-I queues all smaller write transactions into the same chain for transmission to the destination PCIe device, which degrades performance.

[0107] These and other factors significantly reduce the efficiency of transmitting PCIe peer-to-peer transactions, such as write bursts, over a coherent interconnect.

[0108] The disclosed embodiments advantageously provide an efficient messaging protocol for a data processing system including one or more interconnects. While applicable to many messaging protocols across an interconnect, the disclosed embodiments advantageously improve the efficiency of PCIe peer-to-peer transactions. For example, by identifying and preserving PCIe peer write bursts and completing interconnect write transactions early while preserving the semantics of the interconnect protocol, the efficiency of PCIe peer write bursts (one-to-many and many-to-one) across the interconnect (on-chip and chip-to-chip) is significantly improved.

[0109] Advantageously, the PCIe message header overhead caused by the AXI to interconnect bridge is reduced, the number of transactions downstream of the HN-I is reduced by combining PCIe endpoint writes and responses, transaction streams from different RN-Is are separated and sorted at the HN-I, the number of interconnect responses is reduced by combining certain interconnect responses, and the lifespan of all interconnect write transactions except the latter interconnect write transaction is significantly shortened, which improves PCIe bus utilization, efficiency and performance, improves interconnect transaction bandwidth, and reduces overall dynamic power consumption.

[0110] Figure 6A A block diagram of another SoC 12 is depicted, according to an embodiment of the present disclosure.

[0111] SoC 12 includes Figure 1A 1 and described above, and also includes a write combiner 160. More specifically, the SoC 12 includes an interconnect 100 coupled to one or more processors 110, one or more accelerators or specialized processors 120, an SMMU 130, one or more memory controllers 140-1, 140-2, and the like, and a NIC device 150. Each memory controller 140-1, 140-2 is coupled to a respective memory 142-1, 142-2, which may be integrated on the SoC 10 or connected externally. PCIe devices 152-1, 152-2 are connected to both the SMMU 130 and the NIC 150.

[0112] In this embodiment, the write combiner 160 is disposed between the interconnect 100 and the NIC 150 and is coupled to the interconnect and the NIC; in other embodiments, the write combiner 160 may be disposed between the NIC 150 and the PCIe devices 152-1 and 152-2 and is coupled to the NIC and the PCIe devices. The write combiner 160 includes a microprocessor, a microcontroller, an ASIC, an FPGA, a logic circuit, etc. to provide related functions, which will be discussed in detail below. In some embodiments, the functions provided by the write combiner 160 may be provided by the NIC 150, such as Figure 1A As described in .

[0113] PCIe endpoint A on PCIe device 152 - 1 and PCIe endpoint B on PCIe device 152 - 2 exchange PCIe peer-to-peer traffic through interconnect 100 , SMMU 130 - 1 , NIC 150 - 1 , and write combiner 160 - 1 .

[0114] Figure 6B Depicts using protocol node naming according to an embodiment of the present disclosure Figure 6A A block diagram of SoC 12 is depicted in FIG.

[0115] SoC 12 includes Figure 1A More specifically, RN-F 210 represents processor 110, RN-F 220 represents accelerator or special purpose processor 120, RN-I 206-1 represents PCIe device 152-1 (master interface), RN-I 206-2 represents PCIe device 152-2 (master interface), HN-F 202-1 manages the address space of memory 142-1 and is associated with SN-F 240-1 representing memory controller 140-1, HN-F 202-2 manages the address space of memory 142-2 and is associated with SN-F 240-2 representing memory controller 140-2, and HN-I 204 manages the address space of PCIe devices 152-1, 152-2 (slave interfaces). In this embodiment, the NIC 150-1 acts as a bridge to the slave interfaces of the PCIe devices 152-1, 152-2, and the SMMU 130 acts as a bridge from the RN-Is 206-1, 206-2 to the master interfaces of the PCIe devices 152-1, 152-2.

[0116] Many aspects of the present disclosure help remove the overhead associated with PCIe write transactions on the interconnect. Fundamentally, RN-I maintains the OWO attributes (e.g., write transaction identifier or AWID) for each flow. Write transactions are always processed sequentially for a specific RN-I and HN-I node pair because RN-I always processes AXI requests in chronological order, so the interconnect maintains requests in the same order that RN-I sends to HN-I, and HN-I processes requests in the same order as they were received, even in the presence of retries. When a large PCIe peer write burst is broken into smaller transactions on the interconnect, the AXI completion (Bresp) depends only on the interconnect completion of the next write in the burst because the next write completion ensures that all previous writes have completed. The write completion of the next write transaction that is not an AXI burst does not have a watcher and is discarded by RN-I.

[0117] The disclosed embodiments advantageously provide a write combiner that uniquely identifies incoming write streams from RN-1 and combines fragmented write transactions back into a single write burst. RN-1 generates a hash of the incoming write burst, such as a burst identifier or BurstID, that uniquely identifies the write burst at its interface and sorts the writes based on the BurstID. For example, the BurstID can be an AWID, a hash of the AWID, etc. RN-1 then sends the BurstID with the write stream to HN-1, which uniquely identifies the source based on information in the interconnect message and generates a stream identifier or StreamID based on the BurstID and source information. The StreamID uniquely identifies the stream.

[0118] HN-1 sorts all published writes based on StreamID to preserve the OWO properties of each stream. The write combiner combines writes with the same StreamID into a burst. RN-1 sends the next write identifier to HN-1 along with the next interconnect transaction of the broken burst. HN-1 then sends the next write identifier to the write combiner with an AXI write transaction. After the write combiner receives the next write identifier, it stops combining AXI write transactions and sends the combined AXI write burst to the PCIe device.

[0119] Multiple write bursts from a single RN-I to multiple HN-Is, and a single HN-I receiving multiple write bursts from multiple RN-Is, are handled in the same way. The source information and BurstID in the interconnect message are sufficient to create a unique StreamID for each flow. This "tunneling" aspect ensures the ordering of the interconnect transactions from a given flow.

[0120] Advantageously, the disclosed embodiments easily accommodate off-chip PCIe peer-to-peer write bursts by propagating the BurstID and RN-I source information across the entire chip. The functionality of the HN-I remains the same.

[0121] In addition, the disclosed embodiments use the last write identifier in the last interconnect write burst at HN-1 to issue a completion notification for the write by providing a response that combines the DBID_Rsp and Comp response into a "CompDBID_Rsp" for all writes except the last fractured write in the stream, even if HN-1 is not the POS of the write. The CompDBID_Rsp combined response advantageously reduces the number of response packets sent through the interconnect and shortens the lifespan of such early completed writes. The early completion feature can be particularly important for embodiments that include CCIX connections with very large latencies for off-chip transactions. The last write in the burst receives a completion response Bresp from the destination PCIe device (slave interface) and incurs a full lifespan due to the full round-trip delay of the write completion. This ensures that all interconnect writes are completed before a Bresp is sent from RN-1 to the originating PCIe device (master interface).

[0122] Figure 7A Depicted is a method for Figure 6A and Figure 6B The protocol flow 400 of the ordered write flow of the SoC 10 is depicted in FIG.

[0123] More specifically, Figure 7A A protocol flow 400 is depicted for an ordered write stream of PCIe peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer traffic flows between PCIe device 152-1 (master interface) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and HN-I 204 over a CHI connection, between HN-I 204 and write combiner 160 over an AXI connection, and between write combiner 160 and PCIe device 152-2 (slave interface) over an AXI connection.

[0124] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written of the AXI write burst to RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. The AXI write burst includes a write transaction identifier or AWID transmitted via the AW channel. For example, the AWID may have a value of 0×01. Because the size of the data to be written is 128 bytes (128B), the RN-I 206-1 breaks or divides the AXI write burst into an ordered write stream, which includes two separate 64-byte (64B) write transactions to be transmitted via the AMBA CHI connection, i.e., a WriteA request message and a WriteB request message. RN-I 206-1 also generates a BurstID and adds it to the WriteA and WriteB request messages. For example, the BurstID may be a hash of the AWID and have a value of 0×10. RN-I 206 - 1 then sends WriteA and WriteB request messages in sequence to HN-I 204 via the request channel of interconnector 100 . These request messages include the address to be written and the BurstID.

[0125] Because the WriteB request message will include the last 64B write transaction for the original AXI write burst, RN-I 206-1 sends a last write identifier with the WriteB request message. For example, the value of the last write identifier can be 1. In other embodiments, RN-I 206-1 sets the value of the last write identifier to 1 for the last write request message and to 0 for the first write request message, and sends the corresponding last write identifier with each write request message. Based on the last write identifier, HN-I 204 determines how many write request messages are included in the ordered write stream for the BurstID.

[0126] In response to receiving the WriteA and WriteB request messages, HN-I 204 identifies the source of the AXI write burst, i.e., PCIe device 152-1 (master interface), based on the information provided in the WriteA request message, generates a StreamID based on the BurstID and the source information, and assigns the StreamID to the WriteA and WriteB request messages. For example, the StreamID may have a value of 0x11.

[0127] Then, HN-I 204 sends a CompDBID_RspA response message and a DBID_RspB response message to RN-I 206-1 in sequence through the response channel of interconnector 100. The CompDBID_RspA response message not only provides the data buffer identifier and StreamID for the WriteA request message, but also indicates that the WriteA write transaction is completed, which significantly shortens the lifetime LA of the WriteA write transaction. Figure 7A As depicted, the DBID_RspB response message provides a data buffer identifier for the WriteB request message and also includes a StreamID.

[0128] In response to receiving the CompDBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to HN-I 204 via the data channel of interconnect 100. The Wr_DataA data message includes the first 64B of the data to be written and the StreamID. Similarly, in response to receiving the DBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to HN-I 204 via the data channel of interconnect 100. The Wr_DataB data message includes the second 64B of the data to be written and the StreamID.

[0129] In response to receiving the Wr_DataA data message, HN-I 204 sends the address to be written and the first 64B of the data to be written to write combiner 160 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204 encodes the StreamID into an AWID and sends the AWID via the AW channel. Similarly, in response to receiving the Wr_DataB data message, HN-I 204 sends the address to be written and the second 64B of the data to be written to write combiner 160 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204 encodes the StreamID into an AWID and sends the AWID via the AW channel. Because the second AXI write request is for the next 64B write transaction of the original AXI write burst, HN-I 204 sends the next write identifier along with the second AXI write request via the AW channel.

[0130] In response to receiving a first AXI write request from HN-I 204, write combiner 160 identifies the StreamID and stores the first 64B of data to be written to PCIe device 152-2 in a local register, memory, etc. In response to receiving a second AXI write request from HN-I 204, write combiner 160 identifies the StreamID and appends the second 64B of data to be written to PCIe device 152-2 in a local register, memory, etc. after the first 64B of data; advantageously, 128B of data are now available for transfer to PCIe device 152-2 in a single AXI write burst. Write combiner 160 also determines that a subsequent write identifier is set (e.g., a value of 1) in the second AXI write request. Generally speaking, the combining process begins when a first AXI write request with a particular StreamID is received and ends when it is determined that the subsequent write identifier is set in a subsequent AXI write request with the same StreamID. Since the latter write identifier has been determined to be set in the second AXI write request, the write combiner 160 completes the combining process and sends 128B of the address to be written and the data to be written to the PCIe device 152-2 (slave interface) through the AXI write address channel (AW) and the write data channel (W), respectively.

[0131] After the 128B write transaction is completed, PCIe device 152-2 (slave interface) sends a BResp write response to write combiner 160 via the write response channel, indicating the status of the combined write transaction. In response to receiving the BResp write response, write combiner 160 sequentially sends BRespA and BRespB write response messages to HN-I 204.

[0132] In response to receiving the BRespA and BRespB write response messages, HN-I 204 sends a CompB write complete response message to RN-I 206-1 via the response channel. As described above, HN-I 204 has sent the BRespA write response as part of the CompDBID_RspA message, which advantageously shortens the life of the WriteA write transaction. A .

[0133] Finally, in response to receiving the CompB write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 7A The lifetime L of the WriteB write transaction is also depicted in B .

[0134] As described above, in some embodiments, the functionality provided by write combiner 160 may be provided by Figure 1AIn these embodiments, the NIC 150 is provided by RN-I 206-1 and HN-I 204 (relative to Figure 7A Discussion) can be provided by RN-I 206-1 and HN-I 204 (relative to Figure 1B Discussion) provided.

[0135] Figure 7B Described is a method for Figure 6A and Figure 6B The protocol flow 402 of the ordered write flow of the SoC 10 is depicted in FIG.

[0136] Protocol flow 402 illustrates an ordered write flow of PCIe peer-to-peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer-to-peer traffic flows between PCIe device 152-1 (master interface) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and HN-I 204 over a CHI connection, between HN-I 204 and write combiner 160 over an AXI connection, and between write combiner 160 and PCIe device 152-2 (slave interface) over an AXI connection.

[0137] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written of the AXI write burst to RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. The AXI write burst includes a write transaction identifier or AWID transmitted via the AW channel. For example, the AWID may have a value of 0×01. Because the size of the data to be written is 256 bytes (256B), the RN-I 206-1 breaks or divides the AXI write burst into four separate 64-byte (64B) write transactions (i.e., WriteA, WriteB, WriteC, and WriteD request messages) for transmission via the AMBA CHI connection. RN-I 206-1 also generates a BurstID and adds it to the WriteA, WriteB, WriteC, and WriteD request messages. For example, the BurstID may be a hash of the AWID and have a value of 0×10. RN-I 206 - 1 then sends WriteA, WriteB, WriteC and WriteD request messages in sequence to HN-I 204 via the request channel of interconnector 100 . These request messages include the address to be written and the BurstID.

[0138] Because the WriteD request message will include the next 64B write transaction for the original AXI write burst, RN-I 206-1 sends the next write identifier along with the WriteD request message. Based on the next write identifier, HN-I 204 determines how many write request messages to include in the ordered write stream for this BurstID.

[0139] In response to receiving the WriteA, WriteB, WriteC, and WriteD request messages, HN-I 204 identifies the source of the AXI write burst, i.e., PCIe device 152-1 (master interface), based on the information provided in the WriteA request message, generates a StreamID based on the BurstID and the source information, and assigns the StreamID to the WriteA, WriteB, WriteC, and WriteD request messages. For example, the StreamID may have a value of 0x11.

[0140] HN-I 204 then sends CompDBID_RspA, CompDBID_RspB, and CompDBID_RspC response messages and a DBID_RspD response message to RN-I 206-1 in sequence via the response channel of interconnector 100. The CompDBID_RspA, CompDBID_RspB, and CompDBID_RspC response messages not only provide the data buffer identifiers for the WriteA, WriteB, and WriteC request messages and StreamID, but also indicate that the WriteA, WriteB, and WriteC write transactions are complete, which significantly shortens the lifespan of the WriteA, WriteB, and WriteC write transactions. A 、L B and L C ,like Figure 7A As depicted, the DBID_RspD response message provides a data buffer identifier for the WriteD request message and also includes a StreamID.

[0141] In response to receiving the CompDBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to HN-I 204 via the data channel of interconnect 100. This data message includes the first 64B of the data to be written and the StreamID. Similarly, in response to receiving the CompDBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to HN-I 204 via the data channel of interconnect 100. This data message includes the second 64B of the data to be written and the StreamID. In response to receiving the CompDBID_RspC response message, RN-I 206-1 sends a Wr_DataC data message to HN-I 204 via the data channel of interconnect 100. This data message includes the third 64B of the data to be written and the StreamID. In response to receiving the DBID_RspD response message, RN-I 206 - 1 sends a Wr_DataB data message to HN-I 204 via the data channel of the interconnector 100 . The data message includes the fourth 64B of the data to be written and the StreamID.

[0142] In response to receiving the Wr_DataA data message, HN-I 204 sends the address to be written and the first 64B of the data to be written to the write combiner 160 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204 encodes the StreamID into an AWID and sends the AWID via the AW channel. Similarly, in response to receiving the Wr_DataB data message, HN-I 204 sends the address to be written and the second 64B of the data to be written to the write combiner 160 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204 encodes the StreamID into an AWID and sends the AWID via the AW channel. In response to receiving the Wr_DataC data message, HN-I 204 sends the address to be written and the third 64B of the data to be written to the write combiner 160 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204 encodes the StreamID into an AWID and sends the AWID via the AW channel. In response to receiving the Wr_DataD data message, the HN-I 204 sends the fourth 64B of the address to be written and the data to be written via the AXI write address channel (AW) and the write data channel (W) to the write combiner 160. The HN-I 204 encodes the StreamID into an AWID and sends the AWID via the AW channel.

[0143] Because the fourth AXI write request is the next 64B write transaction for the original AXI write burst, the HN-I 204 sends the next write identifier along with the fourth AXI write request via the AW channel.

[0144] In response to receiving a first AXI write request from HN-I 204, write assembler 160 identifies the StreamID and stores the first 64B of data to be written to PCIe device 152-2 in a local register, memory, etc. In response to receiving a second AXI write request from HN-I 204, write assembler 160 identifies the StreamID and appends the second 64B of data to be written to PCIe device 152-2 in a local register, memory, etc. In response to receiving a third AXI write request from HN-I 204, write assembler 160 identifies the StreamID and appends the third 64B of data to be written to PCIe device 152-2 in a local register, memory, etc. Furthermore, in response to receiving a fourth AXI write request from HN-I 204, write assembler 160 identifies the StreamID and appends the fourth 64B of data to be written to PCIe device 152-2 in a local register, memory, etc.

[0145] Advantageously, 256B of data are now available for transmission to PCIe device 152-2 in a single AXI write burst. Write combiner 160 also determines that the subsequent write identifier is set (e.g., a value of 1) in the fourth AXI write request. Generally speaking, the combining process begins when the first AXI write request with a particular StreamID is received and ends when it is determined that the subsequent write identifier is set in a subsequent AXI write request with the same StreamID. Since the subsequent write identifier has been determined to be set in the fourth AXI write request, write combiner 160 completes the combining process and sends the address to be written and 256B of data to be written to PCIe device 152-2 (from the interface) via the AXI write address channel (AW) and the write data channel (W), respectively.

[0146] After the 256B write transaction is completed, PCIe device 152-2 (slave interface) sends a BResp write response to write combiner 160 via the write response channel, indicating the status of the combined write transaction. In response to receiving the BResp write response, write combiner 160 sequentially sends BRespA, BRespB, BRespC, and BRespD write response messages to HN-I 204.

[0147] In response to receiving the BRespA, BRespB, BRespC, and BRespD write response messages, HN-I 204 sends a CompD write complete response message to RN-I 206-1 via the response channel. As described above, HN-I 204 has sent the BRespA, BRespB, and BRespC write responses as part of the CompDBID_RspA, CompDBID_RspB, and CompDBID_RspC messages, which advantageously shortens the lifespan of the WriteA, WriteB, and WriteC write transactions. A 、L B and L C .

[0148] Finally, in response to receiving the CompD write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 7A The lifetime L of a WriteD write transaction is also depicted in D .

[0149] As described above, in some embodiments, the functionality provided by write combiner 160 may be provided by Figure 1A In these embodiments, the NIC 150 is provided by RN-I 206-1 and HN-I 204 (relative to Figure 7B Discussion) can be provided by RN-I 206-1 and HN-I 204 (relative to Figure 1B Discussion) provided.

[0150] Figure 8 Depicted is an embodiment of the present disclosure Figure 6B A portion of the block diagram depicted in .

[0151] Generally speaking, Figure 8Two ordered write flows of PCIe peer-to-peer traffic are shown flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. In the first ordered write flow, PCIe peer-to-peer traffic flows between PCIe device 152-1 (master interface) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and HN-I 204 over a CHI connection, between HN-I 204 and write combiner 160 over an AXI connection, and between write combiner 160 and PCIe device 152-2 (slave interface) over an AXI connection. In the second ordered write flow, PCIe peer traffic flows between PCIe device 152-2 (master interface) and RN-I 206-2 over the AXI connection, flows between RN-I 206-2 and HN-I 204 over the CHI connection, flows between HN-I 204 and write combiner 160 over the AXI connection, and flows between write combiner 160 and PCIe device 152-1 (slave interface) over the AXI connection.

[0152] More specifically, a first 128B AXI write burst 410-1 arrives from the PCIe device 152-1 (master interface) to the RN-I 206-1, and a second 128B AXI write burst 410-2 arrives from the PCIe device 152-2 (master interface) to the RN-I 206-2. Because the size of the data to be written is 128 bytes (128B), the RN-I 206-1 breaks or divides the first AXI write burst 410-1 into two separate 64-byte (64B) write transactions 420-1, 422-2 for transmission to the HN-I 204 via the AMBA CHI connection as a first ordered write stream, and the RN-I 206-2 breaks or divides the second AXI write burst 410-2 into two separate 64-byte (64B) write transactions 420-2, 422-2 for transmission to the HN-I 204 via the AMBA CHI connection as a second ordered write stream. RN-I 206-1 includes a subsequent write identifier with the second 64B write transaction 422-1, and RN-I 206-2 includes a subsequent write identifier with the second 64B write transaction 422-2.

[0153] HN-I 204 identifies the source of each ordered write stream and generates a StreamID for each ordered write stream. After exchanging the WriteA and WriteB request messages, CompDBID_RspA and DBID_RspB response messages, and Wr_DataA and Wr_DataB data messages for each write stream (as described above), HN-I 204 sends a first ordered pair of 64B AXI write requests 430-1, 432-1 and a second ordered pair of 64B AXI write requests 430-2, 432-2 to write combiner 160. HN-I 204 includes the latter write identifier with the second AXI write request 432-1, 432-2.

[0154] The write combiner 160 combines the first ordered pair of 64B AXI write requests 430-1 and 432-1 into a 128B AXI write request 440-1 for the first ordered write stream, and combines the second ordered pair of 64B AXI write requests 430-2 and 432-2 into a 128B AXI write request 440-2 for the second ordered write stream. The 128B AXI write request 440-1 for the first AXI write burst 410-1 originating from the PCIe device 152-1 (master interface) is sent to the PCIe device 152-2 (slave interface), while the 128B AXI write request 440-2 for the second AXI write burst 410-2 originating from the PCIe device 152-2 (master interface) is sent to the PCIe device 152-1 (slave interface).

[0155] Figure 9A Depicted is a block diagram of another system 22 having two interconnected SoCs according to an embodiment of the present disclosure.

[0156] System 22 includes Figure 4A 1 and described above, and also includes write combiners 160-1, 160-2. SoC 12-2 and SoC 12-2 are depicted as including a baseline number of components; in other embodiments, SoC 12-1 and SoC 12-2 may include additional components, such as accelerators for specialized processors 120, etc. Additional SoCs 12 may be readily added to system 22, as described below.

[0157] More specifically, SoC 12-1 includes an interconnect 100-1 coupled to one or more processors 110-1, a memory controller 140-1, and a NIC device 150-1. Memory controller 140-1 is coupled to memory 142-1, which may be integrated on SoC 12-1 or connected externally. PCIe device 152-1 (slave interface) is connected to NIC 150-1. In some embodiments, SoC 12-1 may include an SMMU 130-1 (depicted in dashed outline) that connects interconnect 100-1 to PCIe device 152-1 (master interface); in other embodiments, interconnect 100-1 may be directly connected to PCIe device 152-1 (master interface).

[0158] In this embodiment, the write combiner 160-1 is disposed between the interconnect 100-1 and the NIC 150-1 and is coupled to the interconnect and the NIC; in other embodiments, the write combiner 160-1 may be disposed between the NIC 150-1 and the PCIe device 152-1 (slave interface) and is coupled to the NIC and the PCIe device (slave interface). The write combiner 160-1 includes a microprocessor, a microcontroller, an ASIC, an FPGA, a logic circuit, etc. to provide related functions, which will be discussed in detail below. In some embodiments, the functions provided by the write combiner 160-1 may be provided by the NIC 150-1, such as Figure 4A As described in .

[0159] Similarly, SoC 12-2 includes an interconnect 100-2 coupled to one or more processors 110-2, a memory controller 140-2, and a NIC device 150-2. Memory controller 140-2 is coupled to memory 142-2, which may be integrated on SoC 12-2 or connected externally. PCIe device 152-2 (slave interface) is connected to NIC 150-2. In some embodiments, SoC 12-2 may include an SMMU 130-2 (depicted in dashed outline) that connects interconnect 100-2 to PCIe device 152-2 (master interface); in other embodiments, interconnect 100-2 may be directly connected to PCIe device 152-2 (master interface).

[0160] In this embodiment, the write combiner 160-2 is disposed between the interconnect 100-2 and the NIC 150-2 and is coupled to the interconnect and the NIC; in other embodiments, the write combiner 160-2 may be disposed between the NIC 150-2 and the PCIe device 152-2 and is coupled to the NIC and the PCIe device. The write combiner 160-2 includes a microprocessor, a microcontroller, an ASIC, an FPGA, a logic circuit, etc. to provide related functions, which will be discussed in detail below. In some embodiments, the functions provided by the write combiner 160-2 may be provided by the NIC 150-2, such as Figure 4A As described in .

[0161] Interconnect 100-1 may be connected to interconnect 100-2 using two or more CCIX connections 112-1, 112-2; other types of connections are also contemplated. Figure 9A In the depicted embodiment, two CCIX connections 112-1 and 112-2 are depicted, one for each traffic direction. In other embodiments, a single CCIX connection 112 may be shared by all traffic. Generally, one or more CCIX connections 112 form a CCIX interconnect. Additional SoCs 12 can be added to the system 22 simply by adding additional CCIX connections 112 according to a specific connection topology (such as direct, switched, daisy-chained, meshed, etc.). PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2 exchange PCIe peer-to-peer traffic via interconnect 100-1, CCIX connections 112-1 and 112-2, and interconnect 100-2.

[0162] Figure 9B Depicts a method for using a protocol node naming scheme according to an embodiment of the present disclosure. Figure 9A 2 is a block diagram of a system 22 of two interconnected SoCs depicted in FIG.

[0163] System 22 includes Figure 4B1 and all the protocol nodes described above, and also includes write combiners 160-1 and 160-2. More specifically, with respect to SoC 12-1, RN-F 210-1 represents processor 110-1, RN-I 206-1 represents PCIe device 152-1 (master interface), HN-F 202-1 manages the address space of memory 142-1 and is associated with SN-F 240-1 representing memory controller 140-1, and HN-I 204 manages the address space of PCIe device 152-1 (slave interface). In this embodiment, NIC 150-1 acts as a bridge to the slave interface of PCIe device 152-1. If present, SMMU 130-1 can act as a bridge from RN-I 206-1 to the master interface of PCIe device 152-1. CXRA 208-1 performs read and write transactions over CCIX connection 112-1, and CXHA 209-1 manages the consistency of CCIX connection 112-2.

[0164] With respect to SoC 12-2, RN-F 210-2 represents processor 110-2, RN-I 206-2 represents PCIe device 152-2 (master interface), HN-F 202-2 manages the address space of memory 142-2 and is associated with SN-F 240-2, which represents memory controller 140-2, and HN-I 204 manages the address space of PCIe device 152-2 (slave interface). In this embodiment, NIC 150-2 acts as a bridge to the slave interface of PCIe device 152-2. If present, SMMU 130-2 can act as a bridge from RN-I 206-2 to the master interface of PCIe device 152-2. CXRA 208-2 performs read and write transactions over CCIX connection 112-2, and CXHA 209-2 manages the coherency of CCIX connection 112-1.

[0165] Figure 10A Depicted is a method for Figure 9A and Figure 9B The protocol flow 410 of the ordered write stream of the system 22 is depicted in FIG.

[0166] Protocol flow 410 illustrates an ordered write flow of PCIe peer-to-peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer-to-peer traffic flows between PCIe device 152-1 (master) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and CXRA 208-1 over a CHI connection on interconnect 100-1, between CXRA 208-1 and CXHA 290-2 over a CCIX connection, between CXHA 290-2 and HN-I 204-2 over a CHI connection on interconnect 100-2, between HN-I 204-2 and write combiner 160-2 over an AXI connection, and between write combiner 160-2 and PCIe device 152-2 (slave) over an AXI connection.

[0167] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written of the AXI write burst to RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. The AXI write burst includes a write transaction identifier or AWID transmitted via the AW channel. For example, the AWID may have a value of 0×01. Because the size of the data to be written is 128 bytes (128B), the RN-I 206-1 breaks or divides the AXI write burst into two separate 64-byte (64B) write transactions for transmission via the AMBA CHI connection, i.e., a WriteA request message and a WriteB request message. RN-I 206-1 also generates a BurstID and adds it to the WriteA and WriteB request messages. For example, the BurstID may be a hash of the AWID and have a value of 0×10.

[0168] Because the WriteB request message will include the next 64B write transaction for the original AXI write burst, RN-I 206-1 sends the next write identifier along with the WriteB request message. Based on the next write identifier, HN-I 204-2 determines how many write request messages to include in the ordered write stream for this BurstID.

[0169] RN-I 206-1 sends WriteA and WriteB request messages, including the addresses to be written, to CXRA 208-1 via the request channel of interconnector 100-1. CXRA 208-1 forwards the WriteA and WriteB request messages to CXHA 209-2 via the CCIX connection. CXHA then forwards the WriteA and WriteB request messages to HN-I 204-2 via the request channel of interconnector 100-2.

[0170] In response to receiving the WriteA and WriteB request messages, HN-I 204-2 identifies the source of the AXI write burst, i.e., PCIe device 152-1 (master interface), based on the information provided in the WriteA request message, generates a StreamID based on the BurstID and the source information, and assigns the StreamID to the WriteA and WriteB request messages. For example, the StreamID may have a value of 0x11.

[0171] HN-I 204-2 then sends a CompDBID_RspA response message and a DBID_RspB response message, respectively, to CXHA 209-2 via the response channel of interconnect 100-2. These response messages provide the data buffer identifier and StreamID for each write transaction. CXHA 209-2 then forwards the CompDBID_RspA and DBID_RspB response messages to CXRA 208-1 via the CCIX connection. CXRA then forwards the CompDBID_RspA and DBID_RspB response messages to RN-I 206-1 via the response channel of interconnect 100-1.

[0172] The CompDBID_RspA response message not only provides the data buffer identifier and StreamID for the WriteA request message, but also indicates that the WriteA write transaction is completed, which significantly shortens the life of the WriteA write transaction. A ,like Figure 10A As depicted, the DBID_RspB response message provides a data buffer identifier for the WriteB request message and also includes a StreamID.

[0173] In response to receiving the CompDBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to CXRA 208-1 via the data channel of interconnect 100-1. This data message includes the first 64B of the data to be written. Similarly, in response to receiving the DBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to CXRA 208-1 via the data channel of interconnect 100-1. This data message includes the second 64B of the data to be written. CXRA 208-1 forwards the Wr_DataA and Wr_DataB data messages, in sequence, to CXHA 209-2 via the CCIX connection. CXHA then forwards the Wr_DataA and Wr_DataB data messages, in sequence, to HN-I 204-2 via the data channel of interconnect 100-2.

[0174] In response to receiving the Wr_DataA data message, HN-I 204-2 sends the address to be written and the first 64B of the data to be written to write combiner 160-2 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204-2 encodes the StreamID into an AWID and sends the AWID via the AW channel. Similarly, in response to receiving the Wr_DataB data message, HN-I 204-2 sends the address to be written and the second 64B of the data to be written to write combiner 160-2 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204-2 encodes the StreamID into an AWID and sends the AWID via the AW channel. Because the second AXI write request is for the next 64B write transaction of the original AXI write burst, HN-I 204-2 sends the next write identifier along with the second AXI write request via the AW channel.

[0175] In response to receiving a first AXI write request from HN-I 204, write combiner 160-2 identifies the StreamID and stores the first 64B of data to be written to PCIe device 152-2 in a local register, memory, etc. In response to receiving a second AXI write request from HN-I 204-2, write combiner 160-2 identifies the StreamID and appends the second 64B of data to be written to PCIe device 152-2 in a local register, memory, etc. after the first 64B of data; advantageously, 128B of data are now available for transfer to PCIe device 152-2 in a single AXI write burst. Write combiner 160-2 also determines that a subsequent write identifier is set (e.g., a value of 1) in the second AXI write request. Generally speaking, the combining process begins when a first AXI write request with a particular StreamID is received and ends when it is determined that the subsequent write identifier is set in a subsequent AXI write request with the same StreamID. Since the latter write identifier has been determined to be set in the second AXI write request, the write combiner 160-2 completes the combining process and sends 128B of the address to be written and the data to be written to the PCIe device 152-2 (slave interface) through the AXI write address channel (AW) and the write data channel (W), respectively.

[0176] After the 128B write transaction is completed, PCIe device 152-2 (from the interface) sends a BResp write response to write combiner 160-2 via the write response channel, indicating the status of the combined write transaction. In response to receiving the BResp write response, write combiner 160-2 sends BRespA and BRespB write response messages to HN-I 204-2 in sequence.

[0177] In response to receiving the BRespB Write Response, HN-I 204-2 sends a CompB Write Complete Response message to CXHA 209-2 via the response channel of interconnect 100-2. The CXHA, in turn, forwards the CompB Write Complete Response message to CXRA 208-1 via the CCIX connection. The CXRA, in turn, forwards the CompB Write Complete Response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompB Write Complete Response message, HN-I 204-2 must wait for the BRespB Write Response from PCIe device 152-2 to arrive, as PCIe device 152-2 is a PoS in this embodiment.

[0178] Finally, in response to receiving the CompB write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 5A The lifetime L of a WriteB write transaction is depicted in B .

[0179] As described above, in some embodiments, the functionality provided by write combiner 160-2 may be provided by Figure 4A In these embodiments, the NIC 150-2 is provided by RN-I 206-1 and HN-I 204-2 (relative to Figure 10A Discussion) can be provided by RN-I 206-1 and HN-I 204-2 (relative to Figure 4B Similarly, the functionality provided by write combiner 160-1 may be provided by Figure 4A NIC 150-1 etc. as depicted in FIG.

[0180] Figure 10B Described is a method for Figure 9A and Figure 9B The protocol flow 412 of the ordered write stream of the system 22 is depicted.

[0181] Protocol flow 412 illustrates an ordered write flow of PCIe peer-to-peer traffic flowing between PCIe endpoint A on PCIe device 152-1 and PCIe endpoint B on PCIe device 152-2. PCIe peer-to-peer traffic flows between PCIe device 152-1 (master) and RN-I 206-1 over an AXI connection, between RN-I 206-1 and CXRA 208-1 over a CHI connection on interconnect 100-1, between CXRA 208-1 and CXHA 290-2 over a CCIX connection, between CXHA 290-2 and HN-I 204-2 over a CHI connection on interconnect 100-2, between HN-I 204-2 and write combiner 160-2 over an AXI connection, and between write combiner 160-2 and PCIe device 152-2 (slave) over an AXI connection.

[0182] The PCIe device 152-1 (master interface) sends the address to be written and the data to be written of the AXI write burst to RN-I 206-1 via the AXI write address channel (AW) and the write data channel (W), respectively. The AXI write burst includes a write transaction identifier or AWID transmitted via the AW channel. For example, the AWID may have a value of 0×01. Because the size of the data to be written is 256 bytes (256B), the RN-I 206-1 breaks or divides the AXI write burst into four separate 64-byte (64B) write transactions (i.e., WriteA, WriteB, WriteC, and WriteD request messages) for transmission via the AMBA CHI connection. RN-I 206-1 also generates a BurstID and adds it to the WriteA, WriteB, WriteC, and WriteD request messages. For example, the BurstID may be a hash of the AWID and have a value of 0×10.

[0183] Because the WriteD request message will include the next 64B write transaction for the original AXI write burst, RN-I 206-1 sends the next write identifier along with the WriteD request message. HN-I 204-2 determines how many write request messages to include in the ordered write stream for this BurstID based on the next write identifier.

[0184] RN-I 206-1 sequentially sends WriteA, WriteB, WriteC, and WriteD request messages to CXRA 208-1 via the request channel of interconnector 100-1. These request messages include the addresses to be written. CXRA 208-1 sequentially forwards these WriteA, WriteB, WriteC, and WriteD request messages to CXHA 209-2 via the CCIX connection. CXHA then sequentially forwards these WriteA, WriteB, WriteC, and WriteD request messages to HN-I 204-2 via the request channel of interconnector 100-2.

[0185] In response to receiving the WriteA, WriteB, WriteC, and WriteD request messages, HN-I 204-2 identifies the source of the AXI write burst, i.e., PCIe device 152-1 (master interface), based on the information provided in the WriteA request message, generates a StreamID based on the BurstID and the source information, and assigns the StreamID to the WriteA, WriteB, WriteC, and WriteD request messages. For example, the StreamID may have a value of 0x11.

[0186] HN-I 204-2 then sends the CompDBID_RspA, CompDBID_RspB, and CompDBID_RspC response messages, along with a DBID_RspD response message, to CXHA 209-2 via the response channel of interconnect 100-2. These response messages provide the data buffer identifier and StreamID for each write transaction. CXHA 209-2 then forwards the CompDBID_RspA, CompDBID_RspB, CompDBID_RspC, and DBID_RspD response messages to CXRA 208-1 via the CCIX connection. The CXRA then forwards the CompDBID_RspA, CompDBID_RspB, CompDBID_RspC, and DBID_RspD response messages to RN-I 206-1 via the response channel of interconnect 100-1.

[0187] The CompDBID_RspA, CompDBID_RspB, and CompDBID_RspC response messages not only provide the data buffer identifier for the WriteA, WriteB, and WriteC request messages and StreamID, but also indicate that the WriteA, WriteB, and WriteC write transactions are complete, which significantly shortens the lifespan of the WriteA, WriteB, and WriteC write transactions. A 、L B and LC ,like Figure 10B As depicted, the DBID_RspD response message provides a data buffer identifier for the WriteD request message and also includes a StreamID.

[0188] In response to receiving the CompDBID_RspA response message, RN-I 206-1 sends a Wr_DataA data message to CXRA 208-1 via the data channel of interconnect 100-1. The data message includes the first 64B of the data to be written. Similarly, in response to receiving the CompDBID_RspB response message, RN-I 206-1 sends a Wr_DataB data message to CXRA 208-1 via the data channel of interconnect 100-1. The data message includes the second 64B of the data to be written. In response to receiving the CompDBID_RspC response message, RN-I 206-1 sends a Wr_DataC data message to CXRA 208-1 via the data channel of interconnect 100-1. The data message includes the third 64B of the data to be written. In response to receiving the DBID_RspD response message, RN-I 206-1 sends a Wr_DataD data message including the fourth 64B of the data to be written to CXRA 208-1 via the data channel of interconnect 100-1. CXRA 208-1 forwards the Wr_DataA, Wr_DataB, Wr_DataC, and Wr_DataD data messages in sequence to CXHA 209-2 via the CCIX connection. CXHA then forwards the Wr_DataA, Wr_DataB, Wr_DataC, and Wr_DataD data messages in sequence to HN-I 204-2 via the data channel of interconnect 100-2.

[0189] In response to receiving the Wr_DataA data message, HN-I 204-2 sends the address to be written and the first 64B of the data to be written to write combiner 160-2 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204-2 encodes the StreamID into an AWID and sends the AWID via the AW channel. Similarly, in response to receiving the Wr_DataB data message, HN-I 204-2 sends the address to be written and the second 64B of the data to be written to write combiner 160-2 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204-2 encodes the StreamID into an AWID and sends the AWID via the AW channel. In response to receiving the Wr_DataC data message, HN-I 204-2 sends the address to be written and the third 64B of the data to be written to write combiner 160-2 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204-2 encodes the StreamID into an AWID and sends the AWID via the AW channel. Furthermore, in response to receiving the Wr_DataD data message, HN-I 204-2 sends the fourth 64B of the address to be written and the data to be written to write combiner 160-2 via the AXI write address channel (AW) and the write data channel (W), respectively. HN-I 204-2 encodes the StreamID into an AWID and sends the AWID via the AW channel.

[0190] Because the fourth AXI write request is the next 64B write transaction for the original AXI write burst, HN-I 204 - 2 sends the next write identifier along with the fourth AXI write request through the AW channel.

[0191] In response to receiving a first AXI write request from HN-I 204, write assembler 160-2 identifies the StreamID and stores the first 64B of data to be written to PCIe device 152-2 (slave interface) in a local register, memory, etc. In response to receiving a second AXI write request from HN-I 204-2, write assembler 160-2 identifies the StreamID and appends the second 64B of data after the first 64B of data to be written to PCIe device 152-2 (slave interface) in a local register, memory, etc. In response to receiving a third AXI write request from HN-I 204-2, write assembler 160-2 identifies the StreamID and appends the third 64B of data after the second 64B of data to be written to PCIe device 152-2 (slave interface) in a local register, memory, etc. Also, in response to receiving the fourth AXI write request from HN-I 204-2, write combiner 160-2 identifies the StreamID and appends the fourth 64B of data after the third 64B of data to be written to PCIe device 152-2 (slave interface) in a local register, memory, etc.

[0192] Advantageously, 256B of data are now available for transmission to PCIe device 152-2 in a single AXI write burst. Write combiner 160 also determines that the subsequent write identifier is set (e.g., a value of 1) in the fourth AXI write request. Generally speaking, the combining process begins when the first AXI write request with a particular StreamID is received and ends when it is determined that the subsequent write identifier is set in a subsequent AXI write request with the same StreamID. Since the subsequent write identifier has been determined to be set in the fourth AXI write request, write combiner 160 completes the combining process and sends the address to be written and 256B of data to be written to PCIe device 152-2 (from the interface) via the AXI write address channel (AW) and the write data channel (W), respectively.

[0193] After the 256B write transaction is completed, PCIe device 152-2 (slave interface) sends a BResp write response to write combiner 160-2 via the write response channel, indicating the status of the combined write transaction. In response to receiving the BResp write response, write combiner 160-2 sequentially sends BRespA, BRespB, BRespC, and BRespD write response messages to HN-I 204-2.

[0194] In response to receiving the BRespD Write Response, HN-I 204-2 sends a CompD Write Complete Response message to CXHA 209-2 via the response channel of interconnect 100-2. CXHA, in turn, forwards the CompD Write Complete Response message to CXRA 208-1 via the CCIX connection. CXRA, in turn, forwards the CompD Write Complete Response message to RN-I 206-1 via the response channel of interconnect 100-1. Before sending the CompD Write Complete Response message, HN-I 204-2 must wait for the BRespD Write Response from PCIe device 152-2 to arrive, as PCIe device 152-2 is a PoS in this embodiment.

[0195] Finally, in response to receiving the CompD write completion response message, the RN-I 206-1 sends a BResp write completion response message to the PCIe device 152-1 (master interface) through the response channel. Figure 10B The lifetime L of a WriteD write transaction is depicted in D .

[0196] In the above-described embodiment, the first write burst is an AXI write burst having a data size greater than 64 bytes, each write request is an AMBA CHI write request having a data size of 64 bytes or less, each write transaction is an AXI write transaction having a data size of 64 bytes or less, and the second write burst is an AXI write burst having a data size greater than 64 bytes. The embodiments of the present disclosure are not limited to these protocols and data sizes; other write burst protocols, interconnect protocols, data sizes, etc. are also contemplated, such as write bursts of 64 bytes or less, first and subsequent write requests having different data sizes, etc.

[0197] Figure 11 Depicted is a flowchart 500 representing functionality associated with a messaging protocol for a data processing network, according to an embodiment of the present disclosure.

[0198] Functions at 510, 520, and 530 are performed at a requesting node coupled to the interconnect, functions at 540 and 550 are performed at a master node coupled to the interconnect, and functions at 560 and 570 are performed at a write combiner coupled to the master node.

[0199] At 510, a first write burst is received from a first device over a first connection. The first write burst has a data size.

[0200] At 520, the first write burst is divided into an ordered sequence of write requests based on the first write burst data size. The ordered sequence of write requests includes at least a first write request and a subsequent write request. Each write request has the same data size, which is smaller than the first write burst data size.

[0201] The ordered sequence of write requests is sent to the master node over the interconnect at 530. The subsequent write request includes a subsequent write identifier.

[0202] At 540, an ordered sequence of write transactions is generated based on the ordered sequence of write requests. The ordered sequence of write transactions includes at least a first write transaction and a subsequent write transaction. Each write transaction has a data size equal to the data size of each write request.

[0203] The ordered sequence of write transactions is sent to a write combiner at 550. The subsequent write transaction includes a subsequent write identifier.

[0204] The ordered sequence of write transactions is combined into a second write burst based on at least the latter write identifier at 560. The second write burst has a data size equal to the first write burst data size.

[0205] At 570, the second write burst is sent to the second device over the second connection.

[0206] The disclosed embodiments advantageously provide an efficient message protocol for a data processing system including one or more interconnects. While applicable to many message protocols that may pass through an interconnect, the disclosed embodiments advantageously improve the efficiency of PCIe peer-to-peer transactions, as discussed in detail below.

[0207] The embodiments described above and outlined below are combinable.

[0208] In one embodiment, a computer-based method for transmitting data over at least one interconnect is provided. At a requesting node coupled to the interconnect: receiving a first write burst from a first device via a first connection; dividing the first write burst into an ordered sequence of write requests based on the first write burst data size, the ordered sequence of write requests including at least a first write request and a subsequent write request, each write request having a data size less than the first write burst data size; and sending the ordered sequence of write requests to a master node coupled to the interconnect, the subsequent write request including a subsequent write identifier. At the master node: generating an ordered sequence of write transactions based on the ordered sequence of write requests, the ordered sequence of write transactions including at least a first write transaction and a subsequent write transaction, each write transaction having a data size equal to the data size of each write request; and sending the ordered sequence of write transactions to a write combiner coupled to the master node, the subsequent write transaction including the subsequent write identifier. At the write combiner: combining the ordered sequence of write transactions into a second write burst based at least on the subsequent write identifier, the second write burst having a data size equal to the first write burst data size; and sending the second write burst to a second device via a second connection.

[0209] In another embodiment of the method, a sum of the data sizes of the write requests in the ordered sequence of write requests is equal to the first write burst data size.

[0210] In another embodiment of the method, the first write burst includes a write transaction identifier, and dividing the first write burst includes: generating a burst identifier based on the write transaction identifier; and adding the burst identifier to each write request in the ordered sequence of write requests.

[0211] In another embodiment of the method, generating the ordered sequence of write transactions includes generating a stream identifier based on the burst identifier; and sending the ordered sequence of write transactions includes sending the stream identifier with each write transaction in the ordered sequence of write transactions.

[0212] In another embodiment of the method, said combining the ordered sequence of write transactions is further based on the stream identifier.

[0213] In another embodiment of the method, the method further includes: at the master node: in response to receiving the first write request, sending a first write response to the requesting node, the first write response including a first data buffer identifier and an indication that the first write request is completed; and in response to receiving the subsequent write request, sending a subsequent write response to the requesting node, the subsequent write response including a subsequent data buffer identifier, wherein the lifetime of the first write request is equal to the time between sending the first write request and receiving the first write response.

[0214] In another embodiment of the method, the method further includes: at the write combiner: sending a first write transaction response to the master node, the first write transaction response including an indication that the first write transaction is completed; and sending a subsequent write transaction response to the master node, the subsequent write transaction response including an indication that the subsequent write transaction is completed; and, at the master node: in response to receiving the subsequent write transaction response, sending a final write response to the requesting node, the final write response including an indication that the subsequent write request is completed, wherein the lifetime of the subsequent write request is equal to the time between sending the subsequent write request and receiving the final write response.

[0215] In another embodiment of the method, the ordered sequence of write requests includes one or more intermediate write requests, and the method further includes: at the master node: in response to receiving the intermediate write request, sending an intermediate write response to the requesting node, the intermediate write response including an intermediate data buffer identifier and an indication that the intermediate write request is completed; and at the write combiner: sending an intermediate write transaction response to the master node, the intermediate write transaction response including an indication that the intermediate write transaction is completed, wherein the lifetime of each intermediate write request is equal to the time between sending the intermediate write request and receiving the intermediate write response, and wherein the lifetime of the latter write request is greater than the lifetime of each of the first write request and the intermediate write request.

[0216] In another embodiment of the method, the first write burst is an Advanced eXtensible Interface (AXI) write burst having a data size greater than 64 bytes, each write request is an Advanced Microcontroller Bus Architecture (AMBA) Coherent Interconnect (CHI) write request having a data size of 64 bytes or less, each write transaction is an AXI write transaction having a data size of 64 bytes or less, and the second write burst is an AXI write burst having a data size greater than 64 bytes.

[0217] In another embodiment of the method, the requesting node is coupled to a first interconnect within a first system on chip (SoC), the master node is coupled to a second interconnect within a second SoC, the first interconnect being coupled to the second interconnect via a chip-to-chip interconnect, and the computer-based method further comprises: at a requesting agent coupled to the first interconnect and the chip-to-chip interconnect: receiving an ordered sequence of write requests from the requesting node via the first interconnect; and sending an ordered sequence of write requests via the chip-to-chip interconnect; and at a master agent coupled to the first interconnect and the chip-to-chip interconnect: receiving an ordered sequence of write requests from the requesting agent via the chip-to-chip interconnect; and sending an ordered sequence of write requests to the master node via the second interconnect.

[0218] In one embodiment, a system includes: an interconnect; a requesting node coupled to the interconnect and comprising a requesting node processor; a master node coupled to the interconnect and comprising a master node processor; and a write combiner coupled to the master node and comprising a write combiner processor. The requesting node processor is configured to: receive a first write burst having a data size from a first device via a first connection; divide the first write burst into an ordered sequence of write requests based on the first write burst data size, the ordered sequence of write requests comprising at least a first write request and a subsequent write request, each write request having a data size less than the first write burst data size; and send the ordered sequence of write requests to the master node via the interconnect, the subsequent write request comprising a subsequent write identifier. The master node processor is configured to: generate an ordered sequence of write transactions based on the ordered sequence of write requests, the ordered sequence of write transactions comprising at least a first write transaction and a subsequent write transaction, each write transaction having a data size equal to the data size of each write request; and send the ordered sequence of write transactions to the write combiner, the subsequent write transaction comprising the subsequent write identifier. The master node processor is configured to: combine the ordered sequence of write transactions into a second write burst based at least on the latter write identifier, the second write burst having a data size equal to the first write burst data size; and send the second write burst to a second device via a second connection.

[0219] In another embodiment of the system, a sum of the data sizes of the write requests in the ordered sequence of write requests is equal to the first write burst data size.

[0220] In another embodiment of the system, the first write burst includes a write transaction identifier, and dividing the first write burst includes: generating a burst identifier based on the write transaction identifier; and adding the burst identifier to each write request in the ordered sequence of write requests.

[0221] In another embodiment of the system, generating the ordered sequence of write transactions includes generating a stream identifier based on the burst identifier; and sending the ordered sequence of write transactions includes sending the stream identifier with each write transaction in the ordered sequence of write transactions.

[0222] In another embodiment of the system, said combining the ordered sequence of the write transactions is further based on the stream identifier.

[0223] In another embodiment of the system, the master node processor is further configured to: in response to receiving the first write request, send a first write response to the requesting node, the first write response including a first data buffer identifier and an indication that the first write request is completed; and in response to receiving the subsequent write request, send a subsequent write response to the requesting node, the subsequent write response including a subsequent data buffer identifier, wherein the lifetime of the first write request is equal to the time between sending the first write request and receiving the first write response.

[0224] In another embodiment of the system, the write combiner processor is further configured to: send a first write transaction response to the master node, the first write transaction response including an indication that the first write transaction is completed; and send a subsequent write transaction response to the master node, the subsequent write transaction response including an indication that the subsequent write transaction is completed; and the master node processor is further configured to: in response to receiving the subsequent write transaction response, send a final write response to the requesting node, the final write response including an indication that the subsequent write request is completed, wherein the lifetime of the subsequent write request is equal to the time between sending the subsequent write request and receiving the final write response.

[0225] In another embodiment of the system, the ordered sequence of write requests includes one or more intermediate write requests; the master node processor is further configured to: in response to receiving the intermediate write request, send an intermediate write response to the requesting node, the intermediate write response including an intermediate data buffer identifier and an indication that the intermediate write request is completed; and the write combiner processor is further configured to: send an intermediate write transaction response to the master node, the intermediate write transaction response including an indication that the intermediate write transaction is completed, wherein the lifespan of each intermediate write request is equal to the time between sending the intermediate write request and receiving the intermediate write response, and wherein the lifespan of the latter write request is greater than the lifespan of each of the first write request and the intermediate write request.

[0226] In another embodiment of the system, the first write burst is an Advanced eXtensible Interface (AXI) write burst having a data size greater than 64 bytes, each write request is an Advanced Microcontroller Bus Architecture (AMBA) Coherent Interconnect (CHI) write request having a data size of 64 bytes or less, each write transaction is an AXI write transaction having a data size of 64 bytes or less, and the second write burst is an AXI write burst having a data size greater than 64 bytes.

[0227] In another embodiment of the system, the request node is coupled to a first interconnect within a first system on chip (SoC), the master node is coupled to a second interconnect within a second SoC, the first interconnect is coupled to the second interconnect via a chip-to-chip interconnect, and the system further comprises: a request agent coupled to the first interconnect and the chip-to-chip interconnect, comprising a request agent processor configured to: receive an ordered sequence of write requests from the request node via the first interconnect; and send an ordered sequence of write requests via the chip-to-chip interconnect; and a master agent coupled to the first interconnect and the chip-to-chip interconnect, comprising a master agent processor configured to: receive an ordered sequence of write requests from the request agent via the chip-to-chip interconnect; and send an ordered sequence of write requests to the master node via the second interconnect.

[0228] Although the present disclosure is capable of many different forms of implementation, specific embodiments are shown in the drawings and will be described in detail herein, it should be understood that the disclosure should be considered as an exemplification of the principles of the disclosure and is not intended to limit the disclosure to the specific embodiments shown and described. In the above description, like reference numerals may be used to describe the same, similar or corresponding parts in the several views of the drawings.

[0229] In this document, the terms "comprises," "comprising," "containing," "having," or any other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but may also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "comprises ... and" does not, without more constraints, preclude the presence of additional identical elements in the process, method, article, or apparatus that comprises that element.

[0230] Reference throughout this document to "one embodiment," "certain embodiments," "many embodiments," "an embodiment," "implementation," "aspect," or similar terms means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of such phrases in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

[0231] As used herein, the term "or" is to be interpreted as inclusive or meaning any one or any combination. Thus, "A, B, or C" means "any of: A; B; C; A and B; A and C; B and C; A, B, and C." Exceptions to this definition occur only when the combination of elements, functions, steps, or actions is inherently mutually exclusive in some way. Additionally, grammatical conjunctions are intended to express any and all separate and connected combinations of the connected items, sentences, words, etc., unless otherwise specified or clear from the context. Thus, the term "or" should generally be understood to mean "and / or," etc. References to items in the singular should be understood to include the items in the plural, and vice versa, unless expressly stated otherwise or clear from the text.

[0232] Unless otherwise indicated, the description of the range of values ​​herein is not intended to be limiting, but refers individually to any and all values ​​falling within the range, and each individual value in such a range is incorporated into this specification as if individually stated herein. When accompanied by a numerical value, the words "about", "approximately" etc. should be understood to indicate a deviation, which, as will be understood by one of ordinary skill in the art, operates satisfactorily for the intended purpose. The range of values ​​and / or numerical values ​​is provided herein only as an example and does not constitute a limitation on the scope of the embodiments. The use of any and all examples or exemplary language ("for example", "such as", etc.) provided herein is only intended to better illustrate the embodiments, rather than to limit the scope of the embodiments. Any language in this specification should not be understood to represent that any unclaimed element is essential for the practice of the embodiments.

[0233] For simplicity and clarity of description, reference numerals may be repeated in the accompanying drawings to indicate corresponding or similar elements. Many details are set forth to provide an understanding of the embodiments described herein. The embodiments may be practiced without these details. In other cases, well-known methods, procedures, and components are not described in detail to avoid obscuring the embodiments. This specification should not be considered to be limited to the scope of the embodiments described herein.

[0234] Many features and advantages of the present disclosure are apparent from the detailed description, and therefore, the appended claims are intended to cover all such features and advantages of the present disclosure that fall within the scope of the present disclosure. Furthermore, since numerous modifications and variations will readily occur to those skilled in the art, it is not desired to limit the present disclosure to the exact construction and operation shown and described, and therefore, all suitable modifications and equivalents may be resorted to as falling within the scope of the present disclosure.

Claims

1. A computer-based method for transmitting data via at least one interconnect, comprising: At a requesting node coupled to the interconnector: receiving a first write burst having a data size from a first device over a first connection; dividing the first write burst into an ordered sequence of write requests based on the first write burst data size, the ordered sequence of write requests comprising at least a first write request and a subsequent write request, each write request having a data size smaller than the first write burst data size; sending, through the interconnect, the ordered sequence of write requests to a master node coupled to the interconnect, the subsequent write request including a subsequent write identifier; At the master node: generating an ordered sequence of write transactions based on the ordered sequence of write requests, the ordered sequence of write transactions comprising at least a first write transaction and a subsequent write transaction, each write transaction having a data size equal to a data size of each write request; sending the ordered sequence of write transactions to a write combiner coupled to the master node, the subsequent write transaction including the subsequent write identifier; At the write combiner: combining the ordered sequence of write transactions into a second write burst based at least on the latter write identifier, the second write burst having a data size equal to the first write burst data size; and The second write burst is sent to the second device over the second connection. 2 . The computer-based method of claim 1 , wherein a sum of the data sizes of the write requests in the ordered sequence of write requests is equal to the first write burst data size.

3. The computer-based method of claim 1 , wherein the first write burst includes a write transaction identifier, and wherein dividing the first write burst includes: generating a burst identifier based on the write transaction identifier; as well as The burst identifier is added to each write request in the ordered sequence of write requests.

4. The computer-based method of claim 3, wherein: said generating the ordered sequence of write transactions comprises generating a stream identifier based on the burst identifier; and The sending the ordered sequence of write transactions includes sending the stream identifier with each write transaction in the ordered sequence of write transactions.

5. The computer-based method of claim 4, wherein said combining the ordered sequence of write transactions is further based on said stream identifier.

6. The computer-based method of claim 1 , further comprising: At the master node: In response to receiving the first write request, sending a first write response to the requesting node, the first write response including a first data buffer identifier and an indication that the first write request is complete; as well as In response to receiving the subsequent write request, sending a subsequent write response to the requesting node, the subsequent write response including a subsequent data buffer identifier, The lifetime of the first write request is equal to the time between sending the first write request and receiving the first write response.

7. The computer-based method of claim 6, further comprising: At the write combiner: Sending a first write transaction response to the master node, the first write transaction response including an indication that the first write transaction is complete; sending a subsequent write transaction response to the master node, the subsequent write transaction response including an indication that the subsequent write transaction is complete; and At the master node: In response to receiving the subsequent write transaction response, sending a final write response to the requesting node, the final write response including an indication that the subsequent write request is complete, The lifetime of the subsequent write request is equal to the time between sending the subsequent write request and receiving the last write response.

8. The computer-based method of claim 7, wherein the ordered sequence of write requests includes one or more intermediate write requests, and the method further comprises: At the master node: In response to receiving the intermediate write request, sending an intermediate write response to the requesting node, the intermediate write response including an intermediate data buffer identifier and an indication that the intermediate write request is complete; as well as At the write combiner: sending an intermediate write transaction response to the master node, the intermediate write transaction response including an indication of completion of the intermediate write transaction, wherein the lifetime of each intermediate write request is equal to the time between sending the intermediate write request and receiving the intermediate write response, and The lifetime of the subsequent write request is greater than the lifetime of the first write request and each of the intermediate write requests.

9. The computer-based method of claim 1 , wherein the first write burst is an Advanced eXtensible Interface (AXI) write burst having a data size greater than 64 bytes, each write request is an Advanced Microcontroller Bus Architecture (AMBA) Coherent Interconnect (CHI) write request having a data size of 64 bytes or less, each write transaction is an AXI write transaction having a data size of 64 bytes or less, and the second write burst is an AXI write burst having a data size greater than 64 bytes.

10. The computer-based method of claim 1 , wherein the requesting node is coupled to a first interconnect within a first system on chip (SoC), the master node is coupled to a second interconnect within a second SoC, the first interconnect being coupled to the second interconnect via a chip-to-chip interconnect, and the computer-based method further comprising: At a requesting agent coupled to the first interconnect and the chip-to-chip interconnect: receiving the ordered sequence of write requests from the requesting node over the first interconnect; sending the ordered sequence of write requests through the chip-to-chip interconnect; At a host agent coupled to the first interconnect and the chip-to-chip interconnect: receiving the ordered sequence of write requests from the requesting agent over the chip-to-chip interconnect; and The ordered sequence of write requests is sent to the master node over the second interconnect.

11. A data processing system comprising: Interconnectors; a requesting node coupled to the interconnector, comprising a requesting node processor; a master node, the master node coupled to the interconnect, comprising a master node processor; and a write combiner coupled to the master node, comprising a write combiner processor, The request node processor is configured as follows: receiving a first write burst having a data size from a first device over a first connection, dividing the first write burst into an ordered sequence of write requests based on the first write burst data size, the ordered sequence of write requests comprising at least a first write request and a subsequent write request, each write request having a data size smaller than the first write burst data size, sending the ordered sequence of write requests to the master node via the interconnect, the subsequent write request including a subsequent write identifier, The master node processor is configured as follows: generating an ordered sequence of write transactions based on the ordered sequence of write requests, the ordered sequence of write transactions comprising at least a first write transaction and a subsequent write transaction, each write transaction having a data size equal to a data size of each write request, sending the ordered sequence of write transactions to the write combiner, the subsequent write transaction including the subsequent write identifier, wherein the write combiner is configured to: combining the ordered sequence of write transactions into a second write burst based at least on the latter write identifier, the second write burst having a data size equal to the first write burst data size, and The second write burst is sent to the second device over the second connection. 12 . The system of claim 11 , wherein a sum of the data sizes of the write requests in the ordered sequence of write requests is equal to the first write burst data size.

13. The system of claim 11 , wherein the first write burst includes a write transaction identifier, and the dividing the first write burst includes: generating a burst identifier based on the write transaction identifier; as well as The burst identifier is added to each write request in the ordered sequence of write requests.

14. The system of claim 13, wherein: said generating the ordered sequence of write transactions comprises generating a stream identifier based on the burst identifier; and The sending the ordered sequence of write transactions includes sending the stream identifier with each write transaction in the ordered sequence of write transactions.

15. The system of claim 14, wherein the ordered sequence of combining the write transactions is further based on the stream identifier.

16. The system of claim 11, wherein the master node processor is further configured to: In response to receiving the first write request, sending a first write response to the requesting node, the first write response including a first data buffer identifier and an indication that the first write request is complete; and In response to receiving the subsequent write request, sending a subsequent write response to the requesting node, the subsequent write response including a subsequent data buffer identifier, The lifetime of the first write request is equal to the time between sending the first write request and receiving the first write response.

17. The system of claim 16, wherein: The write combiner processor is further configured to: Sending a first write transaction response to the master node, the first write transaction response including an indication that the first write transaction is complete; sending a subsequent write transaction response to the master node, the subsequent write transaction response including an indication that the subsequent write transaction is complete; and The master node processor is further configured to: In response to receiving the subsequent write transaction response, sending a final write response to the requesting node, the final write response including an indication that the subsequent write request is complete, The lifetime of the subsequent write request is equal to the time between sending the subsequent write request and receiving the last write response.

18. The system of claim 17, wherein: The ordered sequence of write requests includes one or more intermediate write requests; The master node processor is further configured to: In response to receiving the intermediate write request, sending an intermediate write response to the requesting node, the intermediate write response including an intermediate data buffer identifier and an indication that the intermediate write request is complete; The write combiner processor is further configured to: Sending an intermediate write transaction response to the master node, the intermediate write transaction response including an indication of completion of the intermediate write transaction; The lifetime of each intermediate write request is equal to the time between sending the intermediate write request and receiving the intermediate write response; and The lifespan of the subsequent write request is greater than the lifespan of the first write request and each of the intermediate write requests.

19. The system of claim 11, wherein the first write burst is an Advanced eXtensible Interface (AXI) write burst having a data size greater than 64 bytes, each write request is an Advanced Microcontroller Bus Architecture (AMBA) Coherent Interconnect (CHI) write request having a data size of 64 bytes or less, each write transaction is an AXI write transaction having a data size of 64 bytes or less, and the second write burst is an AXI write burst having a data size greater than 64 bytes.

20. The system of claim 11 , wherein the requesting node is coupled to a first interconnect within a first system on chip (SoC), the master node is coupled to a second interconnect within a second SoC, the first interconnect being coupled to the second interconnect via a chip-to-chip interconnect, and the system further comprising: a request agent coupled to the first interconnect and the chip-to-chip interconnect, comprising a request agent processor configured to: receiving the ordered sequence of write requests from the requesting node over the first interconnect; sending the ordered sequence of write requests through the chip-to-chip interconnect; and a master agent coupled to the first interconnect and the chip-to-chip interconnect, comprising a master agent processor configured to: receiving the ordered sequence of write requests from the requesting agent over the chip-to-chip interconnect; and The ordered sequence of write requests is sent to the master node over the second interconnect.

Images