Unified address space for multiple hardware accelerators using dedicated low latency link

JP2024099640A5Active Publication Date: 2024-11-06XILINX INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024067099
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2018-07-26
Filing Date
2024-04-17
Publication Date
2024-11-06
Estimated Expiration
2039-07-25

AI Technical Summary

Technical Problem

In traditional heterogeneous computing platforms, adding additional hardware accelerators increases bus contention and bandwidth constraints, requiring updates to applications and data transfer through the host processor, which hampers system performance.

Method used

A unified address space and direct accelerator links between hardware accelerators are implemented, allowing data transfer without passing through the host processor, reducing bus bandwidth usage and enabling seamless addition of accelerators without modifying applications.

Benefits of technology

This approach enhances system performance by reducing bus bandwidth constraints and allowing efficient addition of hardware accelerators without application modifications, facilitating faster data transfers and improved computational efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

To provide a system, an integrated circuit, and a method by which a bandwidth used by hardware accelerators on a bus may be significantly reduced, thereby increasing overall system performance.SOLUTION: A system 100 includes a host processor 105 coupled to a communication bus, a first hardware accelerator 135-1 and a second hardware accelerator 135-2 communicatively linked to the host processor 105 through the communication bus. The first hardware accelerator 135-1 and the second hardware accelerator 135-2 are directly coupled through an accelerator link independent of the communication bus. The host processor 105 is configured to initiate data transfer between the first hardware accelerator 135-1 and the second hardware accelerator 135-2 directly through the accelerator link.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to hardware acceleration, and more particularly to enabling the use of multiple hardware accelerators through a unified address space and low latency communication links. [Background technology]

[0002] A heterogeneous computing platform (HCP) refers to a data processing system that includes a host processor connected to one or more other devices through interface circuitry. The devices are generally architecturally distinct from the host processor. The host processor can offload tasks to the devices. The devices can perform the tasks and make results available to the host processor. As an illustrative example, the host processor is generally implemented as a central processing unit and the devices are implemented as graphics processing units (GPUs) and / or digital signal processors (DSPs).

[0003] In other HCPs, one or more of the devices performing tasks offloaded from the host processor include devices adapted for hardware acceleration (called "hardware accelerators"). A hardware accelerator includes circuitry capable of performing tasks offloaded from the host as opposed to executing software or program code to perform the tasks. The circuitry of a hardware accelerator is functionally equivalent to executing software, but is generally capable of completing the task in less time.

[0004] Examples of hardware accelerators include programmable ICs, such as field programmable gate arrays (FPGAs), partially programmable integrated circuits (ICs), application specific ICs (ASICs), etc. Obviously, an HCP may include a combination of different devices, one or more adapted to execute program code and one or more others adapted for hardware acceleration. Summary of the Invention

[0005] In one or more embodiments, a system may include a host processor connected to a communication bus, a first hardware accelerator communicatively linked to the host processor through the communication bus, and a second hardware accelerator communicatively linked to the host processor through the communication bus. The first hardware accelerator and the second hardware accelerator are directly connected through an accelerator link that is independent of the communication bus. The host processor is configured to initiate a data transfer between the first hardware accelerator and the second hardware accelerator directly through the accelerator link.

[0006] In one or more embodiments, the hardware accelerator may include an endpoint configured to communicate with a host processor over a communication bus, a memory controller connected to a memory local to the hardware accelerator, and a linking circuit connected to the endpoint and the memory controller. The linking circuit is configured to establish an accelerator link with a target hardware accelerator also connected to the communication bus. The accelerator link is a direct connection between the hardware accelerator and the target hardware accelerator, independent of the communication bus.

[0007] In one or more embodiments, a method may include receiving, in a first hardware accelerator, an instruction and a target address for a data transfer sent from a host processor via a communication bus; the first hardware accelerator comparing the target address with an upper limit of an address range corresponding to the first hardware accelerator; and in response to determining that the target address exceeds the address range based on the comparing, the first hardware accelerator initiating a transaction with a second hardware accelerator to perform the data transfer using an accelerator link directly connecting the first hardware accelerator and the second hardware accelerator.

[0008] This summary section is provided merely to introduce some concepts, and is not intended to identify key or essential features of the claimed subject matter. Other features of the inventive subject matter will be apparent from the accompanying drawings and the detailed description that follows.

[0009] Configurations of the present invention are illustrated by way of example in the accompanying drawings. However, the drawings should not be construed as limiting the configuration of the present invention to only the particular implementations shown. Various aspects and advantages will become apparent upon consideration of the following detailed description and upon reference to the drawings. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of a system having multiple hardware accelerators. [Diagram 2] FIG. 1 illustrates an exemplary implementation of a hardware accelerator. [Diagram 3] FIG. 2 illustrates an example of a retransmit engine (RTE). [Figure 4] FIG. 2 illustrates an exemplary method of operation for a system with multiple hardware accelerators. [Diagram 5] FIG. 1 illustrates an example of a system with multiple hardware accelerators and one or more additional devices. [Figure 6] FIG. 1 illustrates an exemplary architecture for an integrated circuit (IC). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present disclosure concludes with claims defining novel features, but it is believed that the various features described within the present disclosure will be better understood by considering the description in conjunction with the drawings. The process(es), machine(s), product(s) and any variations thereof described herein are provided for illustrative purposes. The specific structural and functional details described within the present disclosure should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art to employ the described features in various ways in nearly all appropriately detailed structures. Moreover, the terms and phrases used within the present disclosure are not limiting, but rather provide an understandable description of the described features.

[0012] The present disclosure relates to hardware acceleration, and more particularly to enabling the use of multiple hardware accelerators through a unified address space and low latency communication links. The use of hardware accelerators with data processing systems has become an effective technique for offloading tasks from a host processor, thereby reducing the workload on the host processor. Hardware accelerators are typically attached to the host processor through a bus. For example, hardware accelerators may be attached to a circuit board that is inserted into an available bus slot of the host system. Typically, each hardware accelerator is attached to a corresponding circuit board. Adding additional hardware accelerators to a system typically involves inserting additional circuit boards with the hardware accelerators into available bus slots.

[0013] In conventional systems, applications executed by the host processor must be updated and / or rewritten, particularly to access any newly added hardware accelerators (e.g., by hardware address). Furthermore, to transfer data from one hardware accelerator to another, the data is moved from the source hardware accelerator to the host processor and then from the host processor to the target hardware accelerator. Data moves to and from each hardware accelerator through the host processor via the bus. Thus, each additional hardware accelerator added to the system increases the number of devices on the bus, thereby creating contention for bandwidth on the bus. As the complexity, number, and / or size of the tasks performed by the hardware accelerators (or other devices) increase, the available bandwidth on the bus becomes even more constrained.

[0014] According to the inventive configurations described within this disclosure, a unified address space for the devices is provided. Additionally, a direct communication link between the hardware accelerators, referred to herein as an "accelerator link," is provided that can operate independently of the bus. Runtime libraries and drivers executed by the host can leverage the unified address space so that applications executed by the host processor can operate without directly referencing (e.g., addressing) specific hardware accelerators in the system. The runtime libraries can determine the appropriate addresses to use to effectuate data transfers between the hardware accelerators. Thus, applications do not need to be modified to access additional hardware accelerators that may be added to the system. Furthermore, data transfers can be performed over the accelerator link, which allows data to be transferred directly from one hardware accelerator to another without passing through the host processor, effectively bypassing the bus. Thus, the bandwidth used by the hardware accelerators on the bus can be significantly reduced, thereby increasing overall system performance.

[0015] As stated, additional hardware accelerators may be added to the system using existing address space without requiring corresponding changes or modifications to the program code (e.g., applications) executed by the host processor. This is supported, at least in part, through the implementation of an automatic discovery process for hardware accelerator boards and adding such boards to the system, the use of remote buffer flags versus local buffer flags, automatic switching to the accelerator link for data transfers in at least some cases, and automatic address translation for remote buffers.

[0016] Further aspects of the inventive configuration are described in more detail below with reference to the figures. For simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, dimensions of some of the elements may be exaggerated relative to other elements for clarity. Furthermore, where considered appropriate, reference numerals have been repeated among the figures to indicate corresponding, similar or similar features.

[0017] 1 illustrates an example of a system 100 having multiple hardware accelerators. System 100 is an example of computer hardware that may be used to implement a computer, server, or other data processing system. System 100 is also an example of a heterogeneous computing system. As depicted, system 100 includes at least one host processor 105 connected to a host memory 110 through an interface circuit 115.

[0018] System 100 also includes multiple hardware accelerators 135. In the example of Figure 1, system 100 includes three hardware accelerators 135-1, 135-2, and 135-3. Although the example of Figure 1 shows three hardware accelerators, it should be appreciated that system 100 may include fewer than three hardware accelerators or more than three hardware accelerators. Additionally, system 100 may include one or more other devices, such as a graphics processing unit (GPU) or a digital signal processor (DSP).

[0019] System 100 may store computer-readable instructions (also referred to as "program code") in host memory 110. Host memory 110 is an example of a computer-readable storage medium. Host processor 105 may execute program code accessed from host memory 110 via interface circuitry 115. In one or more embodiments, host processor 105 communicates with host memory 110 through a memory controller (not shown).

[0020] The host memory 110 may include one or more physical memory devices, such as, for example, local memory and bulk storage devices. Local memory generally refers to a non-persistent memory device(s) used during the actual execution of the program code. Examples of local memory include random access memory (RAM) and / or any of the various types of RAM suitable for use by the processor during execution of the program code, such as DRAM, SRAM, DDR SDRAM, etc. Bulk storage devices refer to persistent data storage devices. Examples of bulk storage devices include, but are not limited to, hard disk drives (HDD), solid state drives (SSD), flash memory, read only memory (ROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other suitable memory. The system 100 may also include one or more cache memories (not shown) that provide temporary storage of at least some program code to reduce the number of times the program code must be retrieved from a bulk storage device during execution.

[0021] Host memory 110 may store program code and / or data. For example, host memory 110 may store operating system 120, instructions 125, and data 130. In the example of FIG. 1, instructions 125 may include one or more applications 170, runtime library 172 (referred to herein as “runtime”), and driver 174 that may communicate with hardware accelerator 135. Runtime 172 may handle completion events, manage command queues, and provide notifications to application(s) 170. Data 130 may include buffer objects, such as buffer objects 176 and 178, that enable direct data transfer between hardware accelerator 135, among other types of data items. Buffer object 176 includes remote flag 180, and buffer object 178 includes remote flag 182. For purposes of illustration, remote flag 180 is not set and remote flag 182 is set. System 100, for example, host processor 105, is capable of executing operating system 120 and instructions 125 to perform the operations described within this disclosure.

[0022] Examples of interface circuitry 115 include, but are not limited to, a system bus and an input / output (I / O) bus. Interface circuitry 115 may be implemented using any of a variety of bus architectures. Examples of bus architectures may include, but are not limited to, an Extended Industry Standard Architecture (EISA) bus, an Accelerated Graphics Port (AGP), a Video Electronics Standards Association (VESA) local bus, a Universal Serial Bus (USB), and a Peripheral Component Interconnect Express (PCIe) bus. Host processor 105 may be connected to host memory 110 through an interface circuit different from that used to couple to hardware accelerator 135. For purposes of illustration, endpoints for interface circuitry 115 through which host processor 105 communicates with other devices are not shown.

[0023] System 100 may further include one or more other I / O devices (not shown) connected to interface circuit 115. I / O devices may be connected to system 100, e.g., interface circuit 115, either directly or through an intervening I / O controller. Examples of I / O devices include, but are not limited to, a keyboard, a display device, a pointing device, one or more communication ports, and a network adapter. A network adapter refers to circuitry that enables system 100 to become connected to other systems, computer systems, remote printers, and / or remote storage devices through an intervening private or public network. Modems, cable modems, Ethernet cards, and wireless transceivers are examples of different types of network adapters that may be used with system 100.

[0024] In the example of Figure 1, each of the hardware accelerators 135-1, 135-2, and 135-3 is connected to a memory 140-1, 140-2, and 140-3, respectively. The memories 140-1, 140-2, and 140-3 are generally implemented as RAMs as described with respect to the host memory 110. In one or more embodiments, each hardware accelerator 135 is implemented as an IC. The IC may be a programmable IC. One example of a programmable IC is a field programmable gate array (FPGA).

[0025] In the example of FIG. 1, each of the hardware accelerators 135 includes an endpoint 145, a link circuit 150, a memory controller 155 (abbreviated as “MC” in FIG. 1), and an interconnect circuit 168. Each hardware accelerator 135 also includes one or more computation units (abbreviated as “CU” in FIG. 1). A computation unit is a circuit capable of performing tasks offloaded from the host processor 105. For purposes of illustration, each of the hardware accelerators 135 is shown to include a computation unit 160 and a computation unit 165. It should be appreciated that the hardware accelerators 135 may include fewer or more computation units than are shown.

[0026] In one example, each of the endpoints 145 is implemented as a PCIe endpoint. It should be appreciated that the endpoints 145 may be implemented as any type of endpoint suitable for communicating over the particular type or implementation of the interface circuitry 115 used by the system 100. Each of the memory controllers 155 is coupled to a respective memory 140 to enable access (e.g., reading and writing) of the memory 140 by the hardware accelerator 135.

[0027] In one or more embodiments, hardware accelerator 135-1 and memory 140-1 are mounted on a first circuit board (not shown), hardware accelerator 135-2 and memory 140-2 are mounted on a second circuit board (not shown), and hardware accelerator 135-3 and memory 140-3 are mounted on a third circuit board (not shown). Each of these circuit boards may include a suitable connector for coupling to a bus port or slot. For example, each of the circuit boards may have a connector configured for insertion into an available PCIe slot (or other bus / interface connector) of system 100.

[0028] Each of the link circuits 150 is capable of establishing an accelerator link with at least one other, e.g., adjacent, link circuit 150. As used herein, an "accelerator link" refers to a communication link that directly connects two hardware accelerators. For example, each of the circuit boards having hardware accelerators 135 may be connected through a wire that connects to a link circuit 150. The link circuit 150 may establish the accelerator link through the wire.

[0029] In a particular embodiment, the link circuits 150 are communicatively linked using a ring topology. Data sent over the accelerator link(s) established by the link circuits 150 master from left to right as indicated by the directional arrows. For example, referring to the example of FIG. 1, the link circuit on the left (e.g., link circuit 150-1) may act as a master, and the adjacent link circuit on the right (e.g., link circuit 150-2) may act as a slave. Similarly, the link circuit 150-2 may act as a master with respect to the link circuit 150-3. The link circuit 150-3 may act as a master with respect to the link circuit 150-1.

[0030] In one or more embodiments, each link circuit 150 includes a table or register that specifies the amount (or size) of memory 140 for each hardware accelerator (e.g., on each board). Using the table, each link circuit 150 can modify addresses specified in transactions for purposes of exchanging information using the accelerator link. In certain embodiments, the table or register is static. In one or more other embodiments, a driver can read and / or update the information stored in the table or register dynamically, e.g., at run-time.

[0031] For illustrative purposes, the operation of hardware accelerator 135-2 will be described. It should be appreciated that like-numbered components in each respective hardware accelerator may operate in the same or similar manner. Thus, with reference to hardware accelerator 135-2, link circuit 150-2 may receive transactions from any of a variety of different sources or initiators and route transactions to any of a variety of targets. For example, link circuit 150-2 may receive transactions from endpoint 145-2 (e.g., originating from host processor 105), transactions from computation unit 160-2, transactions from computation unit 165-2, transactions from hardware accelerator 135-1 via link circuit 150-1, or transactions from hardware accelerator 135-3 that flow through link circuit 150-3 to link circuit 150-1 and then onto link circuit 150-2. Link circuit 150-2 can route transactions to any target, such as a transaction to endpoint 145-2 (e.g., to host processor 105), a transaction to computation unit 160-2, a transaction to computation unit 165-2, a transaction to memory controller 155-2, a transaction to hardware accelerator 135-1 on link circuit 150-1 via link circuit 150-3, or a transaction to hardware accelerator 135-3 via link circuit 150-3, where the target is different from the source or initiator.

[0032] For example, host processor 105 may access any location in memory 140-1, memory 140-2, and / or memory 140-3 as part of a unified address space. However, in accessing such memory, host processor 105 may access such memory by accessing a selected hardware accelerator, e.g., hardware accelerator 135-2, and then using an accelerator link to reach any target, such as memory 140-1, memory 140-2, or memory 140-3, through the selected hardware accelerator.

[0033] As an illustrative and non-limiting example, host processor 105 may initiate a data transfer involving hardware accelerators 135-2 and 135-3. Hardware accelerator 135-2 may be the initiator. In this example, host processor 105, e.g., runtime 172 and / or driver 174, creates buffer object 176 corresponding to hardware accelerator 135-2 and buffer object 178 corresponding to hardware accelerator 135-3. Host processor 105 sets remote flag 182 to indicate that the target address for the data transfer (in hardware accelerator 135-3) is remote to the initiating hardware accelerator (hardware accelerator 135-2).

[0034] The endpoint 145-2 may receive an offloaded task from the host processor 105 via the interface circuit 115. In one or more embodiments, the host processor 105 may view the hardware accelerator 135 as a unified address space via executing the runtime 172 and the driver 174. The endpoint 145-2 may provide a task (e.g., data) to the computation unit 160-2. The task may specify a target address in the memory 140-3 from which the computation unit 160-2 should retrieve data to perform the offloaded task. The hardware accelerator 135-2 may use the link circuit 150-2 to initiate and perform data transfers directly with the hardware accelerator 135-3 via an accelerator link established between the link circuit 150-2 and the link circuit 150-3.

[0035] Although the data transfer is initiated by the host processor 105, the data transfer is performed using the link circuitry 150 and occurs without involving the host processor 105, the host memory 110, or the interface circuitry 115. The data transfer occurs directly between the hardware accelerators. In a conventional system, the data transfer would occur by the host processor 105 taking data from the hardware accelerator 135-3 via the interface circuitry 115 and then providing the data to the hardware accelerator 135-2 via the interface circuitry 115.

[0036] The ability of the hardware accelerators 135 to read and write data between themselves without moving that data through the host processor 105 significantly reduces the amount of data passed over the interface circuitry 115 (e.g., the PCIe bus). This conserves significant bandwidth of the interface circuitry 115 for use in communicating data between the host processor 105 and other hardware accelerators 135. Additionally, the speed of operation of the system 100 may be increased due to the reduction in the time required for the hardware accelerators 135 to share data.

[0037] System 100 may include fewer components than those shown, or additional components not shown in FIG. 1, depending on the particular type of device and / or system being implemented. Additionally, the particular operating system, application(s), and / or I / O devices included may vary based on the system type. Additionally, one or more of the illustrated components may be incorporated into or otherwise form a portion of another component. For example, a processor may include at least some memory. System 100 may be used to implement a single computer or multiple networked or interconnected computers, each implemented using the architecture of FIG. 1 or a similar architecture.

[0038] Figure 2 illustrates an example implementation of the hardware accelerator 135-2 of Figure 1. An example implementation of the link circuit 150-2 is provided in Figure 2. It should be appreciated that the architecture illustrated for the link circuit 150-2 in Figure 2 can be used to implement any of the link circuits 150 illustrated in Figure 1.

[0039] In one or more embodiments, the link circuit 150-2 can convert transactions to be sent to other hardware accelerators into data stream-based packets and route the packets over accelerator links established between the link circuits 150. In a particular embodiment, the link circuit 150-2 can convert AMBA Extensible Interface (AXI) compliant memory-mapped transactions into an AXI data stream for transmission. Within this disclosure, AXI is used as an exemplary communication protocol. It should be appreciated that other communication protocols may be used. In this regard, the use of AXI is for purposes of illustration and not limitation. The link circuit 150-2 can also handle incoming packets from other hardware accelerators (e.g., hardware accelerators 135-1 and 135-3), convert the packets into memory-mapped transactions, and route the data locally within the hardware accelerator 135-2. Additionally, link circuitry 150-2 can convert received packets into memory-mapped transactions, modify the transactions, convert the memory-mapped transactions back into packets, and pass the packets on to the next hardware accelerator. Data received over the accelerator link can be routed internally within hardware accelerator 135-2 as memory-mapped transactions.

[0040] 2, link circuitry 150-2 includes transceivers 202 and 204, retransmission engines (RTEs) 206 and 208, and memory map-to-stream (MM-stream) mappers 210 and 212. The MM-stream mappers 210 and 212 are connected to interconnect circuitry 214.

[0041] As depicted, transceiver 202 may be connected to a corresponding transceiver in hardware accelerator 135-1, and transceiver 204 is connected to a corresponding transceiver in hardware accelerator 135-3. Transceivers 202 and 204 implement the physical layer of an accelerator link established with another hardware accelerator. Each of transceivers 202 and 204 may implement a lightweight serial communication protocol for a multi-gigabit communication link. In one or more embodiments, each of transceivers 202 and 204 may implement a bidirectional interface to a transceiver in an adjacent IC. Transceivers 202 and 204 may automatically initialize an accelerator link with another hardware accelerator. In general, transceivers 202 and 204 are capable of bidirectional communication to implement low-level signaling related to flow control and low PHY level protocols. However, data flow may be implemented using a ring topology as previously described and flowing from master to slave (eg, in a single direction around the ring).

[0042] For example, transceiver 202 may communicate bidirectionally with a corresponding transceiver in link circuit 150-1 of hardware accelerator 135-1. Transceiver 204 may communicate bidirectionally with a corresponding transceiver in link circuit 150-3 of hardware accelerator 135-3. Each of transceivers 202 and 204 may communicate with an adjacent transceiver using a data stream, e.g., an AXI data stream.

[0043] In a particular embodiment, the transceivers 202 and 204 are capable of sending and receiving data to an adjacent hardware accelerator using 8B / 10B coding rules, and each of the transceivers 202 and 204 is capable of detecting single-bit errors and most multi-bit errors using the 8B / 10B coding rules.

[0044] In one or more embodiments, each of the transceivers 202 and 204 is implemented as an Aurora 8B / 10B IP core, available from Xilinx, Inc. of San Jose, Calif. However, it should be appreciated that the particular core referenced is provided for purposes of illustration and is not intended as a limitation. Other transceivers capable of operating as described herein may be used.

[0045] The transceiver 202 is connected to an RTE 206. The transceiver 202 and the RTE 206 are capable of communicating through multiple data streams moving in each direction supporting bidirectional communication. The transceiver 204 is connected to an RTE 208. The transceiver 204 and the RTE 208 are capable of communicating through multiple data streams moving in each direction supporting bidirectional communication.

[0046] The RTEs 206 and 208 can manage transactions. In one or more embodiments, the RTEs 206 and 208 each implement additional layers of a communication protocol on top of those implemented by the transceivers 202 and 204, respectively. For example, the RTEs 206 and 208 each implement a transaction layer (TL) / link layer (LL) and a user layer. These additional layers provide further guarantees regarding data integrity. After initialization, the application can pass data across the accelerator link as a stream of data. The additional data integrity measures are particularly beneficial when converting memory-mapped transactions to stream data, since control signals are merged with the data. Data integrity issues can result in corrupted control signals. The on-chip interconnects and / or buses cannot tolerate data loss on control signals.

[0047] The TL / LL implements a token-based flow control to ensure lossless data communication. In one or more embodiments, the communication channels between adjacent transceivers and between the transceivers and the RTE are 128 bits wide. When sending data, each RTE can check that the receiving link circuitry in the target hardware accelerator has sufficient buffering resources (e.g., tokens) to receive the entire transaction before actually sending the transaction to the physical layer implemented by the transceiver. For example, the RTE 206 may check that the receiving link circuitry 150-1 in the hardware accelerator 135-1 has sufficient buffer resources to receive the data before providing the data to the transceiver 202 (in the link circuitry 150-2) for sending.

[0048] The RTEs 206 and 208 can detect data corruption. For example, each of the RTEs 206 and 208 can verify packet length information, packet sequence information, and / or cyclic redundancy check (CRC) checksum for each received packet. When an RTE slave (e.g., a receiving RTE) detects a packet error, the RTE can enter error abort mode. In error abort mode, the RTE drops the packet with the error as a failed packet. The RTE further drops all subsequent packets of the transaction. In a particular embodiment, the initiation of error abort mode causes the RTE to initiate a link retry sequence. If the link retry sequence is successful, the link master (e.g., a sending RTE) can restore the transmission by starting from the point of failure.

[0049] RTE 206 is connected to MM stream mapper 210. RTE 206 can communicate with MM stream mapper 210 via multiple data streams moving in each direction supporting bidirectional communication. RTE 208 is connected to MM stream mapper 212. RTE 208 can communicate with MM stream mapper 212 via multiple data streams moving in each direction supporting bidirectional communication.

[0050] Each of the MM stream mappers 210 and 212 is connected to an interconnect circuit 214. The interconnect circuit 214 can route data between the MM stream mappers 210 and 212 and other master and / or slave circuits of the hardware accelerator 135-2 connected thereto. The interconnect circuit 214 can be implemented as one or more on-chip interconnects. One example of an on-chip interconnect is an AXI bus. An AXI bus is an embedded microcontroller bus interface for use in establishing on-chip connections between circuit blocks and / or systems. Other exemplary implementations of the interconnect circuit can include, but are not limited to, other buses, crossbars, networks-on-chip (NoC), and the like.

[0051] The MM stream mappers 210 and 212 can convert received data streams from the RTEs 206 and 208, respectively, into memory-mapped transactions that can be provided to the interconnect circuit block 214. In this regard, the data streams can be demultiplexed into multiple channels supporting memory-mapped transactions. The MM stream mappers 210 and 212 can also convert received memory-mapped transactions from the interconnect circuit block 214 into stream data that can be provided to the RTEs 206 and 208, respectively. The MM stream mappers 210 and 212 can multiplex multiple channels supporting memory-mapped transactions (e.g., including control signals as described) into a single data stream for sending to the RTEs 206 and 208, respectively.

[0052] In one or more embodiments, each of the MM stream mappers 210 and 212 is capable of adjusting a target address received in a transaction. The MM stream mapper 210 may, for example, subtract an upper limit of an address range for the hardware accelerator 135-2 (e.g., the address range of the memory 140-2) from the target address of the transaction when receiving the transaction from the hardware accelerator 135-1 over the accelerator link. By adjusting the target address as the transaction passes through the link circuitry 150, the transaction may be directed from one hardware accelerator to another over the accelerator link. Further details related to the operation of addresses when using accelerator links are described in more detail with respect to FIG. 4.

[0053] For illustrative purposes, other portions of hardware accelerator 135-2 are described with respect to link circuitry 150-2. In the example of FIG. 2, interconnect circuitry 214 is connected to a direct memory access (DMA) master circuitry 216. DMA master circuitry 216 includes, for example, a memory-mapped interface for communicating with interconnect circuitry block 214. DMA master circuitry 216 is connected to a PCIe endpoint 218. PCIe endpoint 218, an exemplary implementation of endpoint 145-2 of FIG. 1, is communicatively linked to host processor 105.

[0054] 2, the interconnect circuitry 214 is also connected to one or more computation unit masters 220-1 through 220-N. Each computation unit master 220 provides a bidirectional interface between the computation units implemented within the hardware accelerator 135-2 and the interconnect circuitry block 214. Each computation unit master 220 further includes a memory-mapped interface for communicating with the interconnect circuitry block 214. Each of the computation units 160-2 and 165-2 may be connected to the interconnect circuitry 214 via a slave interface (not shown).

[0055] 2, interconnect circuitry 214 is also connected to one or more memory controller slave circuits 225-1 through 225-N. Each memory controller slave circuit 225 enables read and write operations for memory 140-2. Memory 140-2 may be implemented as one or more off-chip memories accessible by hardware accelerator 135-2. Each of memory controllers 225-1 through 225-N further includes a memory-mapped interface for communicating with interconnect circuitry block 214.

[0056] Figure 3 illustrates one example implementation of RTE 206. The example architecture described with respect to Figure 3 implements a credit-based flow control / retransmission control scheme that uses flow control units (FLITs). RTE 206 is capable of translating between FLIT-based protocols and / or interfaces used internally to protocols and / or interfaces that may be used by applications.

[0057] The RTE 206 includes a transmit channel 330. The transmit channel 330 is capable of de-encapsulating data (e.g., AXI) streams into FLIT-based transactions. In the example of FIG. 3, the transmit channel 330 includes a transmit (TX) packet cyclic redundancy check (CRC) generator 302, a retry pointer return command (PRET) packet / initial retry command (IRTRY) packet generator and return retry pointer (RRP) padder 304, a token return (TRET) packet generator and sequence (SEQ) number / forward retry pointer (FRP) / return token count (RTC) padder 306, a flow control circuit 308, and an output buffer 310. The TRET generator and SEQ / FRP / RTC padder 306 are also connected to a retry buffer 312.

[0058] The RTE 206 includes a receive channel 340. The receive channel 340 is capable of encapsulating a FLIT-based interface and converting the interface into a data (e.g., AXI) stream. In the example of Figure 3, the receive channel 340 includes a packet boundary detector 316, a receive (RX) packet CRC circuit 318, a RX packet processor 320, and an input buffer 322. The Rx packet processor 320 is connected to an error handler 324 and a retry sequence circuit 314.

[0059] The RTE 206 is provided for purposes of illustration and not limitation. It should be appreciated that other architectures suitable for implementing a credit-based flow control / retransmission control scheme may be used. The architecture described with respect to Figure 3 may also be used to implement the RTE 208 of Figure 2 with a reversed or inverted orientation with respect to data flow.

[0060] 4 illustrates an exemplary method 400 of operation for a system with multiple hardware accelerators. Method 400 illustrates an example of direct data transfer between hardware accelerators. Method 400 may be implemented by a system the same as or similar to system 100 described with respect to FIG. 1. Method 400 illustrates how insufficient bandwidth on a bus coupling a host processor and a hardware accelerator may be mitigated. Data transfers that would normally occur on the bus may be directed to an accelerator link, thereby freeing up bandwidth on the bus for other operations.

[0061] In block 405, the system can automatically discover the hardware accelerator sequence. In one or more embodiments, the hardware accelerators, e.g., boards of the hardware accelerators, are configured in a ring topology in the system. The host processor is aware of the existing PCIe topology and thus the number of hardware accelerators present in the system connected to the PCIe bus. In addition, the host processor is aware, e.g., via runtime, of the specific circuitry (e.g., image or configuration bitstream) loaded into each hardware accelerator. Thus, the host processor is aware that the hardware accelerators support accelerator links as described herein. The host processor must still determine the sequence of the hardware accelerators. The driver can, for example, implement the auto-discovery of the hardware accelerator sequence described. This auto-discovery capability supports the addition of new and / or additional hardware accelerators to the system without the need to modify applications executed by the host processor.

[0062] Each hardware accelerator may have a known and the same address range. For example, each hardware accelerator may be assumed to have an address range of 16 GB, corresponding to the 16 GB of memory 140. In one or more embodiments, the host processor may write unique values ​​to memory addresses at 16 GB intervals. The host processor may then read the values ​​again to determine the sequence of the hardware accelerators in the ring topology based on the written and read values.

[0063] In block 410, the host processor may create buffers on each hardware accelerator upon startup. For example, a runtime executed by the host processor may communicate with each hardware accelerator to create buffers in each respective hardware accelerator's memory. With reference to FIG. 1, hardware accelerator 135-1 creates buffers in memory 140-1. Hardware accelerator 135-2 creates buffers in memory 140-2. Hardware accelerator 135-3 creates buffers in memory 140-3.

[0064] At block 415, the host processor initiates a data transfer between the hardware accelerators. The data transfer may be, for example, part of a task to be offloaded from the host processor to the hardware accelerator. As an illustrative and non-limiting example, the host processor 105 may offload a task for an application to the computation unit 160-1 of the hardware accelerator 135-1. The task may include an instruction and a target address from which the computation unit 160-1 should obtain data for the task. The target address in this example is in the hardware accelerator 135-2 (e.g., in memory 140-2). Thus, to perform the task offloaded from the host processor, the computation unit 160-1 must retrieve data from the target address in memory 140-2.

[0065] In block 420, the runtime may request a data transfer between hardware accelerator 135-1 and hardware accelerator 135-2. For example, the runtime may request a read of hardware accelerator 135-2 by or from hardware accelerator 135-1.

[0066] At block 425, the driver may create a buffer object in host memory corresponding to hardware accelerator 135-2 and a buffer object in host memory corresponding to hardware accelerator 135-1. The buffer objects are shadow data structures implemented in host memory. Each buffer object may correspond to or represent a device in the system. The buffer objects may contain data that supports management functions performed by a runtime executed by the host processor.

[0067] In one or more embodiments, a buffer object created in host memory may include a remote flag. The remote flag may be set to indicate that the buffer object is remote from the perspective of the hardware accelerator initiating the transaction. In this example, hardware accelerator 135-1 is reading data from hardware accelerator 135-2. Thus, hardware accelerator 135-1 is initiating the transaction. The driver sets the remote flag in the buffer object that corresponds to hardware accelerator 135-2 upon creation.

[0068] In block 430, the runtime library initiates access to a buffer object (e.g., a remote buffer object) by the initiating hardware accelerator. The runtime library initiates access of a buffer object corresponding to hardware accelerator 135-2 from hardware accelerator 135-1. For example, the runtime determines that a remote flag is set in the buffer object for hardware accelerator 135-2. In response to determining that the remote flag is set, the runtime library schedules a transfer using the accelerator link established by the link circuitry. In scheduling a transfer using the accelerator link between the hardware accelerators, the runtime determines an address to be used by hardware accelerator 135-1 to access data from hardware accelerator 135-2.

[0069] For illustrative purposes, consider an example in which each of hardware accelerators 135 has an address range of 1 to 1000. In such an example, the runtime may determine that data to be retrieved by hardware accelerator 135-1 from hardware accelerator 135-2 is in a buffer at address 500 corresponding to hardware accelerator 135-2 (e.g., at address 500 corresponding to memory 140-2). In this example, the runtime adds 1000 to the target address, resulting in an address of 1500, which is provided to hardware accelerator 135-1 as the target address for retrieving data to operate on for the offloaded task.

[0070] As another example, if data was stored at address 500 in memory 140-3, the runtime would add 2000 for the transaction to reach hardware accelerator 135-3, assuming each of the hardware accelerators 135 has an address range of 1 to 1000. Generally, as is known, the return path data may be tracked through the on-chip bus interconnect (e.g., AXI interconnect) used. When a read request from a master is issued, for example, the read request is routed through the interconnect to a slave with a series of address decodes and / or address shifts (performed by the mm stream mapper) as the read request traverses across each hardware accelerator. Each individual interconnect may track which master has an outstanding transaction to each slave. When the read data is returned, it may be sent back through the correct interface(s). In some cases, to return a particular read data, an identifier (ID) bit may be used to associate the read data with a particular master.

[0071] In block 435, an initiating hardware accelerator (e.g., a first hardware accelerator) receives a task from a host processor. Endpoint 145-1 may, for example, receive the task and provide the task to computation unit 160-1. The task specifies that data to be operated on by computation unit 160-1 is located at a target address, which in this example is 1500. Computation unit 160-1 may, for example, have a control port on which the target address may be stored. In attempting to access data located at address 1500, computation unit 160-1 recognizes that the address is not within the range of hardware accelerator 135-1. For example, computation unit 160-1 may compare the address to an upper limit of an address range of 1000 and determine that the address exceeds the upper limit. In this example, computation unit 160-1 may initiate a read transaction from address 1500. For example, computation unit 160-1 may initiate the read transaction as a memory-mapped transaction sent over interconnect 214.

[0072] In block 440, the initiating hardware accelerator accesses the target hardware accelerator (e.g., a second hardware accelerator) via an accelerator link. For example, link circuit 150-1 can convert memory-mapped transactions initiated by computation unit 160-1 (e.g., using an MM stream mapper) into stream-based packets. Link circuit 150-1 can further encode the packets with additional data supporting data integrity checking, retransmission, initialization, and error reporting (e.g., using an RPE). The ring topology can master from left to right. Thus, packets can be output by the transceivers of link circuit 150-1 to link circuit 150-2.

[0073] Link circuit 150-2 receives data streams at transceiver 202 and processes transactions at RTE 206. MM stream mapper 210 can perform various operations in response to receiving stream database packets. MM stream mapper 210 can, for example, convert stream-based packets to memory-mapped transactions. Additionally, MM stream mapper 210 can decrement the target address of 1500 by the upper limit of the address range of hardware accelerator 135-2. As mentioned, the upper limit may be stored in a table or register within link circuit 150-2, for example, in MM stream mapper 210. In this example, MM stream mapper 210 decrements the target address of 1500 by 1000, resulting in a target address of 500. Because the target address is local to hardware accelerator 135-2, hardware accelerator 135-2 can act on the received transaction. In this example, the MM stream mapper 210 provides the memory-mapped transaction to the interconnect 214. The memory-mapped transaction may be provided to the memory controller 155-2 (e.g., through a memory controller slave) to perform the read transaction. In this manner, the hardware accelerator 135-1 may read data from (or write data to) the hardware accelerator 135-2. The requested data may be provided to the requester from the memory 140-2 using the same path used to send the read request. For example, data read from the memory 140-2 may be sent from the hardware accelerator 135-2 to the hardware accelerator 135-1 without having to traverse onward through the ring topology to the hardware accelerator 135-3 and then to the hardware accelerator 135-1.

[0074] For example, if the target address was 2500, the decrement result would be 1500. In that case, MM stream mapper 210 determines that the target address is not in hardware accelerator 135-2 because the target address is greater than the upper limit of the address range for hardware accelerator 135-2 (e.g., 1000). In that case, MM stream mapper 210 may send the transaction through the interconnect circuitry to MM stream mapper 212 for forwarding onto the next hardware accelerator.

[0075] In block 445, the compute unit 160-1 in the hardware accelerator 135-1 may generate an interrupt to the host processor to inform the host processor that the data transfer between the hardware accelerators is complete. In block 450, the runtime may provide the necessary notification to the application that the data transfer is complete. The runtime may handle, for example, completion events, command queuing, and notifications to the application.

[0076] In one or more embodiments, a PCIe endpoint and a DMA master can write to a target address that is in a different hardware accelerator. As an illustrative and non-limiting example, a host processor can send data to hardware accelerator 135-1 with a target address that is in hardware accelerator 135-2. The DMA master can then recognize that the target address is in a different hardware accelerator and schedule a data transfer over the accelerator link. For example, the DMA master can compare the target address to an upper limit of an address range for hardware accelerator 135-1. In response to determining that the target address exceeds the upper limit, the DMA master can initiate a memory-mapped transaction over the interconnect circuit to MM stream mapper 212 in link circuit 150-1 for sending over the accelerator link to hardware accelerator 135-2.

[0077] In one or more embodiments, the host processor may use the accelerator links for load balancing purposes. For example, the host processor may use runtime to determine the status of a DMA channel (e.g., a DMA master) in a selected hardware accelerator to which data should be provided or a task should be offloaded. In response to determining that the DMA master is busy or operating above a threshold amount of activity, the host processor may send data over the bus to a different hardware accelerator. The data may specify a target address in the selected hardware accelerator. Upon receiving the data from the host processor, the DMA master in the receiving hardware accelerator may forward the data to the selected hardware accelerator over the accelerator link(s). In a particular embodiment, the host processor may select a receiving hardware accelerator based on a determination that the DMA master in the receiving hardware accelerator is not busy or operating below a threshold amount of activity.

[0078] For illustrative purposes, an example of a write transaction from hardware accelerator 135-1 to hardware accelerator 135-3 will be described as generally initiated by the host processor. The host processor, via the runtime and drivers, sets a remote flag for the target hardware accelerator and determines an address of 2500 (using the previous example where the desired address is located at address 500 in hardware accelerator 135-3). The host processor provides an instruction to hardware accelerator 135-1 to write to address 2500. Within hardware accelerator 135-1, a transaction with an address of 2500 is presented to interconnect 214. Because the address exceeds the upper limit of hardware accelerator 135-1, interconnect 214 sends the transaction to link circuit 150-1. Link circuit 150-1 sends the transaction to link circuit 150-2. The MM stream mapper in hardware accelerator 135-2 decrements the address by 1000, resulting in a new address of 1500. The new address is still remote because 1500 exceeds the upper address limit of hardware accelerator 135-2. Therefore, the transaction is forwarded to hardware accelerator 135-3.

[0079] The MM stream mapper in hardware accelerator 135-3 decrements the address, resulting in a new address of 500. The transaction is then provided in hardware accelerator 135-3 via interconnect 214 to a memory controller, which writes the data to memory 140-3. In the example described, the address is used by each hardware accelerator to determine whether the transaction can be serviced by the hardware accelerator, and, if the transaction can be serviced by the hardware accelerator, where the transaction should be routed internally (e.g., to a memory controller or other circuit block) or forwarded to the next hardware accelerator. In a particular embodiment, the address is different from the actual address where the data is written into memory. The write acknowledgement is sent from hardware accelerator 135-3 through hardware accelerator 135-2 to hardware accelerator 135-1, as described.

[0080] For illustrative purposes, another example of a read transaction initiated by hardware accelerator 135-1 to hardware accelerator 135-3 will be described generally as being initiated by the host processor. The host processor, via the runtime and drivers, sets a remote flag for the target hardware accelerator and determines an address of 2500 (using the previous example where the desired address is located at address 500 in hardware accelerator 135-3). The host processor provides an instruction to hardware accelerator 135-1 to read from address 2500. Within hardware accelerator 135-1, a transaction with an address of 2500 is presented to interconnect 214. Because the address exceeds the upper limit of hardware accelerator 135-1, interconnect 214 sends the transaction to link circuit 150-1. Link circuit 150-1 sends the transaction to link circuit 150-2. The MM stream mapper in hardware accelerator 135-2 decrements the address by 1000, resulting in a new address of 1500. The new address is still remote because 1500 exceeds the upper address limit of hardware accelerator 135-2. Therefore, the transaction is forwarded to hardware accelerator 135-3.

[0081] The MM stream mapper in hardware accelerator 135-3 decrements the address, resulting in a new address of 500. The transaction is then provided in hardware accelerator 135-3 via interconnect 214 to a memory controller, and the data is read from memory 140-3. In the example described, the address is used by each hardware accelerator to determine whether the transaction can be serviced by the hardware accelerator, and if the transaction can be serviced by the hardware accelerator, where the transaction should be routed internally or forwarded to the next hardware accelerator. In a particular embodiment, the address is different from the actual address where the data is read from memory. The read data is sent from hardware accelerator 135-3 through hardware accelerator 135-2 to hardware accelerator 135-1, as described.

[0082] 5 illustrates an example of a system including a hardware accelerator and one or more additional devices. In the example of FIG. 5, hardware accelerators 135-1 and 135-2 are shown connected by an accelerator link using link circuitry in each respective hardware accelerator. For illustrative purposes, hardware accelerator 135-3 is not shown. The system also includes a GPU 515, which is connected to a memory 520 and an I / O device 525.

[0083] In the example of FIG. 5, GPU 515 may write data to or read data from hardware accelerator 135-2. In this example, host processor (not shown) provides handle 505-N to GPU 515. In a particular embodiment, the handle may be implemented as a file descriptor. Handle 505-N may point to buffer object 510-N, which corresponds to hardware accelerator 135-2. By GPU 515 using handle 505-N for a read or write operation, the host processor initiates an action on the buffer object corresponding to handle 505-N, e.g., buffer object 510-N. The host processor determines whether buffer object 510-N is local or remote. The host processor may retrieve data from memory 140-2 over PCIe and provide the data to GPU 515 over PCIe because the remote flag in buffer object 510-N is not set.

[0084] In one or more other embodiments, the host processor may initiate retrieval of data from memory 140-2 by accessing a different hardware accelerator. For example, the host processor may initiate communication with hardware accelerator 135-1 via PCIe to retrieve data from memory 140-2. In that case, hardware accelerator 135-1 may communicate directly with hardware accelerator 135-2 using a link circuit to retrieve data from memory 140-2. Hardware accelerator 135-1 may then provide the data to the host processor, which provides the data to GPU 515 via PCIe.

[0085] In another example, an I / O device 525, e.g., a camera, may write data to hardware accelerator 135-1. In that case, the host processor may provide a handle 505-1 to the I / O device 525. The handle 505-1 may point to a buffer object 510-1, which corresponds to the hardware accelerator 135-1. The I / O device 525 uses the handle 505-1 for a write operation, causing the host processor to initiate an action on the buffer object corresponding to the handle 505-1, e.g., buffer object 510-1. The host processor determines whether the buffer object 510-1 is local or remote. The host processor may receive data from the I / O device 525 and provide such data to the hardware accelerator 135-1 via PCIe for writing into memory 140-1 and / or further processing, since the remote flag in the buffer object 510-1 is not set.

[0086] In one or more embodiments, the driver may set the remote flag in the buffer object only for data transfers between hardware accelerators that may use the accelerator link as described. Figure 5 shows that other types of devices may be used with the hardware accelerator, but data transfers between such other devices and the hardware accelerator occur over a bus and involve the host processor.

[0087] FIG. 6 illustrates an example architecture 600 for an IC. In one aspect, the architecture 600 may be implemented within a programmable IC. For example, the architecture 600 may be used to implement a field programmable gate array (FPGA). The architecture 600 may also represent a system-on-chip (SOC) type of IC. An SOC is an IC that includes a processor that executes program code and one or more other circuits. The other circuits may be implemented as hardwired circuits, programmable circuits, and / or combinations thereof. The circuits may operate in cooperation with each other and / or with the processor.

[0088] As shown, architecture 600 includes several different types of programmable circuits, e.g., logic, blocks. For example, architecture 600 may include a number of different programmable tiles, including multi-gigabit transceivers (MGTs) 601, configurable logic blocks (CLBs) 602, random access memory blocks (BRAMs) 603, input / output blocks (IOBs) 604, configuration and clocking logic (CONFIG / CLOCKS) 605, digital signal processing blocks (DSPs) 606, specialized I / O blocks 607 (e.g., configuration and clock ports), and other programmable logic 608, such as digital clock managers, analog-to-digital converters, system monitoring logic, etc.

[0089] In some ICs, each programmable tile includes a programmable interconnect element (INT) 611 that has standardized connections between corresponding INTs 611 in each adjacent tile. Thus, the INTs 611 collectively implement the programmable interconnect structure for the IC shown. Each INT 611 also includes connections between programmable logic elements within the same tile, as shown by the example included at the top of FIG.

[0090] For example, the CLB 602 may include a configurable logic element (CLE) 612 that may be programmed to implement user logic, and a single INT 611. The BRAM 603 may include one or more INTs 611 plus a BRAM logic element (BRL) 613. In general, the number of INTs 611 included in a tile depends on the height of the tile. As depicted, the BRAM tile has a height of five CLBs, although other numbers (e.g., four) may also be used. The DSP tile 606 may include an appropriate number of INTs 611 plus a DSP logic element (DSPL) 614. The IOB 604 may include, for example, one instance of an INT 611 plus two instances of an I / O logic element (IOL) 615. The actual I / O pads connected to the IOL 615 may not be limited to the area of ​​the IOL 615.

[0091] 6, a columnar area near the center of the die, formed, for example, from regions 605, 607, and 608, may be used for configuration, clocks, and other control logic. A horizontal area 609 extending from this column may be used to distribute clock and configuration signals across the width of the programmable IC.

[0092] Some ICs utilizing the architecture shown in Figure 6 include additional logic blocks that disrupt the regular columnar structure that makes up most of the IC. The additional logic blocks may be programmable blocks and / or dedicated circuits. For example, the processor block shown as PROC 610 spans several columns of CLBs and BRAMs.

[0093] In one aspect, PROC 610 may be implemented as a dedicated circuit fabricated as part of a die that implements the programmable circuitry of an IC, e.g., as a hardwired processor. PROC 610 may represent any of a variety of different processor types and / or systems ranging in complexity from an individual processor, e.g., a single core capable of executing program code, to an entire processor system having one or more cores, modules, coprocessors, interfaces, etc.

[0094] In another aspect, PROC 610 may be omitted from architecture 600 and replaced with one or more of the other types of programmable blocks described. Furthermore, such blocks may be utilized to form a "soft processor," in that various blocks of programmable circuitry may be used to form a processor capable of executing program code, as in the case of PROC 610.

[0095] The phrase "programmable circuitry" refers to programmable circuit elements within an IC, such as the various programmable or configurable circuit blocks or tiles described herein, as well as interconnect circuitry that selectively couples the various circuit blocks, tiles, and / or elements according to configuration data loaded into the IC. For example, the circuit blocks shown in FIG. 6 that are external to PROC 610, such as CLB 602 and BRAM 603, are considered programmable circuitry of the IC.

[0096] Generally, the functionality of a programmable circuit is not established until configuration data is loaded into the IC. A set of configuration bits may be used to program a programmable circuit of an IC, such as an FPGA. The configuration bit(s) are generally referred to as a "configuration bitstream." Generally, a programmable circuit is not operable or functional without first loading a configuration bitstream into the IC. The configuration bitstream effectively implements a particular circuit design within the programmable circuit. The circuit design specifies, for example, the functional aspects of the programmable circuit blocks and the physical connectivity between the various programmable circuit blocks.

[0097] "Hardwired" or "hardened" i.e., non-programmable circuits are fabricated as part of an IC. Unlike programmable circuits, hardwired circuits or circuit blocks are not implemented after the fabrication of an IC through the loading of a configuration bitstream. Hardwired circuits are generally considered to have dedicated circuit blocks and interconnects that are functional without first loading, for example, a configuration bitstream into an IC, e.g., PROC 610.

[0098] In some cases, a hardwired circuit may have one or more operating modes that can be set or selected according to register settings or values ​​stored in one or more memory elements within the IC. The operating modes may be set, for example, through the loading of a configuration bitstream into the IC. Despite this capability, hardwired circuits are not considered programmable circuits because they are operational and have a particular function when fabricated as part of an IC.

[0099] In the case of an SOC, the configuration bitstream may specify the circuitry to be implemented in the programmable circuitry and the program code to be executed by PROC 610 or a soft processor. In some cases, architecture 600 includes a dedicated configuration processor that loads the configuration bitstream into appropriate configuration memory and / or processor memory. The dedicated configuration processor does not execute user-specified program code. In other cases, architecture 600 may utilize PROC 610 to receive the configuration bitstream, load the configuration bitstream into the appropriate configuration memory, and / or extract the program code for execution.

[0100] FIG. 6 is intended to illustrate an exemplary architecture that may be used to implement a programmable circuit, e.g., an IC including a programmable fabric. For example, the number of logic blocks in a column, the relative width of the columns, the number and order of columns, the types of logic blocks included in the columns, the relative sizes of the logic blocks, and the interconnect / logic implementations included in the upper portion of FIG. 6 are merely exemplary. In an actual IC, for example, two or more adjacent columns of CLBs are generally included wherever CLBs appear to facilitate efficient implementation of a user circuit design. However, the number of adjacent CLB columns may vary with the overall size of the IC. Additionally, the size and / or placement of blocks such as PROC 610 within an IC are for illustrative purposes only and are not intended as limitations.

[0101] Architecture 600 may be used to implement a hardware accelerator as described herein. In certain embodiments, one or more or each of the endpoints, linking circuits, and memory controller may be implemented as hardwired circuit blocks. In certain embodiments, one or more or each of the endpoints, linking circuits, and memory controller may be implemented using programmable circuitry. In still other embodiments, one or more of the circuit blocks mentioned may be implemented as hardwired circuit blocks and others are implemented using programmable circuitry.

[0102] The embodiments described within this disclosure may be used in any of a variety of applications, such as, for example, database acceleration, processing multiple video streams, real-time network traffic monitoring, machine learning, or any other application that may involve multiple hardware accelerators.

[0103] For purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the various inventive concepts disclosed herein. However, the terminology used herein is for the purpose of describing particular aspects of the inventive configurations only and is not intended to be limiting.

[0104] As defined herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise.

[0105] The term "about" as defined herein means near, but not exactly, to the nearest exact or exact value or amount. For example, the term "about" can mean that a recited property, parameter, or value is within a given amount of the exact property, parameter, or value.

[0106] The terms "at least one," "one or more," and "and / or," as defined herein, are open-ended expressions that are both conjunctive and disjunctive in operation, unless otherwise specified. For example, each of the phrases "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, and / or C" means A only, B only, C only, A and B together, A and C together, B and C together, or A, B, and C together.

[0107] As defined herein, the term "automatically" means without user intervention. As defined herein, the term "user" means a human being.

[0108] The term "computer-readable storage medium" as defined herein means a storage medium that contains or stores program code for use by or with an instruction execution system, apparatus, or device. A "computer-readable storage medium" as defined herein is not itself a transitory propagating signal. A computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. The various forms of memory described herein are examples of computer-readable storage media. A non-exhaustive list of more specific examples of computer-readable storage media may include portable computer diskettes, hard disks, RAM, read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electronically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, and the like.

[0109] The term "if" as defined herein means "when" or "upon" or "in response to" or "responsive to," depending on the context. Thus, the phrases "when it is determined" or "when [the stated condition or event] is detected" may be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.

[0110] The term "in response to" as defined herein and similar phrases as explained above, e.g., "when," "when," or "upon," means to readily respond or react to an action or event. The response or reaction is performed automatically. Thus, when a second action is performed "in response to" a first action, there is a causal relationship between the occurrence of the first action and the occurrence of the second action. The term "in response to" indicates a causal relationship.

[0111] As defined herein, the terms "one embodiment," "an embodiment," "one or more embodiments," "particular embodiment," or similar phrases mean that a particular feature, structure, or characteristic described with respect to an embodiment is included in at least one embodiment described within the disclosure. Thus, appearances of the phrases "in one embodiment," "in an embodiment," "in one or more embodiments," "in a particular embodiment," and similar phrases throughout this disclosure may, but do not necessarily, all refer to the same embodiment. The terms "embodiment" and "configuration" are used interchangeably within this disclosure.

[0112] The term "processor" as defined herein means at least one hardware circuit. The hardware circuit may be configured to execute instructions contained in a program code. The hardware circuit may be an integrated circuit. Examples of processors include, but are not limited to, a central processing unit (CPU), an array processor, a vector processor, a digital signal processor (DSP), an FPGA, a programmable logic array (PLA), an ASIC, a programmable logic circuit, and a controller.

[0113] The term "output" as defined herein means storing in a physical memory element, e.g., a device, writing to a display or other peripheral output device, sending or transmitting to another system, exporting, etc.

[0114] The term "real-time" as defined herein means a level of processing responsiveness that a user or system perceives as immediate enough for a particular process or decision to occur, or that allows the processor to keep up with some external process.

[0115] The term "substantially" as defined herein means that the recited property, parameter, or value need not be achieved exactly, but that deviations or variations, including, for example, tolerances, measurement errors, measurement accuracy limits, and other factors known to those of ordinary skill in the art, may occur in an amount that does not interfere with the effect that the property is intended to provide.

[0116] Terms such as first, second, etc. may be used herein to describe various elements. These terms are only used to distinguish one element from another, unless otherwise stated or the context clearly dictates otherwise, and therefore these elements should not be limited by these terms.

[0117] A computer program product may include a computer-readable storage medium(s) having computer-readable program instructions thereon for causing a processor to perform aspects of the inventive arrangements described herein. Within this disclosure, the term "program code" is used interchangeably with the term "computer-readable program instructions." The computer-readable program instructions described herein may be downloaded from the computer-readable storage medium to the respective computing / processing device or to an external computer or external storage device via a network, e.g., the Internet, a LAN, a WAN, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmission, edge devices including routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0118] The computer readable program instructions for performing operations for the inventive arrangements described herein may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, or source or object code written in any combination of one or more programming languages, including object-oriented and / or procedural programming languages. The computer readable program instructions may include state setting data. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a LAN or WAN, or the connection may be to an external computer (e.g., through the Internet using an Internet Service Provider). In some cases, for example, an electronic circuit including a programmable logic circuit, an FPGA, or a PLA may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuit to implement aspects of the inventive arrangements described herein.

[0119] Certain aspects of the inventive arrangements have been described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions, e.g., program code.

[0120] These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to manufacture a machine, such that the instructions executing via the processor of the computer or other programmable data processing apparatus create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored in a computer-readable storage medium capable of directing a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored comprises an article of manufacture including instructions that implement aspects of the operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0121] The computer-readable program instructions may also be loaded into a computer, other programmable apparatus, or other device to cause a sequence of operations to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0122] The flowcharts and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions that comprises one or more executable instructions for implementing the specified operations.

[0123] In some alternative implementations, the actions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. In other examples, the blocks may generally be executed in ascending numerical order, and in still other examples, one or more blocks may be executed in a varying order, with the results stored and utilized in other blocks that follow or do not immediately follow. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or a combination of dedicated hardware and computer instructions.

[0124] The corresponding structures, materials, acts, and equivalents of all means or step and functional elements that may appear in the following claims are intended to include any structures, materials, or acts taken together for performing a function.

[0125] In one or more embodiments, a system may include a host processor connected to a communication bus, a first hardware accelerator communicatively linked to the host processor through the communication bus, and a second hardware accelerator communicatively linked to the host processor through the communication bus. The first hardware accelerator and the second hardware accelerator are directly connected through an accelerator link that is independent of the communication bus. The host processor is configured to initiate a data transfer between the first hardware accelerator and the second hardware accelerator directly through the accelerator link.

[0126] In one aspect, the host processor is configured to communicate with the first hardware accelerator and the second hardware accelerator via a communication bus.

[0127] In another aspect, the data transfer includes the first hardware accelerator accessing a memory of the second hardware accelerator through an accelerator link.

[0128] In another aspect, the host processor is configured to access the memory of the second hardware accelerator by sending data including a target address to the first hardware accelerator, the target address being translated by the host processor to correspond to the second hardware accelerator, and the first hardware accelerator initiates a transaction to access the memory of the second hardware accelerator via the accelerator link based on the target address.

[0129] In another aspect, the second hardware accelerator is configured to, in response to receiving a transaction over the accelerator link, decrement a target address for the data transfer by an upper limit of an address range for the second hardware accelerator and determine whether the decremented target address is local.

[0130] In another aspect, the host processor is configured to initiate a data transfer between the first hardware accelerator and the second hardware accelerator based on a status of a direct memory access circuit of the second hardware accelerator connected to the communication bus.

[0131] In another aspect, the host processor is configured to automatically determine a sequence of the first hardware accelerator and the second hardware accelerator in the ring topology.

[0132] In another aspect, the host processor is configured to track buffers corresponding to the first hardware accelerator and the second hardware accelerator using a remote buffer flag.

[0133] In one or more embodiments, the hardware accelerator may include an endpoint configured to communicate with a host processor over a communication bus, a memory controller connected to a memory local to the hardware accelerator, and a linking circuit connected to the endpoint and the memory controller. The linking circuit is configured to establish an accelerator link with a target hardware accelerator also connected to the communication bus. The accelerator link is a direct connection between the hardware accelerator and the target hardware accelerator, independent of the communication bus.

[0134] The link circuitry is configured to initiate a data transfer with a target hardware accelerator over the accelerator link, the data transfer occurring in response to an instruction from the host processor received by the hardware accelerator over the communications bus.

[0135] In another aspect, the link circuitry includes a first memory map to stream mapper circuit and a second memory map to stream mapper circuit, each configured to convert data streams into memory mapped transactions and memory mapped transactions into data streams.

[0136] In another aspect, each memory map to stream mapper circuit is configured to decrement the target address in a received transaction by the upper limit of the address range of the hardware accelerator.

[0137] In another aspect, the link circuit includes a first transceiver configured to send and receive stream data, and a first retransmission engine coupled to the first transceiver and to the first memory map-to-stream mapper circuit.

[0138] In another aspect, the link circuitry further includes a second transceiver configured to send and receive stream data, and a second retransmission engine coupled to the second transceiver and to the second memory map-to-stream mapper circuit.

[0139] In one or more embodiments, a method may include receiving, in a first hardware accelerator, an instruction and a target address for a data transfer sent from a host processor via a communication bus; the first hardware accelerator comparing the target address with an upper limit of an address range corresponding to the first hardware accelerator; and in response to determining that the target address exceeds the address range based on the comparing, the first hardware accelerator initiating a transaction with a second hardware accelerator to perform the data transfer using an accelerator link directly connecting the first hardware accelerator and the second hardware accelerator.

[0140] In another aspect, the accelerator link is independent of the communication bus.

[0141] In another aspect, initiating the transaction includes initiating a memory-mapped transaction and converting the memory-mapped transaction into a data stream to be sent over the accelerator link.

[0142] In another aspect, the method includes, in response to receiving the transaction at the second hardware accelerator, the second hardware accelerator subtracting an upper limit of an address range of the second hardware accelerator from the target address and determining whether a result of the subtraction is within the address range of the second hardware accelerator.

[0143] In another aspect, a second hardware accelerator receives the transactions as a data stream and converts the data stream into memory-mapped transactions.

[0144] In another aspect, a method includes determining a status of a direct memory access circuit of a second hardware accelerator and initiating a data transfer in response to the status of the direct memory access circuit of the second hardware accelerator.

[0145] The description of the inventive configurations provided herein is for illustrative purposes and is not exhaustive or limited to the disclosed forms and examples. The terminology used herein is chosen to explain the principles of the inventive configurations, practical applications, or technical improvements to the art found in the market, and / or to enable others skilled in the art to understand the inventive configurations disclosed herein. Modifications and variations may become apparent to those skilled in the art without departing from the scope and spirit of the inventive configurations described. Accordingly, reference should be made to the following claims, rather than to the above disclosure, as indicating the scope of such features and implementations.

Claims

1. a host processor connected to a communication bus; a first hardware accelerator communicatively linked to the host processor through the communications bus; a second hardware accelerator communicatively linked to the host processor through the communications bus; A system comprising: the first hardware accelerator and the second hardware accelerator are directly connected through an accelerator link that is independent of the communication bus; the host processor is configured to initiate a data transfer between the first hardware accelerator and the second hardware accelerator directly over the accelerator link; the second hardware accelerator is configured to, in response to receiving a transaction over the accelerator link, decrement a target address for the data transfer by an upper limit of an address range for the second hardware accelerator and determine whether the decremented target address is local; and, in response to detecting that the decremented target address is not local, forward the data transfer to a next hardware accelerator.

2. 2. The system of claim 1, wherein the data transfer includes the first hardware accelerator accessing a memory of the second hardware accelerator through the accelerator link.

3. 3. The system of claim 2, wherein the host processor is configured to access the memory of the second hardware accelerator by sending data including the target address to the first hardware accelerator, the target address being translated by the host processor to correspond to the second hardware accelerator, and the first hardware accelerator initiates a transaction to access the memory of the second hardware accelerator over the accelerator link based on the target address.

4. 2. The system of claim 1, wherein the host processor is configured to initiate the data transfer between the first hardware accelerator and the second hardware accelerator instead of sending data over the communication bus to the second hardware accelerator in response to determining that direct memory access circuitry of the second hardware accelerator connected to the communication bus is operating above a threshold amount of activity.

5. 2. The system of claim 1, wherein the host processor is configured to automatically determine a sequence of the first hardware accelerator and the second hardware accelerator in a ring topology.

6. 2. The system of claim 1, wherein the host processor is configured to track buffers corresponding to the first hardware accelerator and the second hardware accelerator using a remote buffer flag.

7. 1. An integrated circuit comprising: an endpoint configured to communicate with a host processor over a communications bus; a memory controller coupled to a memory local to the integrated circuit; a link circuit connected to the endpoint and the memory controller; Equipped with the link circuitry is configured to establish an accelerator link with a target hardware accelerator also connected to the communication bus, the accelerator link being a direct connection between the integrated circuit and the target hardware accelerator independent of the communication bus; in response to the link circuitry determining that a target address of a transaction originating from the host processor is not local to the integrated circuit, forwarding the transaction directly from the integrated circuit over the accelerator link to the target hardware accelerator; the target hardware accelerator is configured to decrement the target address of the transaction by a selected range of addresses and, in response to determining that the decremented target address is local to the target hardware accelerator, execute the transaction.

8. 8. The integrated circuit of claim 7, wherein the transactions are transferred over the accelerator link in response to instructions from the host processor received by the integrated circuit over the communications bus.

9. 8. The integrated circuit of claim 7, wherein the transaction is forwarded over the accelerator link based on a remote flag set by the host processor to indicate that the target address of the transaction is remote with respect to other hardware accelerators than the target hardware accelerator.

10. receiving, within the first hardware accelerator, an instruction and a target address for a data transfer sent from the host processor over a communications bus; the first hardware accelerator comparing the target address to an upper limit of an address range corresponding to the first hardware accelerator; in response to determining that the target address exceeds the address range based on the comparing, the first hardware accelerator initiates a transaction with the second hardware accelerator to perform a data transfer using an accelerator link directly connecting the first hardware accelerator and the second hardware accelerator; in response to receiving the transaction at the second hardware accelerator, the second hardware accelerator subtracting an upper limit of an address range of the second hardware accelerator from the target address, determining whether a result of the subtraction is within the address range of the second hardware accelerator, and selectively forwarding the transaction to a next hardware accelerator based on the result; A method comprising:

11. The method of claim 10 , wherein the accelerator link is independent of the communication bus.

12. determining that a direct memory access circuit of the second hardware accelerator coupled to the communications bus is operating above a threshold amount of activity; initiating the data transfer between the first hardware accelerator and the second hardware accelerator in response to the determining, instead of sending data to the second hardware accelerator over the communication bus; The method of claim 10 further comprising:

13. 11. The method of claim 10, wherein the data transfer includes the first hardware accelerator accessing a memory of the second hardware accelerator through the accelerator link.