High-Throughput Circuit Architecture for Hardware Acceleration

Abstract

The hardware acceleration device may include a switch communicatively linked to a host central processing unit (CPU), an adapter coupled to the switch via a control bus, the control bus configured to communicate addresses of descriptors from the host central CPU to the adapter, and a random access memory (RAM) coupled to the switch via a data bus. The RAM is configured to store descriptors received from the host CPU via the data bus. The hardware acceleration device may include a compute unit coupled to the adapter, the compute unit configured to perform operations specified by the descriptors. The adapter may be configured to retrieve the descriptors from the RAM via the data bus, provide arguments from the descriptors to the compute unit, and provide control signals to the compute unit to initiate operations using the arguments.

Description

[Technical Field]

[0001] Technical Field This disclosure relates to integrated circuits (ICs) and hardware acceleration, and more particularly to achieving high data throughput for hardware accelerators. [Background technology]

[0002] background Hardware acceleration refers to the use of specialized hardware, called a hardware acceleration device, to perform a function more efficiently than would be possible if the function were performed by a general-purpose central processing unit (CPU) of a computer system, such as a server. A hardware acceleration device typically provides one or more advantages not attainable by performing the function by the CPU. The advantages may include, but are not limited to, one or more of faster operation, reduced power consumption, and redundancy. The process of offloading a function from a CPU to specialized hardware is called "hardware acceleration," even though the advantages of using a hardware acceleration device may or may not include faster operation.

[0003] Modern computer systems and peripherals are capable of handling ever-increasing data throughput. For example, certain Peripheral Component Interconnect Express (PCIe) devices, such as Non-Volatile Memory Express (NVMe) storage devices and / or Ethernet (e.g., networking) devices, can operate at data rates on the order of millions of operations per second. To remain practical, hardware acceleration devices, which often operate as peripherals to the CPU and / or host computer, must also be able to do so. Summary of the Invention [Means for solving the problem]

[0004] overview In one aspect, a hardware acceleration device can include a switch communicatively linked to a host central processing unit (CPU), an adapter coupled to the switch via a control bus, the control bus configured to communicate addresses of descriptors from the host CPU to the adapter, and a random-access memory (RAM) coupled to the switch via a data bus. The RAM is configured to store descriptors received from the host CPU via the data bus. The hardware acceleration device can include a compute unit coupled to the adapter, the compute unit configured to perform operations specified by the descriptors. The adapter can be configured to retrieve the descriptors from the RAM via the data bus, provide arguments from the descriptors to the compute unit, and provide control signals to the compute unit to initiate operations using the arguments.

[0005] In another aspect, a method of hardware acceleration can include receiving a plurality of descriptors and a plurality of addresses of the plurality of descriptors via a switch of a hardware acceleration device, the switch being communicatively linked to a host CPU. The descriptors specify operations to be performed by a computation unit of the hardware acceleration device. The method can include communicating the plurality of descriptors to a RAM via a data bus for storage therein and communicating the plurality of addresses to an adapter circuit via a control bus. The method can include using the adapter to retrieve the descriptors from the RAM via the data bus, provide arguments from the descriptors to the computation unit, and provide a control signal to the computation unit to initiate an operation using the arguments.

[0006] This summary section is provided merely to introduce certain concepts and is not intended to identify any key or essential features of the claimed subject matter. Other features of the inventive arrangements will become apparent from the accompanying drawings and the detailed description that follows.

[0007] BRIEF DESCRIPTION OF THE DRAWINGS Arrangements of the present invention are illustrated by way of example in the accompanying drawings. However, the drawings should not be construed as limiting the arrangements of the present invention to only the particular implementations shown. Various aspects and advantages will become apparent upon consideration of the following detailed description and upon reference to the drawings. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an exemplary circuit architecture for hardware acceleration. [Figure 2] FIG. 2 illustrates an exemplary implementation of a switch from the architecture of FIG. 1. [Figure 3] FIG. 1 illustrates another exemplary circuit architecture for hardware acceleration. [Figure 4]FIG. 4 illustrates an exemplary method of hardware acceleration using the exemplary architecture of FIG. 1 or FIG. 3. [Figure 5] FIG. 4 illustrates another exemplary method of hardware acceleration using the exemplary architecture of FIG. 1 or FIG. 3. [Figure 6] FIG. 1 illustrates an exemplary computing environment for hardware acceleration. [Figure 7] FIG. 1 illustrates an example architecture for an integrated circuit (IC) that may be used for hardware acceleration. [Figure 8] FIG. 1 illustrates another exemplary architecture for an IC that may be used for hardware acceleration. DETAILED DESCRIPTION OF THE INVENTION

[0009] Detailed Description While this disclosure concludes with claims defining novel features, it is believed that the various features described within this disclosure will be better understood by considering the description in conjunction with the drawings. The processes, machines, articles of manufacture, and any variations thereof described herein are provided for illustrative purposes. The specific structural and functional details described within this disclosure should not be construed as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art how to employ the described features in various ways in virtually any appropriately detailed structure. Furthermore, the terms and phrases used within this disclosure are not intended to be limiting, but rather to provide an understandable description of the described features.

[0010] The present disclosure relates to integrated circuits (ICs) and hardware acceleration. More specifically, the present disclosure relates to achieving high data throughput for hardware accelerators. Typically, hardware accelerators process data at rates on the order of tens of thousands of operations per second (OPS) to approximately 100,000 OPS. To achieve data rates in this range, input batching is often used. Data rates in this range cannot keep up with more modern peripherals capable of operating at data rates on the order of 1 million OPS.

[0011] The inventive arrangements described within this disclosure provide circuit architectures capable of providing high-speed data throughput for hardware accelerators. The circuit architectures can provide data throughput that matches the line rate of high-speed input / output (I / O) interfaces, such as non-volatile memory express (NVMe), often found on storage devices. For example, data throughput rates of approximately 1 million OPS can be achieved using the exemplary circuit architectures described herein. It should be understood that the inventive arrangements described within this disclosure can be used in any of a variety of hardware acceleration devices and / or applications beyond those related to data storage.

[0012] In an exemplary implementation, the circuit architecture described herein may be used with a hardware accelerator and the accelerator engines, e.g., compute units, included therein. The hardware accelerator, with one or more accelerator engines implemented therein, may be communicatively linked to a host central processing unit (CPU) via a communications bus, such as a Peripheral Component Interconnect Express (PCIe) bus. The host CPU may execute operational software such that a host application executed by the host CPU can provide fine-grained control over the operation of the accelerator engines included in the hardware accelerator. Furthermore, the circuit architecture provides a hardware interface to the accelerator engines that allows the host CPU to keep the accelerator engines busy while executing the operational software, thereby achieving sustained data throughput while also reducing transaction overhead between the host CPU and the accelerator engines.

[0013] Further aspects of the inventive arrangements are described in more detail below with reference to the figures. For simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Furthermore, where considered appropriate, reference numerals have been repeated among the figures to indicate corresponding, similar, or like features.

[0014] 1 illustrates an exemplary circuit architecture 100 for hardware acceleration. Circuit architecture 100 can increase data throughput that can be processed by a computing unit 110. In one aspect, circuit architecture 100 increases data throughput, at least in part, by reducing transaction overhead for a host CPU to offload operations to a computing unit for purposes of hardware acceleration.

[0015] In the example of FIG. 1, circuit architecture 100 includes a host CPU 102, a switch 104, a random access memory (RAM) 106, an adapter 108, and a computing unit 110. Host CPU 102 is coupled to switch 104 by a communication channel 112. In an exemplary implementation, host CPU 102 is included in a host computer (not shown), while switch 104, adapter 108, computing unit 110, and, optionally, RAM 106 are located within a hardware accelerator. An exemplary host computer 602 is shown in FIG. 6. The hardware accelerator may be implemented as an IC. In that case, communication channel 112 is implemented as a communication bus. An example of a communication bus is a Peripheral Component Interconnect Express (PCIe) bus. In such an implementation, switch 104 may be implemented as a PCIe endpoint.

[0016] In another exemplary implementation, the host CPU 102 is incorporated into a hardware accelerator. For example, the host CPU 102 is incorporated into the same IC as the switch 104, the adapter 108, the compute unit 110, and optionally the RAM 106. In that case, the host CPU 102 may be connected to the switch 104 via one or more on-chip wires or interconnects. In such an implementation, the switch 104 may be implemented as an on-chip interconnect. For example, the switch 104 may be implemented as a crossbar switch. Exemplary IC architectures including a processor / CPU are shown in FIGS. 7 and 8. Whether implemented within a host computer or incorporated into an IC with the other described components, the host CPU 102 may operate as described below.

[0017] The switch 104 is coupled to the RAM 106, the adapter 108, and optionally the computing unit 110 via a data bus 114. In one exemplary implementation, the data bus 114 may be implemented as a high-speed data bus. The data bus 114 may also be implemented as a memory-mapped bus. An example of a memory-mapped bus is the Advanced Microcontroller Bus Architecture (AMBA). Advanced Extensible Interface ( Advanced The memory-mapped AXI bus is an extensible interface (AXI) bus. The memory-mapped AXI bus allows the switch 104 to establish on-chip connections and communicate with one or more slave circuit blocks coupled to the bus. Within this disclosure, AXI is provided as an example bus and is not intended as a limitation. Other types of buses may be used instead of the AXI bus.

[0018] The switch 104 is also coupled to the adapter 108 via a control bus 116. The control bus 116 may be a slower bus than the data bus 114. Furthermore, the control bus 116 may be implemented as a point-to-point interface, while the data bus 114 is not. In an exemplary implementation, the control bus 116 may be implemented as an AXI-Lite (e.g., AXI4-Lite) compliant bus. In the exemplary implementation described herein, the data bus 114 may operate at a higher clock rate and provide a higher data throughput or bandwidth than the control bus 116. Within this disclosure, the data bus 114 and the control bus 116 may be considered examples of on-chip interconnects.

[0019] Adapter 108 may include counter 118, controller 120, and first-in-first-out (FIFO) memory 122. In an exemplary implementation, controller 120 may be implemented as a state machine circuit that can coordinate the operation of counter 118 and FIFO memory 122. Adapter 108 is coupled to computation unit 110 to provide arguments via communication channel 124 and control signals via control channel 126.

[0020] In one aspect, RAM 106 may be implemented as one or more block RAMs (BRAMs) or other RAM circuit blocks contained within the programmable circuitry or programmable logic of the same IC that includes switch 104, adapter 108, and compute unit 110. For example, if the hardware accelerator is a programmable IC, such as a field programmable gate array (FGPA), or other IC that includes some programmable circuitry and / or logic, RAM 106 may be implemented using the programmable circuitry and / or logic of the IC.

[0021] In another embodiment, RAM 106 may be implemented as off-chip memory. For example, RAM 106 may be implemented as double data rate (DDR) RAM located on the same circuit board or card as the hardware accelerator. In that case, the hardware accelerator may include a memory controller (not shown) coupled to switch 104 and adapter 108 via data bus 114. The memory controller may be coupled to and perform read and write operations to the off-chip memory.

[0022] In another embodiment, the RAM 106 may be implemented as high bandwidth memory (HBM). In an example, the hardware accelerator may be a multi-die IC in which the HBM is implemented on a die different from the die on which the switch 104, adapter 108, and controller 120 of the hardware accelerator are implemented. The HBM may still be implemented in the same package. In another example, the HBM may be off-chip memory located on the same circuit board or card as the IC.

[0023] 1, host CPU 102 is implemented within a host computer system that is separate and distinct from the hardware accelerator, and RAM 106 is considered to be local to the hardware accelerator, i.e., RAM 106 is distinct from any RAM or memory that may be included in the host computer system that may include host CPU 102.

[0024] Using architecture 100, host CPU 102 can offload operations to compute units 110 to achieve higher data throughput than would be possible without architecture 100. For example, host CPU 102 can combine arguments to be provided to compute units 110 into descriptors. The arguments in the descriptors are data, and compute units 110 will operate on the data to perform the offloaded operations. In accordance with the inventive arrangements described herein, the descriptors provided by host CPU 102 are stored in RAM 106. The descriptors are provided to RAM 106 using data bus 114.

[0025] Because the descriptors are provided in memory, the host CPU 102 can transfer the descriptors using memory write transactions as opposed to register write operations. For example, in conventional hardware acceleration architectures in which a CPU provides arguments to a compute unit, the CPU does so by writing the arguments (e.g., data) to the compute unit's register-based interface. The CPU writes to the compute unit's register interface by executing the CPU's native register write instructions. Typically, each register write instruction conveys a small or limited amount of data, e.g., approximately four bytes. Examples of register write operations include, but are not limited to, "readb()," "readl()," "readq()," or other equivalent write operations used for memory-mapped register access. Operations such as the listed examples convey data in amounts of, for example, one byte, four bytes, or eight bytes. Therefore, transferring a large argument to a compute unit requires greater transaction overhead in that more register write instructions must be executed, which requires more time.

[0026] In comparison, host CPU 102 native instructions that support memory writes are often capable of implementing burst data transfers. In contrast to register write operations used to access device control registers, a “memcpy()” operation is an example of a memory write operation that can transfer larger amounts of data. As an illustrative and non-limiting example, one register write instruction executed by host CPU 102 may convey four bytes of data, while a memory write instruction initiated by host CPU 102 may burst 32 bytes of data. Transferring descriptors from host CPU 102 to RAM 106 using memory transfers as described, instead of writing to registers in compute unit 110, may reduce the transaction overhead and time required to send data to the hardware acceleration device and compute unit 110, thereby increasing data throughput.

[0027] Host CPU 102 can write the descriptor directly to RAM 106 via data bus 114. Additionally, host CPU 102 can provide adapter 108 with the address at which the descriptor was written in RAM 106. In providing the address, host CPU 102 performs a register write of the address value. The address of the descriptor as provided by host CPU 102 is provided to adapter 108 via control bus 116. The address may be stored in FIFO 122 under the control of controller 120.

[0028] In some cases, the host CPU 102 may implement a heuristic in which a register write operation is internally queued until a register read operation is executed. The example architecture of Figure 1 defeats such a heuristic by utilizing a memory write operation to transfer arguments instead of a register write operation. This prevents the host CPU 102 from queuing or otherwise delaying the provision of data to the hardware accelerator, because the memory write can be executed without being affected by any register read / write heuristic.

[0029] FIFO 122 can store a list of addresses of descriptors stored in RAM 106. Adapter 108 can control the operation of computing unit 110 to continuously provide arguments to computing unit 110 so that computing unit 110 can continue to perform operations without delay. For example, controller 120 can extract an address from FIFO 122 and retrieve a descriptor stored at the address from RAM 106. Controller 120 can provide the argument from the retrieved descriptor to computing unit 110 via communication channel 124. Once the argument is provided to computing unit 110, controller 120 can initiate computing unit 110 via control bus 126 to begin an operation based on the argument.

[0030] In one aspect, the adapter 108 serves as a high-speed interface to the compute unit 110. The host CPU 102 does not need to spend time writing to registers of the compute unit 110 to provide arguments thereto. Instead, the host CPU 102 can continuously provide descriptors to the RAM 106 using the data bus 114 by executing more efficient memory write instructions. The number of register write operations performed by the host CPU 102 is reduced in that the host CPU 102 only needs to write the address of each descriptor stored in the RAM 106 to the adapter 108. The adapter 108 can retrieve the descriptor using the address via the data bus 114. The adapter 108 then performs the register write operation to provide the compute unit 110 with the arguments needed to perform the operation offloaded from the host CPU 102.

[0031] In the example of FIG. 1 , the host CPU 102 does not need to know the specific register interface of the compute unit 110. The host CPU 102 may write a descriptor containing arguments to memory. The adapter 108 is configured to parse the descriptor to extract the arguments contained therein and provide the arguments to the appropriate registers of the register interface of the compute unit 110. This means that while the adapter 108 manages the operation of the compute unit 110, the host CPU 102 can continue writing descriptors to RAM 106 to queue further offloaded operations for the compute unit 110 to execute. This allows the compute unit 110 to continue processing the next set of arguments after completing an operation, rather than waiting for the host CPU 102 to send more arguments. The compute unit 110 may continue to operate on the arguments while the descriptors await processing in RAM 106, based on addresses taken from the FIFO 122.

[0032] Counter 118 can track the number of completed jobs, e.g., descriptors, sent from host CPU 102 to the hardware accelerator. A completed job or descriptor is one that compute unit 110 has finished processing. In one aspect, controller 120 can poll a particular register in compute unit 110 to determine when compute unit 110 completes execution of a descriptor. In another aspect, compute unit 110 can generate an interrupt to adapter 108, e.g., controller 120, indicating that compute unit 110 has completed execution of a descriptor. In either case, controller 120 increments counter 118 in response to determining that compute unit 110 has completed execution of a descriptor. Furthermore, controller 120 can then transfer another descriptor to compute unit 110 via communication channel 124 and initiate compute unit 110 via control channel 126.

[0033] The host CPU 102 may maintain an internal count of jobs offloaded to the hardware accelerator. The host CPU 102 may further track the order in which jobs were sent to the hardware accelerator. In one aspect, the host CPU 102 may read the contents of the counter 118 via the control bus 116. By comparing the internal count maintained by the host CPU 102 with the count read from the counter 118, the host CPU 102 may determine the number of outstanding jobs for the hardware accelerator. That is, the host CPU 102 may determine the difference between the internal count and the count read from the counter 118. The difference indicates the number of jobs offloaded to the hardware accelerator that have not yet completed. Furthermore, because the descriptors are processed by the hardware accelerator in FIFO order based on the storage of addresses in the FIFO 122, and the host CPU 102 tracks the order in which jobs are offloaded to the hardware accelerator, the host CPU 102 may determine which of the jobs offloaded to the hardware accelerator have completed and which have not. Thus, at any given time, the host CPU 102 can determine how many of the jobs (e.g., descriptors and corresponding addresses) that have been offloaded to the hardware accelerator remain to be processed (e.g., not completed), how many have been processed, and the particular jobs that remain to be processed.

[0034] The FIFO 122 may include one or more registers that indicate the amount of data stored therein and / or the amount of available space therein. For example, the FIFO 122 may include a free register that counts the free space in the FIFO. The free register may be adjusted (e.g., incremented and / or decremented) as data is read from and / or written to the FIFO 122 to indicate the amount of available space in the FIFO 122. In one aspect, such a register may be read by the controller 120 and / or the host CPU 102 to determine the amount of available space in the FIFO 122.

[0035] FIG. 2 illustrates an exemplary implementation of the switch 104 of FIG. 1. In the example of FIG. 2, the switch 104 is implemented as a PCIe endpoint that can communicate with the host CPU 102 over a PCIe communication link. The switch 104 may include a DMA circuit 202 that can be bypassed. The switch 104 further includes a memory base address register (BAR) 204 and a control BAR 206. The memory BAR 204 is coupled to a memory-mapped interface 208 and the data bus 114. The control BAR 206 is coupled to a control interface 210 for communicating over the control bus 116. The use of two different BARs, one for the data bus 114 and the other for the control bus 116, allows the host CPU 102 to send data, such as descriptors, over the data bus 114 separately from the addresses of the descriptors that are provided to the FIFO 122 in the adapter 108 over the control bus 116. For example, a memory write instruction directs data to memory BAR 204, while a register write operation directs data to control BAR 206.

[0036] In an exemplary implementation in which the host CPU 102 is embedded in a hardware accelerator, the switch 104 may be implemented as an on-chip interconnect. As an illustrative and non-limiting example, the switch 104 may be implemented as an AXI bus. In that case, rather than using BARs as shown in FIG. 2, the BARs of the switch 104 may be replaced with apertures (e.g., a subset of the overall addressing range of the slaves / endpoints) that use general-purpose ports that connect to the data bus 114 and the control bus 116, respectively. In this example, the slaves may be the adapters 108 and / or RAM 106.

[0037] Figure 3 shows another example of an architecture 100 for hardware acceleration. In the example of Figure 3, the hardware accelerator includes an embedded processor 302 and a network interface 304. The RAM 106 may be implemented as any of the different types of memory described in connection with Figure 1.

[0038] In one embodiment, the embedded processor 302 may be implemented as a hardwired processor. In another embodiment, the processor 302 may be implemented as a soft processor. A soft processor is a processor implemented using programmable circuitry or programmable logic in an IC. In either case, the embedded processor 302 is located in the same hardware accelerator or IC as the switch 104, adapter 108, compute unit 110, optionally RAM 106 depending on the type of memory used, and optionally the host CPU 102 depending on the type of IC used to implement the architecture 100.

[0039] The network interface 304 may be an input / output (I / O) interface, such as a network interface capable of communicating with other off-chip devices and / or nodes over a network 306, such as an Ethernet network. In the example of FIG. 1, the architecture 100 may be a network-interface-controller (NIC) implementation.

[0040] In the example of FIG. 3 , data, e.g., descriptors, may be received from nodes and / or devices on the network 306 via the network interface 304. In that case, the operations performed by the compute unit 110 may originate from the network 306 and / or the host CPU 102. The embedded processor 302 may receive the descriptors received via the network 306 from the network interface 304 and write the descriptors to the RAM 106 via the data bus 114. Additionally, the embedded processor 302 may write the address of the descriptor as stored in the RAM 106 to the FIFO 122 via the control bus 308. The control bus 308 may be implemented as a point-to-point communication link between the embedded processor 302 and the adapter 108, separate and independent from the control bus 116. For example, the control bus 308 may also be implemented as an AXI-Lite compliant bus. The adapter 108 and the compute unit 110 may operate as described such that the compute unit 110 can perform hardware-accelerated operations that originate from the host CPU 102 and / or the network 306. The compute unit 110 may perform operations that originate from both sources in the order in which the addresses of the descriptors are stored in the FIFO 122.

[0041] 4 illustrates an example method 400 of hardware acceleration using the example architecture of FIG. 1 or FIG. 3. In block 402, a host CPU generates a descriptor including one or more arguments for a compute unit. The host CPU may be included in a host computer separate from a hardware accelerator including the compute unit. For example, the hardware accelerator may be located on a hardware acceleration card or board that is peripheral to the host computer, with the host CPU and hardware accelerator communicating via a PCIe communication link. In another example, the host CPU is a processor embedded in the same hardware accelerator as the compute unit.

[0042] In block 404, the host CPU initiates a memory write operation to write the descriptor to RAM local to the hardware accelerator. For example, the host CPU executes one or more native memory write instructions, where each such memory write instruction may perform a burst data transfer. The descriptor is written to a specified address in RAM local to the hardware accelerator. In an example where the host CPU is located within a host computer, the host CPU directs the memory write operation to a memory BAR of the hardware accelerator. Whether the host CPU is located within a host computer or incorporated within a hardware accelerator, the host CPU directs the descriptor to a data bus so that the descriptor is communicated to the RAM via the data bus and written to the RAM.

[0043] In block 406, the host CPU initiates a register write operation to the adapter to store the address of the descriptor in the FIFO. For example, the host CPU executes one or more native register write instructions. In examples where the host CPU is located within a host computer, the host CPU directs the register write operation to a control BAR of the hardware accelerator. Whether the host CPU is located within a host computer or embedded within a hardware accelerator, the host CPU directs the address to the control bus so that it is communicated via the control bus to the adapter and written into the FIFO.

[0044] Blocks 408-412 illustrate example operations that may be performed in an implementation in which the hardware accelerator includes a network interface and an embedded processor. In an implementation in which the hardware accelerator includes a network interface and an embedded processor as described in connection with FIG. 3, the operations described in blocks 408-412 may be performed instead of, alternately with, or simultaneously with blocks 402-406.

[0045] In block 408, the embedded processor receives a descriptor containing one or more arguments for the compute unit from the network via the network interface. In block 410, the embedded processor initiates a memory write operation to write the descriptor to RAM. For example, the embedded processor executes one or more native memory write instructions, where each such memory write instruction may perform a burst data transfer. The embedded processor can write the descriptor to RAM via a data bus. The descriptor is written to a specified address in RAM local to the hardware accelerator. In block 412, the embedded processor initiates a register write operation to the adapter to store the address of the descriptor in a FIFO therein. For example, the embedded processor executes one or more native register write instructions to write the address of the descriptor to the adapter's FIFO. In one aspect, the embedded processor can write the address via the bus connection illustrated in FIG. 3. In another aspect, the embedded processor may have a separate control interface (e.g., a further point-to-point connection) through which the control interface is established with the adapter and through which the embedded processor communicates the address of the descriptor.

[0046] In block 414, the FIFO may adjust a control register to indicate the amount of space available therein. In block 416, a controller in the adapter reads an address from the FIFO and uses the address to retrieve a descriptor from RAM. In block 418, the controller parses the descriptor to extract the arguments contained therein. The controller provides the arguments from the descriptor to the compute unit. For example, the controller writes the arguments to a register interface of the compute unit via communication channel 124.

[0047] In block 420, the controller begins execution of the compute unit. For example, the controller provides a start signal to the compute unit via control channel 126. In block 422, the controller detects completion of execution of the compute unit. As described above, the controller may poll a register of the compute unit to detect when the compute unit completes the operation, or the compute unit may generate an interrupt to the controller indicating that the compute unit has completed the operation. In block 424, the controller increments a count stored in a counter within the adapter in response to determining that the compute unit has completed the operation.

[0048] 4 are provided for purposes of illustration and not limitation. It should be understood that the described method may continue to iterate to process additional descriptors. Portions of method 400 may also operate simultaneously and in parallel with others. For example, blocks 402-406 may be performed simultaneously with blocks 408-412 and blocks 414-424.

[0049] FIG. 5 illustrates another example method 500 of hardware acceleration using the example architecture of FIG. 1 or FIG.

[0050] In block 502, a hardware accelerator located on a hardware acceleration device, such as a switch in an IC, receives a plurality of descriptors and a plurality of addresses of the plurality of descriptors. The switch is communicatively linked to a host CPU, which transmits the plurality of descriptors and the plurality of addresses. The descriptors specify operations to be performed by computation units in the IC.

[0051] In block 504, the plurality of descriptors are communicated from the switch to RAM for storage therein via a data bus. In block 506, the plurality of addresses are communicated to an adapter circuit via a control bus. For example, the adapter circuit may include a FIFO configured to store the addresses. The data bus is configured to operate at a higher frequency or data rate than the control bus.

[0052] In block 508, the adapter may retrieve the descriptor from the RAM via the data bus, provide arguments from the descriptor to the compute unit, and provide control signals to the compute unit to initiate an operation using the arguments.

[0053] These and other implementations can each optionally include one or more of the following features, either alone or in combination: Some example implementations include all of the following features in combination.

[0054] In one embodiment, the switch is a PCIe switch. The PCIe switch includes a first BAR, e.g., a control BAR, coupled to a control bus and a second BAR, e.g., a data BAR, coupled to a data bus. BAR , for example, a memory BAR.

[0055] In another embodiment, the hardware accelerator of the hardware acceleration device is a programmable IC that includes a switch, an adapter, and a computation unit. The hardware accelerator may also include a RAM used to store the descriptors. In an exemplary implementation, the RAM is implemented using programmable circuitry of the programmable IC.

[0056] In another aspect, the hardware accelerator includes an embedded processor and a network interface. The embedded processor may be configured to store in RAM an additional descriptor received from a network via the network interface. The embedded processor may also be configured to write an address of the additional descriptor into a FIFO of the adapter.

[0057] 6 illustrates an exemplary computing environment 600 for hardware acceleration. The computing environment 600 includes a host computer 602 coupled to a hardware acceleration device 604.

[0058] Components of the host computer 602 may include, but are not limited to, a host CPU 606, memory 608, and a bus 610 that couples various system components, including the memory 608, to the host CPU 606. The host CPU 606 may be implemented as any of a variety of processors capable of executing program code. Exemplary processor types include, but are not limited to, processors having x86-type architectures (e.g., Intel Architecture 32-bit, Intel Architecture 64-bit), Power Architecture, ARM processors, etc. Furthermore, while the host computer 602 is shown as including a single host CPU 606, in other examples, multiple host CPUs may be included.

[0059] Bus 610 represents any one or more of several types of communication buses, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, a Peripheral Component Interconnect (PCI) bus, and a PCI Express (PCIe) bus.

[0060] The host computer 602 typically includes a variety of computer-readable media. Such media may be any available media that can be accessed by the host computer 602 and may include any combination of volatile, nonvolatile, removable, and / or non-removable media.

[0061] The memory 608 may include computer-readable media in the form of volatile memory, such as RAM 612 and / or cache memory 614. The host computer 602 may also include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example, the storage system 616 may be provided for reading from and writing to non-removable, non-volatile magnetic and / or solid-state media (e.g., a “hard drive”) contained therein. Although not shown, one or more disk drives may be provided for reading from and writing to removable, non-volatile magnetic and / or optical and / or solid-state media. In such cases, each may be connected to the bus 610 by one or more data media interfaces. The memory 608 is an example of at least one computer program product having a set (e.g., one or more) of program modules (e.g., program code) configured to perform the functions and / or operations described within this disclosure.

[0062] For example, the programs / utilities 618 include a set (e.g., one or more) of program modules 620. The program modules 620 stored in the memory 608 may include, but are not limited to, an operating system, one or more application programs (e.g., user applications), other program modules, and / or program data. The program modules 620 generally perform the functions and / or methodologies described herein, at least with respect to operations performed by the host computer 602. For example, the program modules 620 may include user applications and / or a software stack. The software stack may implement a runtime environment capable of executing the host computer 602 operations described herein with respect to the hardware acceleration device 604. In one aspect, the program modules 620 include a driver or daemon capable of communicating with the IC 632.

[0063] The programs / utilities 618 are executable by the host CPU 606. The programs / utilities 618 and any data items are functional data structures that impart functionality when used by the host CPU 606, where the host CPU 606 uses, generates, and / or operates on any such data items. As defined within this disclosure, a "data structure" is a physical implementation of the organization of data of a data model in physical memory. Thus, a data structure is formed of specific electrical or magnetic structural elements in memory. The data structure imposes a physical organization on the data stored in memory when used by an application program executed using a processor.

[0064] The host computer 602 may include one or more input / output (I / O) interfaces 628 communicatively linked to the bus 610. The I / O interfaces 628 enable the host computer 602 to communicate with external devices, couple to external devices that allow a user to interact with the host computer 602, couple to external devices that allow the host computer 602 to communicate with other computing devices, etc. For example, the host computer 602 may be communicatively linked to a display 630 and a hardware acceleration device 604 via the I / O interface 628. The host computer 602 may also be coupled to other external devices, such as a keyboard (not shown), via the I / O interface 628. Examples of the I / O interface 628 may include, but are not limited to, a network card, a modem, a network adapter, a hardware controller, etc.

[0065] In an exemplary implementation, I / O interface 628, through which host computer 602 communicates with hardware acceleration device 604, is a PCIe adapter. Hardware acceleration device 604 may be implemented as a circuit board, e.g., a card, that couples to host computer 602. Hardware acceleration device 604 may be inserted, for example, into an available card slot, e.g., an available bus and / or PCIe slot, of host computer 602.

[0066] The hardware acceleration device 604 includes an IC (e.g., a hardware accelerator) 632. In one embodiment, the IC 632 is a programmable IC. The hardware acceleration device 604 also includes volatile memory 634 coupled to the IC 632 and non-volatile memory 636 also coupled to the IC 632. The volatile memory 634 may be implemented as RAM external to the IC 632. The volatile memory 634 is considered "local memory" of the programmable IC 632, whereas the memory 608 within the host computer 602 is not considered local to the programmable IC 632. The non-volatile memory 636 may be implemented as flash memory. The non-volatile memory 636 may also be external to the IC 632, and the non-volatile memory 636 may be considered local to the IC 632.

[0067] In an exemplary implementation, the RAM 106 of FIG. 1 or FIG. 3 may be implemented as volatile memory 634. In another exemplary implementation, the RAM 106 of FIG. 1 or FIG. 3 may be implemented as a hardwired circuit block or in a programmable circuit, such as an IC 6 32.

[0068] In an exemplary implementation, IC 632 includes programmable circuitry (e.g., programmable logic). For example, IC 632 may be implemented as an FPGA or another type of IC that includes at least some amount of programmable circuitry. Although not explicitly shown, it should be understood that IC 632 may include one or more hardwired circuit blocks that can be used in cooperation with the programmable circuitry.

[0069] In one or more exemplary implementations, the IC 632 is connected to a network 638. The network 638 may be implemented as an Ethernet-type network. The network 638 may operate at any of a variety of different speeds. For example, the network 638 may be a 10G, 25G, 50G, 100G, 200G, 400G, or other speed network. In particular implementations, the network 638 may be, include, or be coupled to a 5G network. Accordingly, the IC 632 may include an Ethernet interface, such as the network interface 304 of FIG. 3 (not shown in the example of FIG. 6 ), that connects to the network 638, e.g., used to communicatively link the IC 632 to the network 638. For example, the IC 632 may be connected to an Ethernet switch or one or more other network-connected devices via the network 638. For purposes of illustration, the term “network” refers to the network 638, e.g., an Ethernet network, herein.

[0070] 6 is not intended to suggest any limitation as to the scope of use or functionality of the examples described herein. Host computer 602 is an example of computer hardware (e.g., a system) capable of performing various operations described within this disclosure attributed to a host computer and / or host CPU.

[0071] The host computer 602 is merely one exemplary implementation of a computer that may be used with the hardware acceleration device 604. The host computer 602 is shown in the form of a computing device, e.g., a computer or server. In one aspect, the host computer 602 may be implemented within a data center. For example, the host computer 602 may be implemented as a standalone device, as a bare metal server, in a cluster, or in a distributed cloud computing environment. In a distributed cloud computing environment, tasks are performed by remote processing devices linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0072] As used herein, the term "cloud computing" refers to a computing model that facilitates convenient, on-demand network access to a shared pool of configurable computing resources, such as networks, servers, storage, applications, ICs (e.g., programmable ICs), and / or services. These computing resources can be rapidly provisioned and released with minimal administrative effort or service provider interaction. Cloud computing fosters availability and can be characterized by on-demand self-service, pervasive network access, resource pooling, rapid flexibility, and measured service.

[0073] Some computing environments, such as cloud computing environments, edge computing environments, and / or data centers, commonly support the FPGA-as-a-Service (FaaS) model, in which user and / or network functions are hardware-accelerated as circuit designs implemented in programmable integrated circuits (ICs) operating under the control of a host computer. Other examples of cloud computing models are described by the National Institute of Standards and Technology (NIST), and more specifically, NIST's Information Technology Laboratory.

[0074] The host computer 602 is operational with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of computing systems, environments, and / or configurations that may be suitable for use with the host computer 602 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0075]

[0033] Figure 7 shows an example architecture 700 for an IC that may be used for hardware acceleration. Architecture 700 may be used to implement a hardware accelerator such as those described in connection with Figures 1 and / or 3. For example, in the example of Figure 1, architecture 100 may be used to implement switch 104, adapter 108, compute unit 110, and optionally RAM 106. In the example of Figure 3, architecture 100 may be used to implement switch 104, adapter 108, compute unit 110, optionally RAM 106, embedded processor 302, and network interface 304.

[0076] In one embodiment, architecture 700 may be used to implement a hardware accelerator that includes a host CPU embedded therein. In another embodiment, architecture 700 may be used to implement a hardware accelerator such as that described in connection with Figure 6, where the host CPU is included in a host computer.

[0077] Architecture 700 may be implemented within a programmable integrated circuit. For example, architecture 700 may be used to implement an FPGA. Architecture 700 may also represent a system-on-chip (SoC) type IC. An SoC is an IC that includes a processor that executes program code and one or more other circuits. The other circuits may be implemented as hardwired circuitry, programmable circuitry and / or logic, and / or a combination thereof. The circuits may operate in cooperation with each other and / or with the processor.

[0078] As shown, architecture 700 includes several different types of programmable circuitry, e.g., logic, blocks. For example, architecture 700 may include a number of different programmable tiles, including multi-gigabit transceivers (MGTs) 701, configurable logic blocks (CLBs) 702, random access memory blocks (BRAMs) 703, input / output blocks (IOBs) 704, configuration and clocking logic (CONFIG / CLOCKS) 705, digital signal processing blocks (DSPs) 706, specialized I / O blocks 707 (e.g., configuration and clock ports), and other programmable logic 708, such as digital clock managers, analog-to-digital converters, system monitoring logic, etc.

[0079] In some ICs, each programmable tile includes a programmable interconnect element (INT) 711 with standardized connections to and from corresponding INTs 711 in each adjacent tile. The INTs 711, taken together, implement the programmable interconnect structure for the illustrated IC. Each INT 711 also includes connections to and from programmable logic elements within the same tile, as shown by the example included on the right side of FIG. 7.

[0080] For example, the CLB 702 may include a configurable logic element (CLE) 712 that can be programmed to implement a single INT 711 in addition to user logic. The BRAM 703 may include a BRAM logic element (BRL) 713 in addition to one or more INTs 711. Typically, the number of INTs 711 included in a tile depends on the tile's height. As depicted, a BRAM tile has a height of five CLBs, although other numbers (e.g., four) may also be used. The DSP tile 706 may include a DSP logic element (DSPL) 714 in addition to an appropriate number of INTs 711. The IOB 704 may include, for example, one instance of an INT 711 in addition to two instances of an I / O logic element (IOL) 715. The actual I / O pads connected to the IOL 715 need not remain within the area of ​​the IOL 715.

[0081] 7, a horizontal area near the center of the die, formed by regions 705, 707, and 708, for example, may be used for configuration, clocks, and other control logic. Extending from this horizontal area, a vertical area 709 may be used to distribute clock and configuration signals across the width of the programmable IC.

[0082] Some ICs utilizing the architecture shown in Figure 7 include additional logic blocks that disrupt the regular columnar structure that makes up large portions of the IC. The additional logic blocks may be programmable blocks and / or dedicated circuits. For example, the processor block depicted as PROC 710 spans several columns of CLBs and BRAMs.

[0083] In one aspect, PROC 710 may be implemented as a dedicated circuit, e.g., a hardwired processor, fabricated as part of a die that implements the programmable circuitry of an IC. PROC 710 may represent any of a variety of different processor types and / or systems, ranging in complexity from an individual processor, e.g., a single core capable of executing program code, to an entire processor system having one or more cores, modules, coprocessors, interfaces, etc.

[0084] In another embodiment, PROC 710 may be omitted from architecture 700 and replaced with one or more of the various other described programmable blocks. Furthermore, such blocks may be utilized to form a "soft processor," in that various blocks of programmable circuitry may be used to form a processor capable of executing program code, as in the case of PROC 710.

[0085] In an exemplary implementation, PROC 710 may be used to implement a host CPU. Thus, if the IC further includes an embedded processor, the embedded processor may be implemented as a soft processor. In another example where the host CPU is included in a host computer, PROC 710 may implement the embedded processor. If architecture 700 does not include a hardwired processor, the embedded processor, if included, may be implemented as a soft processor.

[0086] The phrase "programmable circuitry" refers to programmable circuit elements within an IC, such as the various programmable or configurable circuit blocks or tiles described herein, as well as interconnect circuitry that selectively couples the various circuit blocks, tiles, and / or elements according to configuration data loaded into the IC. For example, circuit blocks shown in FIG. 7 that are external to PROC 710, such as CLB 702 and BRAM 703, are considered programmable circuitry of the IC.

[0087] Generally, the functionality of a programmable circuit is not established until configuration data is loaded into the IC. A set of configuration bits can be used to program the programmable circuit of an IC, such as an FPGA. The configuration bits are typically referred to as a "configuration bitstream." Generally, a programmable circuit is not operational or functional without first loading a configuration bitstream into the IC. The configuration bitstream effectively implements a particular circuit design within the programmable circuit. The circuit design specifies, for example, the functional aspects of the programmable circuit blocks and the physical connectivity between the various programmable circuit blocks.

[0088] In one or more exemplary implementations, circuits such as switch 104, RAM 106, adapter 108, computing unit 110, and / or network interface 304 may be implemented using programmable circuitry of an IC. For example, RAM 106 may be implemented using one or more of BRAM 703 or other RAM circuit blocks included in the programmable circuitry. In another exemplary implementation, one or more of the described circuits, such as switch 104 and / or network interface 304, may be implemented as hardwired circuitry.

[0089] In another exemplary implementation, the architecture 700 may be used to implement a die of a multi-die IC, where another die in the same package implements the HBM, which may operate as RAM 106.

[0090] "Hardwired" or "hardened" circuits, i.e., non-programmable circuits, are fabricated as part of an IC. Unlike programmable circuits, hardwired circuits or circuit blocks are not implemented after the IC is fabricated by loading a configuration bitstream. Hardwired circuits are generally considered to have dedicated circuit blocks and interconnections that are functional without first loading a configuration bitstream into an IC, e.g., PROC 710.

[0091] In some cases, a hardwired circuit may have one or more operating modes that can be set or selected according to register settings or values ​​stored in one or more memory elements within the IC. The operating modes may be set, for example, by loading a configuration bitstream into the IC. Despite this capability, a hardwired circuit is not considered a programmable circuit because it is operable and has a specific function when fabricated as part of an IC.

[0092] In the case of an SoC, the configuration bitstream may specify the circuit to be implemented in the programmable circuit and the program code to be executed by PROC 710 or a soft processor. In some cases, architecture 700 includes a dedicated configuration processor that loads the configuration bitstream into appropriate configuration memory and / or processor memory. The dedicated configuration processor does not execute user-specified program code. In other cases, architecture 700 may utilize PROC 710 to receive the configuration bitstream, load the configuration bitstream into appropriate configuration memory, and / or extract the program code for execution.

[0093] FIG. 7 is intended to illustrate an exemplary architecture that may be used to implement a programmable circuit, e.g., an IC including a programmable fabric. For example, the number of logic blocks within a column, the relative width of the columns, the number and order of columns, the types of logic blocks included in the columns, the relative sizes of the logic blocks, and the interconnect / logic implementation included on the right side of FIG. 7 are purely exemplary. In an actual IC, for example, to facilitate efficient implementation of a user's circuit design, two or more adjacent columns of CLBs are typically included wherever a CLB appears. However, the number of adjacent CLB columns may vary with the overall size of the IC. Additionally, the size and / or positioning of blocks such as PROC 710 within an IC are for illustrative purposes only and are not intended as limitations.

[0094] 8 shows another exemplary architecture 800 for an IC for hardware acceleration. The architecture 800 may be used to implement a hardware accelerator and represents an SoC-type IC. In the example, the architecture 800 includes a processor system (PS) 802. Although described as a "system," the PS 802 is an example of a subsystem of the architecture 800. Generally, the PS 802 is implemented as a hardwired circuit that is manufactured as part of the architecture 800. The PS 802 may be implemented as or include any of a variety of different processor types, each capable of executing program code. The processors may have different architectures and / or instruction sets.

[0095] In the example, the PS 802 includes various processors, such as a real-time processing unit (RPU) 806, an application processing unit (APU) 808, a graphics processing unit (GPU) 810, and a configuration and security unit (CSU) 812. Each of the processors is capable of executing program code.

[0096] The PS 802 also includes various support circuits, such as on-chip memory (OCM) 814, a transceiver 816, peripheral devices 818, an interconnect 820, DMA circuitry 822, a memory controller 824, peripheral devices 826, and multiplexed input / output (MIO) circuitry 828. The processor and support circuits are interconnected by the interconnect 820.

[0097] The transceiver 816 may be coupled to external pins 830. Programmable logic (PL) 850, described in more detail below, may be coupled to external pins 832. The memory controller 82 4 may be coupled to external pin 834. MIO 828 may be coupled to external pin 836. PS 802 may be coupled to external pin 838. APU 808 may include a CPU 840, a memory 842, and support circuits 844.

[0098] 8, each of processors 806, 808, and 810 may include one or more central processing units (CPUs) and associated circuitry, such as memory, an interrupt controller, a direct memory access (DMA) controller, a memory management unit (MMU), a floating point unit (FPU), etc. Interconnect 820 includes various switches, buses, communication links, etc. configured to interconnect processors 806, 808, 810 and to interconnect other components within PS 802 to processors 806, 808, and 810.

[0099] In an exemplary implementation, the host CPU may be implemented using CPU 840 or another processor within APU 808. Thus, if the IC further includes an embedded processor, the embedded processor may be implemented as a soft processor, as RPU 806, or as a separate hardwired processor that may be included in APU 808. In another example, if the host CPU is included in a host computer, the embedded processor may be implemented as CPU 840, as a separate hardwired processor within APU 808, as RPU 806, or as a soft processor.

[0100] The OCM 814 includes one or more RAM modules that may be distributed throughout the PS 802. For example, the OCM 814 may include battery-backed RAM (BBRAM), tightly coupled memory (TCM), etc. The memory controller 824 may include a dynamic DRAM interface for accessing external DRAM. The peripherals 818, 826 may include one or more components that provide an interface to the PS 806. For example, the peripherals 818,826 The peripherals 826 may be coupled to the MIO 828. The peripherals 818 may be coupled to the transceiver 816. The transceiver 816 may include a graphics processing unit (GPU), a display interface (e.g., DisplayPort, high-definition multimedia interface (HDMI) port, etc.), a universal serial bus (USB) port, an Ethernet port, a universal asynchronous transceiver (UART) port, a serial peripheral interface (SPI) port, a general purpose IO (GPIO) port, a serial advanced technology attachment (SATA) port, a PCIe port, etc. The peripherals 826 may be coupled to the MIO 828. The peripherals 818 may be coupled to the transceiver 816. The transceiver 816 may include a serializer / deserializer (SERDES) circuit, an MGT, etc.

[0101] 8, device 800 optionally includes a programmable logic 850 subsystem. Programmable logic 850 is circuitry that can be programmed to perform specified functions. By way of example, programmable logic 850 may be implemented as a field programmable gate array type circuit. Programmable logic 850 may include an array of programmable circuit blocks.

[0102] In the example of Figure 8, programmable logic 850 may be coupled to interconnect 820. In one aspect, programmable logic 850 may be coupled to PS 802 via an AXI interconnect. Thus, any compute unit implemented within programmable logic 850 may be accessed by PS 802 via interconnect 820, which may be implemented as an AXI interconnect. As discussed in connection with Figure 7, the topology of programmable logic 850 is highly configurable, unlike hardwired circuitry.

[0103] Device 800 may include one or more other subsystems, including a processor array (e.g., a data processing engine (DPE) array), a network-on-chip (NoC), a platform management controller (PMC), and one or more hardwired circuit blocks, which refer to dedicated circuit blocks fabricated as part of architecture 800.

[0104] Although hardwired, hardwired circuit blocks may be configured by loading configuration data into control registers to implement one or more different operating modes therein. Examples of hardwired circuit blocks may include input / output (I / O) blocks, transceivers for transmitting and receiving signals to circuits and / or systems external to architecture 800, memory controllers, etc. Examples of different I / O blocks may include single-ended and pseudo-differential I / O. An example of a transceiver may include a high-speed differential clock transceiver. Other examples of hardwired circuit blocks include, but are not limited to, cryptographic engines, digital-to-analog converters (DACs), analog-to-digital converters (ADCs), etc. Generally, hardwired circuit blocks are application-specific circuit blocks.

[0105] For purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the various inventive concepts disclosed herein. However, the terminology used herein is for the purpose of describing particular aspects of the inventive arrangements only and is not intended to be limiting.

[0106] As defined herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0107] As defined herein, the term "about" means nearly correct or exact; that a value or amount is close but not exact. For example, the term "about" can mean that a recited property, parameter, or value is within a given amount of the exact property, parameter, or value.

[0108] As defined herein, the terms "at least one," "one or more," and "and / or" are open-ended expressions that are both conjunctive and disjunctive in operation unless expressly stated otherwise. For example, each of the phrases "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, and / or C" means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.

[0109] As defined herein, the term "automatically" means without human intervention.

[0110] As defined herein, the term "computer-readable storage medium" means a storage medium that contains or stores program code for use by or in connection with an instruction execution system, apparatus, or device. As defined herein, a "computer-readable storage medium" is not a transitory, propagating signal per se. A computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Various forms of memory as described herein are examples of computer-readable storage media. A non-exhaustive list of more specific examples of computer-readable storage media may include portable computer diskettes, hard disks, RAM, read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electronically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, and the like.

[0111] As defined herein, the term "if" can mean "when" or "upon" or "in response to" or "responsive to," depending on the context. Thus, the phrase "when determined" or "when [the stated condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.

[0112] As defined herein, the term "responsive to," and similar words such as "if," "when," or "upon," means to readily respond or react to an action or event. The response or reaction is performed automatically. Thus, when a second activity is performed "responsively to" a first activity, there is a causal relationship between the occurrence of the first activity and the occurrence of the second activity. The term "responsive to" indicates a causal relationship.

[0113] As defined herein, the term "substantially" means that the specified property, parameter, or value need not be achieved exactly, and that deviations or variations, including, for example, tolerances, measurement errors, measurement accuracy limits, and other factors known to those skilled in the art, may occur in an amount that does not eliminate the effect that the property was intended to provide.

[0114] Terms such as first, second, etc. may be used herein to describe various elements. Unless otherwise stated or clearly indicated by context, these elements should not be limited by these terms, as they are only used to distinguish one element from another.

[0115] A computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to execute aspects of the inventive arrangements described herein. Within this disclosure, the term "program code" is used interchangeably with the term "computer-readable program instructions." The computer-readable program instructions described herein may be downloaded from the computer-readable storage medium to a respective computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge devices, including edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0116] Computer-readable program instructions for performing operations for the inventive arrangements described herein may be either assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, or source or object code written in any combination of one or more programming languages, including object-oriented and / or procedural programming languages. The computer-readable program instructions may include state configuration data. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a LAN or WAN, or a connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some cases, electronic circuitry, including, for example, programmable logic circuits, FPGAs, or PLAs, may execute the computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry to perform aspects of the inventive arrangements described herein.

[0117] Certain aspects of the inventive arrangements are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions, e.g., program code.

[0118] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram blocks. These computer-readable program instructions may also be stored on a computer-readable storage medium capable of directing a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer-readable storage medium having instructions stored therein comprises an article of manufacture containing instructions that implement aspects of the operations specified in the flowchart and / or block diagram blocks.

[0119] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operations to produce computer-implemented processing, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / acts specified in the flowchart and / or block diagram blocks.

[0120] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified operations.

[0121] In some alternative implementations, the actions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may be executed substantially concurrently, or the blocks may possibly be executed in reverse order depending on the functionality involved. In other examples, the blocks may generally be executed in ascending numerical order, while in still other examples, one or more blocks may be executed out of order, with results being stored and utilized in subsequent or other blocks that do not immediately follow. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may execute a combination of dedicated hardware and computer instructions.

[0122] The corresponding structure, material, acts, and equivalents of all means-plus-function or step-plus-function elements that may be found in the following claims are intended to include any structure, material, or acts for performing the function in combination with other elements recited in the claim, as specifically recited in the claim.

[0123] The description of the inventive arrangements provided herein is for illustrative purposes and is not intended to be exhaustive or limited to the disclosed forms and examples. The terminology used herein has been selected to explain the principles, practical applications, or technical improvements to commercially available technology of the inventive arrangements, and / or to enable others skilled in the art to understand the inventive arrangements disclosed herein. Modifications and variations may be apparent to those skilled in the art without departing from the scope and spirit of the inventive arrangements described. Accordingly, reference should be made to the following claims, rather than the foregoing disclosure, as indicating the scope of such features and implementations.

Claims

1. a switch communicatively linked to a host central processing unit (CPU); an adapter circuit coupled to the switch via a control bus configured to communicate an address of a descriptor from the host CPU to the adapter circuit, the host CPU initiating a register write via the control bus to write the address of the descriptor to a register within the adapter circuit; a random access memory coupled to the switch via a data bus, the random access memory configured to store descriptors received from the host CPU via the data bus; a computation unit coupled to the adapter circuit and configured to perform the operations specified by the descriptor; Equipped with the adapter circuit is configured to retrieve the descriptor from the random access memory via the data bus using the address received via the control bus, provide arguments from the descriptor to the computation unit, and provide control signals to the computation unit to initiate the operation using the arguments.

2. 2. The hardware acceleration device of claim 1, wherein the data bus is a memory-mapped bus.

3. the switch is a Peripheral Component Interconnect Express (PCIe) switch; the control bus and the data bus are implemented as on-chip interconnects; 10. The hardware acceleration device of claim 1.

4. 4. The hardware acceleration device of claim 3, wherein the PCIe switch includes a first BAR coupled to the control bus and a second BAR coupled to the data bus.

5. 2. The hardware acceleration device of claim 1, wherein the adapter circuit comprises a first-in-first-out memory configured to store the address.

6. 6. The hardware acceleration device of claim 5, wherein the adapter circuitry comprises a counter configured to store a number of operations that have completed processing by the computation unit.

7. The hardware acceleration device of claim 1 comprising a programmable integrated circuit including the switch, the adapter circuit, and the computation unit.

8. 8. The hardware acceleration device of claim 7, wherein the programmable integrated circuit includes the random access memory.

9. 9. The hardware acceleration device of claim 8, wherein the random access memory is implemented within the programmable integrated circuit, implemented external to the programmable integrated circuit and located on a circuit board with the programmable integrated circuit, or implemented as a high bandwidth memory.

10. The programmable integrated circuit an embedded processor; A network interface; Equipped with 8. The hardware acceleration device of claim 7, wherein the embedded processor is configured to store in the random access memory a further descriptor received from a network via the network interface and to provide an address of the further descriptor to the adapter circuit.

11. the programmable integrated circuit including the host CPU embedded therein; 8. The hardware acceleration device of claim 7, wherein the switch includes a first port coupled to the control bus and a second port coupled to the data bus.

12. receiving, via a switch of a hardware acceleration device, a plurality of descriptors and a plurality of addresses of the plurality of descriptors, the switch being communicatively linked to a host central processing unit (CPU), the descriptors specifying operations to be performed by computational units of the hardware acceleration device; communicating said plurality of descriptors to a random access memory via a data bus for storage therein; communicating the addresses to an adapter circuit via a control bus, the host CPU initiating a register write via the control bus to write the addresses of the descriptor to a register within the adapter circuit; using the adapter circuit to retrieve the descriptor from the random access memory via the data bus using the plurality of addresses received via the control bus, provide arguments from the descriptor to the compute unit, and provide control signals to the compute unit to initiate the operation using the arguments.

13. The method of claim 12 , wherein the switch is a Peripheral Component Interconnect Express (PCIe) switch.

14. 14. The method of claim 13, wherein the PCIe switch includes a first BAR coupled to the control bus and a second BAR coupled to the data bus.

15. the hardware acceleration device includes a programmable integrated circuit; The method of claim 12 , wherein the programmable integrated circuit includes the switch, the adapter circuit, and the computation unit.

Images