Domain-supported processor-peer for coherent acceleration

Abstract

The examples herein describe peripheral I / O devices equipped with a domain assist processor (DAP) and a domain specific accelerator (DSA) located within the same coherent domain as the CPU and memory within the host computing system. Previously, it was not possible for peripheral I / O devices to participate in the cache coherent shared memory multiprocessor paradigm using hardware resources within the host computing system. Consequently, domain assist processing for light-load processor functions (e.g., open source functions such as gzip, open source encryption libraries, open source network switches, etc.) is achieved by using CPU resources within the host or by provisioning a special processing system in the peripheral I / O device (e.g., using programmable logic in an FPGA). The embodiments herein use the DAP within the peripheral I / O device to perform light-load processor functions that would otherwise be performed by hardware resources within the host or by a special processing system within the peripheral I / O device.

Description

Technology Field

[0001] Examples of the present disclosure generally relate to a peripheral device equipped with a domain support processor and a domain-specific accelerator. Background Technology

[0002] In the traditional I / O model, a host computing system interfaces with a peripheral I / O device when executing accelerator tasks or functions using the peripheral I / O device's own custom I / O device. Having multiple I / O devices, or even multiple instances of the same I / O device, means that the host interfaces with multiple I / O device drivers or multiple execution copies of the same I / O device driver. This can lead to security and reliability issues because, although I / O device drivers are typically developed by the vendor supplying the peripheral I / O device, they must be integrated with both the software and hardware within the host computing system.

[0003] Meanwhile, the hardware cache coherent shared-memory multiprocessor paradigm utilizes a generic ISA (instruction set architecture) independent interface model for tasks or functions executed on multiprocessor CPUs. This generic ISA independent (e.g., C code) interface model scales based on both the number of processing units and the amount of shared memory available to those processing units. Traditionally, peripheral I / O devices could not benefit from the coherent paradigm used by CPUs running on a host computing system. Prior art literature

[65535] U.S. Patent Application Publication US 2016 / 0344629 A1 (2016.11.24.)

[0004] A technique for operating a domain assist processor on a peripheral device is described. An example is a peripheral I / O device comprising a domain assist processor (DAP), a domain accelerator, and a gateway, wherein the gateway is configured to connect the peripheral I / O device to a host communicably using a coherent interconnect protocol, wherein the coherent interconnect protocol extends a coherent domain within the host to the peripheral I / O device so that the DAP becomes a peer processor to a processor within the host. The gateway is configured to receive a first request from the host using the coherent interconnect protocol, the first request instructs the DAP to perform processor functions and receive a second request from the host using the coherent interconnect protocol, and the second request instructs the domain accelerator to perform accelerator tasks.

[0005] In some embodiments, the DAP includes at least one processing core and an operating system, and the operating system is one of an embedded standalone operating system and an extension of an operating system within a host.

[0006] In some embodiments, the instruction set architecture (ISA) used by the processing core is different from the ISA used by the processor in the host.

[0007] In some embodiments, peripheral I / O devices include a programmable logic array, and at least part of the domain accelerator is executed by the programmable logic array.

[0008] In some embodiments, at least a portion of the domain accelerator is executed on hardened circuitry.

[0009] In some embodiments, the peripheral I / O device includes a network on a chip (NoC) that enables communication between the gateway and the domain accelerator.

[0010] In some embodiments, the gateway is configured to establish an I / O domain and a coherent domain from peripheral I / O devices.

[0011] In some embodiments, the gateway includes an endpoint coherent interconnect coupled to communicate with both the DAP and the domain accelerator.

[0012] In some embodiments, the gateway includes an endpoint coherent interconnect and a cache coupled to enable communication between the DAP and the domain accelerator.

[0013] Another example is a method, which includes the steps of: identifying a processor function to offload from a CPU-memory complex within a host to a DAP within a peripheral I / O device—a coherent domain containing the CPU-memory complex of the host extending to a DAP within a peripheral I / O device—using a coherent interconnect protocol to instruct the DAP to perform a processor function and store resulting data in a memory element within the coherent domain, using a coherent interconnect protocol, instructing a domain accelerator within a peripheral I / O device to process the resulting data when performing an accelerator task for the host, and receiving the resulting data from the domain accelerator.

[0014] In some embodiments, the DAP includes at least one processing core and an operating system, and the operating system is one of an embedded standalone operating system and an extension of an operating system within the post.

[0015] In some embodiments, at least part of the domain accelerator is executed by a programmable logic array.

[0016] In some embodiments, the CPUs within the DAP and the host are part of the same cache coherent shared memory multiprocessor paradigm.

[0017] In some embodiments, the method includes the step of using an I / O device driver in a host to command a compute resource assigned to an I / O domain in a peripheral I / O device to perform an I / O task.

[0018] Another example is a computing system, which includes a host containing a CPU-memory complex in a coherent domain, and peripheral I / O devices including a domain support processor, a domain accelerator, and a gateway, wherein the gateway is configured to communicatively couple the peripheral I / O devices to the host using a coherent interconnect protocol, and the coherent interconnect protocol extends the coherent domain within the host to include a DAP. The gateway is configured to communicatively couple the peripheral I / O devices to the host using a coherent interconnect protocol, and the coherent interconnect protocol extends the coherent domain within the host to include a DAP. The gateway is configured to receive a first request from the host instructing the DAP to perform processor functions and to receive a second request from the host instructing the domain accelerator to perform accelerator tasks. Brief explanation of the drawing

[0019] To enable a detailed understanding of the aforementioned features, the briefly summarized above may be described in more detail with reference to exemplary embodiments, some of which are illustrated in the attached drawings. However, since the attached drawings only illustrate typical embodiments, they should not be construed as limiting the scope. FIG. 1 is a block diagram of a host coupled to peripheral I / O devices including a domain support processor and a domain-specific accelerator, according to an example. FIG. 2 is a block diagram of a peripheral I / O device equipped with a hybrid gateway according to an example. FIG. 3 is a flowchart for assigning tasks to a domain-supported processor and a domain-specific accelerator according to an example. FIG. 4a is a block diagram of a gateway having an endpoint coherent interconnect connected to a domain support processor and a domain-specific accelerator, according to an example. FIG. 4b is a block diagram of a gateway having a cache connected to a domain support processor and a domain-specific accelerator, according to an example. FIG. 5 is a block diagram of a host coupled to a peripheral I / O device having an I / O and coherent domain according to an example. FIG. 6 is a block diagram of a peripheral I / O device having a network-on-chip logically divided into programmable logic, memory, and I / O and coherent domains according to an example. FIG. 7 illustrates an example of a field-programmable gate array implementation of a programmable IC according to one example. Specific details for implementing the invention

[0020] Various features are described below with reference to the drawings. It should be noted that the drawings may or may not be drawn to scale, and that elements of similar structure or function are indicated by the same reference number throughout the drawings. It should be noted that the drawings are intended to facilitate the description of features. The drawings are not intended to serve as a comprehensive description of the specification or as a limitation on the claims. Furthermore, the illustrated examples are not required to possess all the aspects or advantages shown. The aspects or advantages described in relation to a particular example are not necessarily limited to that example and may be practiced in other examples even if they are not so exemplified or so explicitly described.

[0021] The examples herein describe peripheral I / O devices equipped with a domain assist processor (DAP) and a domain specific accelerator (DSA) located within the same coherent domain as the CPU and memory within the host computing system. As previously mentioned, it was previously impossible for peripheral I / O devices to participate in the cache coherent shared memory multiprocessor paradigm using hardware resources within the host computing system. Consequently, domain assist processing for lightweight general processor functions (e.g., open source functions such as gzip, open source encryption libraries, and open source network switches) is achieved by using CPU resources within the host or by provisioning a specialized processing system in the peripheral I / O device (e.g., using programmable logic on an FPGA). However, using CPUs in the host to perform lightweight processor functions implies that these CPUs are unavailable for performing heavyweight functions and also wastes power. In addition, provisioning a specialized processing system on an FPGA means that there is less programming logic available on the FPGA for performing accelerator tasks (tasks that are different from light-load processor functions).

[0022] The embodiments herein use a DAP within a peripheral I / O device to perform light-duty processor functions that would otherwise be performed by hardware resources within the host or by a special processing system within the peripheral I / O device. In one embodiment, the DAP is a processor subsystem that includes a processor running an operating system. Using a peripheral coherent interconnect allows the DAP within the peripheral I / O device to share the same coherent domain as the hardware resources within the host computing system. Without a peripheral coherent interconnect, the DAP becomes an embedded system invisible to the computing resources within the host. In this case, the host can communicate with the DAP only through an I / O device driver. By adding the DAP to the same coherent domain as the hardware resources in the host computing system, the DAP becomes a peer of the CPU in the host computing system. In other words, both the DAP and the host CPU are managed by the same operating system (OS) or kernel in the host computing system. As a result, the OS or kernel within the host can intelligently determine which light-load processor functions can be offloaded to the DAP within the peripheral I / O device and which functions should be performed by the CPU within the host computing system.

[0023] In one embodiment, the OS offloads processor functions that are part of an accelerator task assigned to a DAS within a peripheral I / O device. For example, the OS may instruct the DAP to perform a gzip function to decompress a compressed data file. Additionally, the OS may then instruct the DSA to perform a word or phrase search on the decompressed data. In this example, rather than having the CPU within the host perform the gzip function first, the OS may offload the execution of both the gzip function and the accelerator task (e.g., word search) to a peripheral I / O device.

[0024] FIG. 1 is a block diagram of a host coupled to a peripheral I / O device including a domain-supported processor and a domain-specific accelerator, according to one example. The computing system (100) of FIG. 1 includes a host (105) that is communicably coupled to a peripheral I / O device (135) using a PCIe connection (130). The host (105) may represent a single computer (e.g., a server) or a plurality of interconnected physical computing systems. In some cases, the host (105) includes an operating system (OS) (110), a plurality of CPUs (115), and memory (120). The OS (110) may be any OS capable of performing the functions described herein. In one embodiment, the OS (110) (or hypervisor or kernel) establishes a cache-coherent shared-memory multiprocessor paradigm for the CPUs (115) and memory (120) (referred to as a CPU-memory complex). In other words, the CPU (115) and memory (120) are subject to OS management (or kernel / hypervisor management) to form a coherent domain following the cache coherent shared memory multiprocessor paradigm. However, as previously mentioned, in the traditional I / O model, peripheral I / O devices (135) (and their entire computational resources) are excluded from the coherent domain established on the host (105). In the traditional model, the host (105) relies solely on the I / O device driver (125) stored in the corresponding memory (120) to manage the computational resources within the I / O device (135) and to communicate with the computational resources. That is, the peripheral I / O device (135) is controlled by the I / O device driver (125) and can be accessed through the I / O device driver (125).

[0025] In the embodiment herein, the coherent shared memory multiprocessor paradigm is extended to peripheral I / O devices (135) along with all performance benefits, software flexibility, and reduced overhead of the paradigm. Additionally, by adding compute resources within the I / O devices to the same coherent domain as the CPU (115) and memory (120), a general ISA-independent development environment may be allowed in which the hardware resources within the host (105) use a different ISA than the hardware resources within the peripheral I / O devices (135). That is, the compute system is ISA-agnostic. However, in other embodiments, the coherent shared memory multiprocessor paradigm may require that the hardware resources of both the host (105) and the I / O devices (135) use the same ISA rather than different ISQs.

[0026] As illustrated in FIG. 1, the peripheral I / O device (135) includes a coherent gateway (140) that uses a coherent interconnect protocol to extend a coherent domain, including the CPU (115) and memory (120) of the host (105), to the peripheral I / O device (135). That is, the DAP (145) and DSA (150) within the I / O device (135) share the same coherent domain as the CPU (115) and memory (120). As a result, the DAP (145) is a peer processor of the CPU (115) within the host (105). In one embodiment, both the CPU (115) and the DAP (145) are managed by an OS (110), kernel, or hypervisor running on the host (105). Additionally, the cache coherent shared memory multiprocessor paradigm may allow the use of non-uniform memory access (NUMA) attributes when the OS (110) can make intelligent decisions regarding task assignment based on proximity. That is, by using NUMA attributes, the OS (110) can know that the CPU (115) on the host computing system (105) is located in the same location as memory, and the DAP (145) (and any associated memory within the peripheral I / O device (135)) is located in the I / O device (135). Accordingly, the CPU (115) and the DAP (145) are peers, but they have different proximity to memory within the system, and thus may result in different latencies. In this way, the OS (110) can intelligently assign tasks based on the location and proximity of the CPU (115) and DAP (145) to various memory elements within the system.

[0027] The DAP (145) includes its own OS (155). In one embodiment, the OS (155) is an independent embedded OS. However, in another embodiment, the shared memory multiprocessor paradigm allows the OS (155) to be the same as the OS (110) within the host (105) or a subset of the OS (100). For example, the OS (155) may be an extension of the OS (110) that manages memory and computes resources on peripheral devices (135).

[0028] Regardless of whether the OS (155) is an independent OS or part of or an extension of the OS (110), since the DAP (145) is managed by the OS (110) within the host (105), the OS (110) can submit tasks to the OS (155), and the OS (155) then controls hardware elements within the DAP (145) to complete the tasks. For example, the DAP (145) may be an ARM processor, but it may also be a processor using other types of ISAs, such as x86 or PowerPC®. In some cases, the DAP (145) and the OS (155) may execute general functions that need to be performed by the CPU (115) or by a special processing system within a peripheral I / O device (135) (e.g., using programmable logic (PL) if the I / O device (135) is an FPGA). In one embodiment, the DAP (145) is a CPU, although its computational power may be far inferior to that of the CPU (115) within the host (105). Additionally, the DAP (145) may include one or more processing cores (e.g., multiple ARM cores) and may include local memory, such as local DDR memory storing the OS (155). In the case of a traditional model, the OS (155) is opaque or invisible to the OS (110) on the host (105). That is, the OS (110) simply views the entire I / O device (1335) as a whole without knowing what different types of hardware are present within the I / O device (135). With the addition of a coherent gateway (140) and a coherent interconnect protocol, the OS (100) can view the DAP (145) as a peer processor to the CPU (115).The OS (110) can determine whether the DAP (145) has available computational resources (or available computational time), available local memory, I / O devices (135), and the proximity of the DAP (145) to memory elements on the host (105), and whether there are other advantages possible by adding the DAP (145) to the same coherent domain as the host (105).

[0029] The DSA (150) is an accelerator that performs acceleration tasks for the host (105). In one embodiment, the DSA (150) is part of the same coherent domain as the DAP (145) and the host (105), but this is not mandatory. For example, the DSA (150) may be assigned to an I / O domain managed by an I / O device driver (125) rather than being managed by the OS (110). In another embodiment, some of the hardware elements within the DSA (150) are assigned to a coherent domain (so they are subject to OS / kernel management), and the rest of the hardware elements within the DSA are assigned to an I / O domain (so they are managed by the I / O device driver (125)).

[0030] The DSA (150) may include hardened circuitry, programmable circuitry (e.g., PL), or a combination thereof. For example, the DSA (150) may include special processing elements (e.g., a multiplicator accumulator or an array of data processing engines) or a graphics processing unit (GPU). The DSA (150) may include a PL array having a plurality of configurable blocks. Additionally, the DSA (150) may include a mix of special processing elements (e.g., a data processing engine array) and PLs.

[0031] FIG. 2 is a block diagram of a peripheral I / O device equipped with a hybrid gateway according to an example. For convenience, various hardware and software within the host (105) are not illustrated. The peripheral I / O device (135) includes a hybrid gateway (205) that logically divides the hardware elements within the device (135) into a coherent domain and an I / O domain. Various methods for dividing the hardware elements into these domains will be discussed below.

[0032] To extend the coherent domain of the host (105) to peripheral I / O devices (135), the hybrid gateway uses a coherent interconnect protocol. For example, the hybrid gateway (205) may use a cache coherent interconnect for accelerators (CCIX) to form a coherent domain. CCIX is a high-performance chip-to-chip interconnect architecture that provides a cache coherent framework for heterogeneous system architectures. CCIX brings kernel-managed semantics to peripheral devices. Cache coherence is always automatically maintained between the CPU on the host (105) and various other accelerators within the system that can be placed on any number of peripheral I / O devices. However, to extend the coherent domain of the host (105) to include computational resources within the peripheral I / O device (135), other coherent interconnect protocols other than CCIX, such as QuickPath Interconnect (QPI) or Omni-Path, may also be used.

[0033] The hybrid gateway (205) includes a peripheral component interconnect express (PCIe) endpoint (210) and a DAP (145). The PCIe endpoint (210) enables the hybrid gateway (205) to use PCIe communication technology to transmit data across the host (105). For example, I / O domain traffic and coherent domain traffic may be transmitted using the same PCIe connection (or connections). In one embodiment, a special header or header value may be used to indicate which data packet is I / O domain traffic and which is coherent domain traffic.

[0034] In FIG. 2, the hybrid gateway (205) is communicably coupled to the PL array (215) and the network-on-chip (NoC) (220). The PL array (215) includes a DSA (150) that performs accelerator tasks provided by the host (105). For example, the DSA (150) may be configured (using the PL) to perform machine learning tasks, encryption acceleration, compression / decompression tasks, digital or analog processing tasks, graphics acceleration, simulation, etc. Additionally, as previously mentioned, the DSA (150) may be implemented using the PL alone, the hardening circuit alone, or a mix of the hardening circuit and the PL.

[0035] In one embodiment, the NoC (220) includes an interface element that enables hardware elements (e.g., DSA (150), memory elements, hybrid gateway (205), etc.) within the I / O device (135) to transmit and receive data. In one embodiment, rather than using a PL to form the NoC (220), some or all of the components forming the NoC (220) are hardened. In some cases, as will be described in more detail below, the NoC (220) may be logically divided between an I / O domain and a coherent domain.

[0036] In FIG. 2, a peripheral I / O device (135) is coupled to an attachment memory (230). The attachment memory (230) may be an integrated circuit separate from the integrated circuit forming the peripheral device (135) (e.g., a system-on-chip (SoC)). The I / O device (135) may be coupled to the attachment memory (230) so as to be communicable using a high-speed bus. However, the use of the attachment memory (230) is optional. Additionally, the peripheral I / O device (135) may include an internal memory element (and an internal memory controller).

[0037] Attached memory (230) includes a device driver managed memory (235) and a coherent memory (240). The device driver managed memory (235) is managed by an I / O device driver (not shown) within the host (105), whereas the coherent memory (240) is managed by an OS / kernel / hypervisor (likewise not shown in FIG. 2) within the host (105). Thus, the device driver managed memory (235) is part of the I / O domain, but the coherent memory (240) is part of the coherent domain. As a result, hardware elements within peripheral I / O devices (135) assigned to the I / O domain can communicate with the device driver managed memory (235) rather than the coherent memory (240), and hardware elements assigned to the coherent domain can communicate with the coherent memory (240) rather than the device driver managed memory (235).

[0038] FIG. 3 is a flowchart of a method (300) for assigning tasks to a DAP and a DSA according to an example. In block 305, an OS (or kernel or hypervisor within the host) identifies light-load general processor functions that can be offloaded to a DAP within a peripheral I / O device. In one embodiment, the DAP is not suitable for performing heavy-load functions (e.g., running an Ethernet TCP / IP software stack or a server database application) but can perform light-load processor functions. In one embodiment, these light-load processor functions are open-source libraries or industry-standard software applications. Examples of light-load processor functions suitable for the DAP include compression / decompression algorithms (e.g., gzip) or secondary processing when performing data plane network routing. These functions may be more suitable for the DAP than for the CPU-memory complex within the host.

[0039] In one embodiment, the processor function offloaded to the DAP is associated with an accelerator task assigned to the DSA. In one embodiment, data generated by the DAP when performing a light-load processor function is transferred to the DSA to be processed when performing the accelerator task. For example, a scheduler within the OS may instruct the DAP to decompress a 2 MB file to obtain uncompressed data of a 100 MB file. The DSA may then perform an accelerator task (e.g., search) on the uncompressed data. In this case, the accelerator task has a data dependency on the light-load function performed by the DAP.

[0040] In one embodiment, the processor function offloaded to the DAP is a light-load OS scheduler module, which is a subset of the OS (110) responsible for scheduling tasks in the DAP and DSA. In one embodiment, the light-load OS scheduler determines which of the available functions offloaded to the DAP is dispatched as the next processor function to be executed by the DAP. In another embodiment, the light-load OS scheduler determines which of the available data-dependent functions completed by the DAP is dispatched as the next task to be executed by the DSA.

[0041] In block 310, the OS within the host uses a coherent interconnect protocol to instruct the DAP to perform a function and store the resulting data in coherent memory. Because at least some of the host and I / O devices are in the same coherent domain, the OS within the host can bypass I / O device drivers by using a coherent interconnect protocol (e.g., CCIX, QPI, or Omni-Path) to instruct the DAP to perform light-duty functions. If all hardware elements are assigned to the same coherent domain as the host, the host may not have installed I / O device drivers (e.g., an I / O device driver is not needed because the I / O device does not have an I / O domain).

[0042] The OS may use any of the tools known in the cache coherent shared memory multiprocessor paradigm to provide instructions to the DAP. For example, since the DAP is a peer processor to the CPU within the host, the OS may communicate with and manage the DAP using the same semantics used with the host CPU. These semantics may include semaphores, doorbells, implicit updates, etc.

[0043] The OS can instruct the DAP to store the results of performing heavy-duty functions in a specific memory. By using NUMA attributes, the OS can determine the proximity of the DAP to memory elements within the system (which may include memory elements within the host, peripheral I / O devices, and attached memory). If data is intended to be used by the DSA to perform an accelerator task, the OS can instruct the DSA to store the data in memory elements within I / O devices or attached memory, as this can provide lower latency compared to storing the data on the host. However, if a light-duty function performed by the DAP is unrelated to the accelerator task assigned to the DSA, the OS can instruct the DAP to store the result data on the host (e.g., if the host's CPU requires the data).

[0044] In block 315, the OS uses a coherent interconnect protocol to instruct the DSA to process data stored by the DAP. In this example, there is a data dependency between the data generated by the DAP and the accelerator task performed by the DSA. In one embodiment, the OS instructs the DSA to wait for a ready signal from the DAP indicating that there is data ready to be processed by the DSA. Using the previous example, if the DAP has finished decompressing the compressed data, the DAP may inform the DSA of the location where the uncompressed data is stored so that the DSA can retrieve the data (e.g., using a NoC) and perform the accelerator task.

[0045] In block 320, the host receives the result of the accelerator task performed by the DSA. Since the DSA may be part of a coherent domain, I / O devices can use gateway and coherent interconnect protocols to deliver the data generated by the DSA to the host as the part performing the accelerator function. The OS can then inform the CPU within the host that the accelerator task is complete and where the result data can be found in memory.

[0046] FIG. 4a is a block diagram of a hybrid gateway (205) having an endpoint coherent interconnect (405) connected to a DAP (145) and a DSA (150) according to an example. The DAP (145) is part of the same coherent domain as the CPU-memory complex within the host (105). Accordingly, all resources can be subject to OS or kernel management, and the shared memory multiprocessor paradigm extends to accelerators, such as I / O devices (135). That is, using a coherent interconnect protocol also enables a shared memory multiprocessor model so that the host (105) and the DAP (145) can have different ISAs. Additionally, the DSA (150) is connected to a PL domain-specific adaptive computer (415), for example, a custom processor running a specific subset of the host (105) ISA or DAP (150) ISA, or a custom processor that may be implemented using PL in a PL array (215). In one embodiment, the PL domain-specific adaptive computer (415) is a PL DAP, so as to provide flexibility to run a specific subset ISA or a custom ISA within programmable logic to improve computational efficiency.

[0047] FIG. 4a illustrates a first DSA / DAP topology, wherein the DSA (150) (also assigned to a coherent domain) has its own DAP (145) connected via the same endpoint coherent interconnect (405). FIG. 4b is similar to the I / O device (135) exemplified in FIG. 4a, but FIG. 4b illustrates a second, different DSA / DAP topology. In FIG. 4b, both the DSA (150) and the DAP (145) are attached to a shared cache (420) for low-latency pre- and post-DSA processed data access.

[0048] In one embodiment, along with sharing the same coherent domain as the CPU-memory complex within the host (105), the I / O device (135) illustrated in FIG. 4a and FIG. 4b may also enable data movement—e.g., data movement of a pre-processed accelerator data footprint or a post-processed footprint to host memory—through a traditional I / O device driver management target model. Additionally, the hybrid gateway (205) includes a gateway NoC (410) that facilitates communication between hardware elements within the gateway (205) (e.g., PCIe endpoint (210) and endpoint coherent interconnect (405)) and the NoC (220).

[0049] FIG. 5 is a block diagram of a host (105) coupled to a peripheral I / O device (135) having an I / O and coherent domain according to an example. The computing system (500) of FIG. 5 includes a host (105) (e.g., the host (105) of FIG. 1) that is communicably coupled to a peripheral I / O device (135) using a PCIe connection (530). As previously described, the CPU (115) and memory (120) are subject to OS management (or kernel / hypervisor management) to form a coherent domain following the cache coherent shared memory multiprocessor paradigm. In the traditional I / O model, the peripheral I / O device (135) (and its entire computational resources (550)) are excluded from the coherent domain built on the host (105). In the traditional model, the host (105) relies solely on the I / O device driver (125) stored in the corresponding memory (120) to manage computational resources (550) within the I / O device (135). That is, the peripheral I / O device (135) is controlled by the I / O device driver (125) and can be accessed through the I / O device driver (125).

[0050] Computational resources (550C and 500D) are logically assigned to the coherent domain (560), while computational resources (550A and 500B) are assigned to the I / O domain (545). Thus, the I / O device (135) has the benefit of having computational resources (550) assigned to both domains (545, 560). The I / O domain (545) provides efficiency when performing large-scale memory transfers between the host (105) and the I / O device (135), and the coherent domain (560) provides the aforementioned performance benefits, software flexibility, and overhead reduction. By logically dividing the hardware computational resources (550) (e.g., programmable logic, network-on-chip (NoC), data processing engine, and / or memory) into the I / O domain (545) and the coherent domain (560), the I / O device (135) can benefit from both types of paradigms.

[0051] To enable the host (105) to transmit and receive both I / O and coherent data traffic, the peripheral I / O device (135) includes a hybrid gateway (205) that separates data received via a PCIe connection (530) into I / O data traffic and coherent data traffic. The I / O data traffic is delivered to computational resources (550A and 550B) within the I / O domain (545), while the coherent data traffic is delivered to computational resources (550C and 550D) within the coherent domain (560). In one embodiment, the hybrid gateway (205) can process the I / O and coherent data traffic in parallel so that the computational resources (550) within the I / O domain (545) can be executed in parallel with the computational resources (550) within the coherent domain (560). That is, the host (105) can execute these tasks in parallel by assigning tasks to both the computational resources (550) in the I / O domain (545) and the coherent domain (560).

[0052] Peripheral I / O devices (135) may be different types of I / O devices, such as pluggable cards (connected to expansion slots of the host (105)), SoCs, GPUs, FPGAs, etc. In the embodiments described herein, computing resources (550) are divided into two domains, but in other embodiments, the hybrid gateway (205) may be modified to support additional domains or multiple subdomains within the I / O and coherent domains (545, 560).

[0053] FIG. 6 is a block diagram of a peripheral I / O device (135) having a PL array (215), a memory block (620), and a NoC (220) logically divided into I / O and coherent domains, according to one example. In this example, the PL array (215) is formed of a plurality of PL blocks (610). These blocks may be individually assigned to an I / O block (545) or a coherent domain (560). That is, PL blocks (610A and 610B) are assigned to an I / O domain (545), while PL blocks (610C and 610D) are assigned to a coherent domain (560). In one embodiment, so that there is no overlap between the blocks (e.g., so that PL blocks (610) are not assigned to both the I / O and coherent domains), the set of PL blocks (610) assigned to the I / O domain is mutually exclusive with the set of PL blocks (610) assigned to the coherent domain. In one embodiment, PL blocks (610A and 610B) may be used to form a DSA in the I / O domain (545), but PL blocks (610C and 610D) are used to form a DSA in the coherent domain (560).

[0054] In one embodiment, assigning hardware resources to one of the I / O domains (545) or coherent domains (560) does not affect (or indicate) the physical location of the hardware resources within the I / O device (135). For example, PL blocks (610A and 610C) may be assigned to different domains even if these blocks are adjacent to each other in the PL array (215). Accordingly, the physical location of the hardware resources in the I / O device (135) may be considered when logically assigning them to the I / O domain (545) and coherent domain (560), but is not essential.

[0055] Additionally, the I / O device (135) includes a memory controller (615) assigned to an I / O domain (545) and a coherent domain (560). In one embodiment, due to the physical interconnection between the memory controller (615) and the corresponding memory block (620), assigning one of the memory controllers (615) to one of the I / O or coherent domains (545, 560) also implies that all memory blocks (620) connected to the memory controller (615) are assigned to the same domain. For example, the memory controller (615) may be connected to a fixed set of memory blocks (620) (not connected to other memory controllers (615)). Accordingly, the memory blocks (620) may be assigned to the same domain as the memory controller (615) to which they are connected. However, in other embodiments, it may be possible to assign memory blocks (620) connected to the same memory controller (615) to different domains.

[0056] In one embodiment, the NoC (220) includes an interface element that enables a hardware element within the I / O device (135) (e.g., a configurable data processing engine, a memory block (620), a PL block (610), etc.) to use the NoC (220) to transmit and receive data. In one embodiment, rather than using programmable logic to form the NoC (220), some or all of the components forming the NoC (220) are hardened. In some cases, the NoC (220) may be logically divided between an I / O domain (545) and a coherent domain (560). In one embodiment, instead of assigning different parts of the NoC (220) to the two domains, the parameters of the NoC are configured to provide different service levels for data traffic corresponding to the I / O domain (545) and the coherent domain (560). That is, data traffic for both domains flowing through the NoC (220) may use the same hardware elements (e.g., switches and communication links) but may be handled differently by the hardware elements. For example, the NoC (220) may provide different quality of service (QoS), latency, and bandwidth for two different domains. Additionally, the NoC (220) may also isolate traffic from the I / O domain (545) from traffic from the coherent domain (560) for security reasons.

[0057] In another embodiment, the NoC (220) may prevent computational resources within the I / O domain (545) from communicating with computing resources within the coherent domain (560). However, in one embodiment, it may be advantageous to allow computational resources assigned to the I / O domain (545) to communicate with computing resources assigned to the coherent domain (560). Previously, this communication would have occurred between the I / O device driver (125) and the OS of the host (105). Instead, communication between domains may occur using the NoC (220) within the I / O device (135) (where the computational resources are far from the device (135)) or using a fabric-to-fabric connection in the PL array (215) (where two PL blocks (610) assigned to two different domains are close to each other and need to communicate).

[0058] FIG. 7 illustrates an example of an implementation of an FPGA (700) of the PL array (215) of FIG. 2, more specifically, an I / O peripheral device (135) comprising a number of different programmable tiles including a transceiver (37), a CLB (33), a BRAM (34), an input / output block ("IOB") (36), configuration and clocking logic ("config / clock") (42), a DSP block (35), a special input / output block ("IO") (41) (e.g., a configuration port and a clock port), and other programmable logic (39) such as a digital clock, an analog-to-digital converter, and system monitoring logic. The FPGA may also include a PCIe interface (40), an analog-to-digital converter (ADC) (38), etc.

[0059] In some FPGAs, each programmable tile may include at least one programmable interconnect element ("INT") (43) having a connection to an input / output terminal (48) of a programmable logic element within the same tile, as shown in the example included at the top of FIG. 7. Each programmable interconnect element (43) may also include a connection to an interconnect segment (49) of an adjacent programmable interconnect element within the same tile or another tile. Each programmable interconnect element (43) may also include a connection to an interconnect segment (50) of a general routing resource between logic blocks (not shown). A general routing resource may include a routing channel comprising a track of an interconnect segment (e.g., interconnect segment (50)) between logic blocks (not shown) and a switch block (not shown) for connecting the interconnect segments. An interconnect segment of a general routing resource (e.g., an interconnect segment (50)) may span one or more logic blocks. A programmable interconnect element (43) taken together with the general routing resource implements a programmable interconnect structure ("programmable interconnect") for the illustrated FPGA.

[0060] In an exemplary implementation, the CLB (33) may include a configurable logic element ("CLE") (44) that can be programmed to implement user logic and a single programmable interconnect element ("INT") (43). The BRAM (34) may include a BRAM logic element (BRL) (45) along with one or more programmable interconnect elements. In one embodiment, the BRAM (34) is a part of memory capable of holding stored data during reconfiguration. Typically, the number of interconnect elements contained within a tile depends on the height of the tile. In the illustrated example, the BRAM tile has the same height as five CLBs, but a different number (e.g., four) may be used. The DSP tile (35) may include a DSP logic element ("DSPL") (46) along with an appropriate number of programmable interconnect elements. The IOB (36) may include, for example, one instance of a programmable interconnect element (43) and two instances of an input / output logic element ("IOL") (47). As will be obvious to those skilled in the art, the actual I / O pad connected to, for example, the I / O logic element (47) is not limited to the area of ​​the input / output logic element (47).

[0061] In the illustrated example, a horizontal area adjacent to the center of the die (illustrated in FIG. 7) is used for configuration, clock, and other control logic. A vertical column (51) extending from this horizontal area or column is used to distribute clock and configuration signals across the width of the FPGA.

[0062] Some FPGAs utilizing the architecture illustrated in Fig. 7 include additional logic blocks that disrupt the regular cylindrical structure that makes up much of the FPGA. The additional logic blocks may be programmable blocks and / or dedicated logic.

[0063] It should be noted that FIG. 7 is intended solely to illustrate an exemplary FPGA architecture. For example, the number of logic blocks in a row, the relative width of the row, the number and order of the rows, the types of logic blocks included in the row, the relative sizes of the logic blocks, and the interconnect / logic implementations included in the top part of FIG. 7 are purely illustrative. For example, in an actual FPGA, multiple adjacent rows of CBLs are typically included wherever CLBs are visible to facilitate the effective implementation of user logic, but the number of adjacent CLB rows varies depending on the overall size of the FPGA.

[0064] For the foregoing, reference has been made to the embodiments presented in this disclosure. However, the scope of this disclosure is not limited to the specific embodiments described. Rather, any combination of the described features and elements, whether or not related to different embodiments, is considered to be implemented and practiced. Furthermore, while the embodiments disclosed herein may achieve advantages over other possible solutions or prior art, whether a specific advantage is achieved by a given embodiment does not limit the scope of this disclosure. Accordingly, the foregoing aspects, features, embodiments, and advantages are merely illustrative and are not to be construed as elements or limitations of the appended claims, except as expressly cited in the claims.

[0065] As will be understood by those skilled in the art, the embodiments disclosed herein may be implemented as systems, methods, or computer program products. Accordingly, the embodiments may take the form of an entire hardware embodiment, an entire software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware embodiments that may all generally be referred to as a “circuit,” “module,” or “system.” Additionally, the embodiments may take the form of a computer program product implemented on one or more computer-readable media containing computer-readable program code.

[0066] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any suitable combination thereof, but is not limited thereto. More specific examples (a non-limiting list) of computer-readable storage media include: electrical connections having one or more wires, portable computer diskettes, hard disks, RAM (Random Access Memory), ROM (Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory) or flash memory, optical fibers, portable CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this document, a computer-readable storage medium is any tangible medium capable of containing or storing a program for use by or in connection with an instruction execution system, device, or device.

[0067] A computer-readable signal medium may include a propagated data signal containing computer-readable program code, for example, in the baseband or as part of a carrier wave. Such propagated signals may take various forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium capable of communicating, propagating, or transmitting a program for use by or in connection with an instruction execution system, device, or device, which is not a computer-readable storage medium.

[0068] Program code implemented on a computer-readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wired, fiber optic cable, RF, etc., or any suitable combination thereof.

[0069] Computer program code for performing operations according to the embodiments of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code may be executed in its entirety on a user's computer, in part as a standalone software package on a user's computer, in part on a user's computer and in part on a remote computer, or in its entirety on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), and this connection may be made to an external computer (e.g., via the Internet using an Internet service provider).

[0070] Aspects of the present disclosure have been described with reference to flowchart examples and / or block diagrams of methods, apparatus (systems), and computer program products according to the embodiments presented in the present disclosure. It will be understood that each block of the flowchart examples and / or block diagrams, and combinations of blocks of the flowchart examples and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to create a machine, and such instructions are executed through a processor of the computer or other programmable data processing device to create means for implementing a function / operation specified in a block or in blocks of the flowcharts and / or block diagrams.

[0071] These computer program instructions may also be stored on a computer-readable medium to instruct a computer, other programmable data processing device, or other device to function in a specific manner, and the instructions stored on the computer-readable medium manufacture an article of manufacture that includes instructions for implementing a specified function / operation in a block or blocks of a flowchart and / or block diagram.

[0072] Computer program instructions can also be loaded into a computer, other programmable data processing device, or other device to enable a series of operational steps to be performed on the computer, other programmable device, or other device, and the instructions executed on the computer or other programmable device provide a computer implementation process to provide a process for implementing a function / operation specified in a block or blocks of a flowchart and / or block diagram.

[0073] The flowcharts and block diagrams in the drawings illustrate the architecture, function, and operation of possible implementations of systems, methods, and computer program products according to various examples of the present invention. In this regard, each block of a flowchart or block diagram may represent a module, segment, or part of an instruction containing one or more executable instructions for implementing specific logic function(s). In some alternative implementations, the functions mentioned in the blocks may occur differently from the order mentioned in the drawings. For example, two blocks shown consecutively may be executed substantially simultaneously, and blocks may sometimes be executed in reverse order depending on the related functions. Each block of the block diagram and / or flowchart examples, and combinations of blocks of the block diagram and / or flowchart examples, may be implemented by a special-purpose hardware-based system that performs a specified function or operation or performs a combination of special-purpose hardware and computer instructions.

[0074] The foregoing relates to specific examples, but other examples and additional examples may be devised without departing from the basic scope of the present disclosure, the scope of which is determined by the following claims.

Claims

Claim 1 A peripheral I / O device comprises a domain assist processor (DAP); a domain accelerator; and a gateway configured to communicately coupled the peripheral I / O device to a host using a coherent interconnect protocol, wherein the coherent interconnect protocol extends a coherent domain within the host to the peripheral I / O device such that the DAP becomes a peer processor to a processor within the host, and the gateway comprises an endpoint coherent interconnect coupled to both the DAP and the domain accelerator, wherein the gateway is configured to: receive a first request from the host instructing the DAP to perform a processor function using the coherent interconnect protocol; and receive a second request from the host instructing the domain accelerator to perform an accelerator task using the coherent interconnect protocol. Claim 2 In claim 1, the DAP comprises at least one processing core and an operating system, and the operating system is one of an embedded independent operating system and an extension of the operating system within the host, a peripheral I / O device. Claim 3 In paragraph 2, the instruction set architecture (ISA) used by the processing core is different from the ISA used by the processor within the host, and is a peripheral I / O device. Claim 4 In claim 1, a peripheral I / O device further comprising a programmable logic array, wherein at least a portion of the domain accelerator is executed by the programmable logic array. Claim 5 In paragraph 4, at least a portion of the domain accelerator is a peripheral I / O device running on hardened circuitry. Claim 6 A peripheral I / O device according to claim 1, further comprising a network on a chip (NoC) that enables communication between the gateway and the domain accelerator. Claim 7 A peripheral I / O device comprises a domain assist processor (DAP); a domain accelerator; and a gateway configured to enable the peripheral I / O device to communicate with a host using a coherent interconnect protocol, wherein the coherent interconnect protocol extends a coherent domain within the host to the peripheral I / O device such that the DAP becomes a peer processor to a processor within the host, and the gateway is a hybrid gateway configured to establish an I / O domain and the coherent domain at the peripheral I / O device, and the gateway is configured to: receive a first request from the host instructing the DAP to perform a processor function using the coherent interconnect protocol; and receive a second request from the host instructing the domain accelerator to perform an accelerator task using the coherent interconnect protocol. Claim 8 In claim 1, the gateway is a peripheral I / O device comprising a cache that is communically coupled between the endpoint coherent interconnect, the DAP, and the domain accelerator. Claim 9 A method performed by a computing system, comprising: identifying a processor function to offload from a CPU-memory complex within a host to a DAP within a peripheral I / O device— wherein a coherent domain including the CPU-memory complex of the host extends to a DAP within the peripheral I / O device, and the DAP includes at least one processing core and an operating system, wherein the operating system is one of an embedded standalone operating system and an extension of the operating system within the host, and the instruction set architecture (ISA) used by the processing core is the same as the ISA used by the CPU within the host—, using a coherent interconnect protocol, instructing the DAP to perform a processor function and store resulting data in a memory element within the coherent domain, using the coherent interconnect protocol, instructing a domain accelerator within the peripheral I / O device to process the resulting data when performing an accelerator task for the host, and the resulting data from performing the accelerator task in the domain accelerator A method including a receiving step. Claim 10 In claim 9, a method wherein at least a portion of the domain accelerator is executed by a programmable logic array. Claim 11 A method according to claim 9, wherein the DAP and the CPU within the host are part of the same cache-coherent shared-memory multiprocessor paradigm. Claim 12 A method according to claim 9, further comprising the step of using an I / O device driver within the host to command a compute resource assigned to an I / O domain within the peripheral I / O device to perform an I / O task. Claim 13 A computing system comprising: a host including a CPU-memory complex within a coherent domain; and peripheral I / O devices, wherein the peripheral I / O devices include: a DAP - the DAP includes at least one processing core and an operating system, the operating system is one of an embedded standalone operating system and an extension of an operating system within the host; and the ISA used by the processing core is different from the ISA used by the CPU within the host -; a domain accelerator; and a gateway configured to enable the peripheral I / O devices to communicate with the host using a coherent interconnect protocol, wherein the coherent interconnect protocol extends the coherent domain within the host to include the DAP, and the gateway is configured to receive a first request from the host instructing the DAP to perform a processor function; and to receive a second request from the host instructing the domain accelerator to perform an accelerator task. Claim 14 In paragraph 13, the peripheral I / O device further comprises a programmable logic array, and at least a portion of the domain accelerator is executed by the programmable logic array, a computing system. Claim 15 delete

Images