Hardware Accelerator Service Aggregation

Abstract

The present disclosure includes a system, method, and computer-readable medium for discovering capabilities of a local hardware (HW) accelerator card and a remote HW accelerator card. The local hardware (HW) accelerator card can provide a listing of acceleration services from the local HW accelerator card via a communication interface. The listing of acceleration services can include a first set of acceleration services provided by one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by one or more accelerators of the remote HW accelerator card. A workload instruction can be received from a processor of a computing device that defines a workload for processing by at least one of the acceleration services of the second set of acceleration services. The workload instruction can be forwarded to the remote HW accelerator card.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Patent Application No. 17 / 525,300, filed November 12, 2021, the disclosure of which is incorporated herein by reference. [Background technology]

[0002] background In most systems, it is difficult for software executing on a computing device, including a computing device component and / or an operating system, to discover functionality and capabilities provided by a hardware accelerator card connected to the computing device by a communication interface such as a PCIe bus. To avoid these problems, a processor may be hard-coded with software, such as a driver, to communicate with a particular hardware accelerator card. However, hard-coding a processor with the necessary software to communicate with a particular hardware accelerator card limits the processor to only that particular hardware accelerator card. Thus, the processor cannot take advantage of the functionality and capabilities of other hardware accelerator cards or hardware accelerator cards developed after the processor is produced.

[0003] Further, some hardware accelerator cards may expose their functionality and capabilities as separate devices within the operating system of the computing device. In this regard, when a hardware accelerator card is connected to a computing device by a communication interface such as a PCIe bus, the operating system may detect or otherwise be notified of the connection and may list each functionality and capability of the hardware accelerator card as a separate device within the operating system according to predefined classes and subclasses. Based on the devices listed within the operating system, the computing device may use the capabilities and functionality of the hardware accelerator card.

[0004] As the capabilities and functionality of hardware accelerator cards improve and become more specialized, these new capabilities and functionality are not clearly identified by the classes and subclasses provided by current operating systems. Thus, some operating systems can indicate the capabilities and functionality provided by a hardware accelerator card, but may not be able to identify all of the capabilities and functionality of the hardware accelerator card. Furthermore, some of the capabilities and functionality of the hardware accelerator card may not be recognized and / or clearly identified within the operating system. Thus, a computing device may not be able to take advantage of, or even recognize, all of the features and capabilities of an available hardware accelerator card.

[0005] A system is typically provided with a limited number of connections to a communication interface. For example, a system including a PCIe bus may only have a small number of PCIe slots for connecting hardware accelerator cards to the PCIe bus. The limited number of connections may be due to cost constraints. In this regard, each additional connection added to a system may increase the cost of physical hardware and increase the overall manufacturing cost. In addition, technical limitations such as power availability may also limit the number of devices that may be connected to a system. For example, a system may include five connections for hardware accelerator cards, but the power supply may only be capable of supplying power to two hardware accelerator cards simultaneously. Thus, the system may be limited in its ability to access acceleration services provided by the hardware accelerator cards due to the limited number of hardware accelerator cards that may be connected to the system. Summary of the Invention

[0006] overview The technology described herein relates to a system and method for service aggregation that aggregates and publishes acceleration services provided by accelerators of a hardware accelerator card. With service aggregation, a hardware accelerator card can communicate with other hardware accelerator cards to aggregate and publish acceleration services provided by accelerators of these other hardware accelerator cards that may be connected locally or remotely. The aggregated and published acceleration services may also include acceleration services provided by accelerators of the hardware accelerator card by performing service aggregation. The acceleration services provided by the hardware accelerator card as well as other locally or remotely connected hardware accelerator cards can then be leveraged by the system.

[0007] One aspect of the disclosure relates to a method that may include providing, by one or more processors of a local hardware (HW) accelerator card via a communication interface, a listing of acceleration services from the local HW accelerator card, the listing of acceleration services including a first set of acceleration services provided by the one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by the one or more accelerators of the remote HW accelerator card, receiving, by the one or more processors, a workload instruction from a processor of a computing device, the workload instruction defining a workload for processing by at least one of the acceleration services of the second set of acceleration services, and forwarding, by the one or more processors, the workload instruction to the remote HW accelerator card.

[0008] Another aspect of the present disclosure relates to a system comprising a communication interface and a local hardware (HW) accelerator card including one or more processors and one or more accelerators. The one or more processors may be configured to: receive a listing of acceleration services from the local HW accelerator card via the communication interface, the listing of acceleration services including a first set of acceleration services provided by the one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by the one or more accelerators of the remote HW accelerator card; receive workload instructions from a processor of a computing device, the workload instructions defining a workload for processing by at least one of the acceleration services of the second set of acceleration services; and forward the workload instructions to the remote HW accelerator card.

[0009] Another aspect of the disclosure relates to a non-transitory tangible computer-readable storage medium having computer-readable instructions of a program stored thereon, the instructions, when executed by one or more computing devices, causing the one or more computing devices to perform a method that can include providing, by one or more processors of a local hardware (HW) accelerator card, via a communication interface, a listing of acceleration services from the local HW accelerator card, the listing of acceleration services including a first set of acceleration services provided by the one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by the one or more accelerators of the remote HW accelerator card, receiving, by the one or more processors, workload instructions from a processor of the computing device, the workload instructions defining a workload for processing by at least one of the acceleration services of the second set of acceleration services, and forwarding, by the one or more processors, the workload instructions to the remote HW accelerator card.

[0010] In some examples, a processed workload may be received from a remote HW accelerator card, the processed workload being the workload after processing by at least one of the acceleration services of the second set of acceleration services.

[0011] In some cases, the processed workload may be transferred to a processor of a computing device. In some examples, the listing of accelerated services is generated by an accelerated services manager (ASM) running on one or more processors.

[0012] In some cases, an ASM running on one or more processors communicates with another ASM running on a remote HW accelerator card.

[0013] In some cases, an ASM may transfer workload instructions to another ASM. Optionally, the ASM requests a listing of a second set of acceleration services from another ASM.

[0014] In some cases, the ASM identifies and removes unhealthy acceleration services from the listing of acceleration services.

[0015] In some examples, identifying the unhealthy acceleration service includes determining, by the ASM, a failure of processing the workload instructions by at least one of the acceleration services of the second set of acceleration services.

[0016] In some examples, removing the unhealthy acceleration service includes marking at least one of the acceleration services of the second set of acceleration services as unhealthy or removing at least one of the acceleration services of the second set of acceleration services from a listing of acceleration services.

[0017] In some cases, The method is: and sending, by the ASM, the updated workload instructions to a different HW accelerator card for processing by at least one of the acceleration services of the different HW accelerator card after determining the failure of the processing of the workload instructions. Includes .

[0018] Optionally, the workload instructions further define processing by at least one acceleration service of at least one other remote HW accelerator card. [Brief description of the drawings]

[0019] [Figure 1]FIG. 1 is an illustration of an example system according to aspects of the present disclosure. [Diagram 2] FIG. 2 is another illustration of an example system according to aspects of the present disclosure. [Diagram 3] FIG. 2 is an illustration of an example listing of an acceleration service according to aspects of the present disclosure. [Figure 4] FIG. 1 is an illustration of an example networked system according to aspects of the present disclosure. [Diagram 5] FIG. 2 is another illustration of an example listing of an acceleration service according to aspects of the present disclosure. [Figure 6] 1 is a flow diagram of an example process for requesting acceleration services from a hardware accelerator card according to aspects of the present disclosure. [Figure 7] 4 is another flow diagram of an example process for requesting acceleration services from a hardware accelerator card and providing a workload that utilizes the acceleration services in accordance with aspects of the present disclosure. [Figure 8] 4 is a flow diagram of an example process for requesting acceleration services from a remote hardware accelerator card and a local hardware accelerator card according to aspects of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Detailed Description The technology described herein relates to a system and method for service aggregation that aggregates and exposes acceleration services provided by accelerators on hardware (HW) accelerator cards. Service aggregation is used to: calculationThe HW accelerator card installed in the device may provide and expose acceleration services from locally and remotely connected HW accelerator cards to the computing device. As described further herein, software executing on the computing core of the HW accelerator card in the computing device may discover or use the preconfiguration file to communicate with other HW accelerator cards locally connected to the computing device and / or other HW accelerator cards remotely networked to the computing device via a communication interface. The computing device may then communicate with other HW accelerator cards through the HW accelerator card installed in the computing device. Thus, the computing device may take advantage of the acceleration services of any number of locally or remotely connected HW accelerator cards.

[0021] To overcome the lack of discovering acceleration services, the techniques described herein use a standardized listing of identifiers corresponding to acceleration services that may be provided by accelerators on a HW accelerator card. In this regard, each HW accelerator card may store a listing of identifiers corresponding to acceleration services provided by accelerators on the card. Because the identifiers may provide finer granularity than the class and subclass of the device currently being used, a processor retrieving the listing from a HW accelerator card may be able to discover more of the identifiers that may be provided by accelerators on the HW accelerator card. Acceleration You will be able to decide and use the service.

[0022] As used herein, the term "acceleration services" refers to capabilities and functionality provided by an accelerator on a HW accelerator card. References to an "acceleration service" on a HW accelerator card refer to the acceleration services of the accelerator on that HW accelerator card. Acceleration services may include capabilities and functionality that an accelerator can leverage to control processing of data, referred to herein as control plane acceleration services. Acceleration services may also include capabilities and functionality that an accelerator can leverage to process data, referred to herein as data plane acceleration services. For example, an accelerator may support an acceleration service that provides control and / or policy for sharing memory between memory on the host (computing device) and the accelerator. This control plane acceleration service may be identified and communicated as an acceleration service.

[0023] Since each HW accelerator card can have many accelerators, each HW accelerator can provide many acceleration services with the same and / or different capabilities and functionalities. Furthermore, each accelerator can include multiple functions and capabilities.

[0024] Example System 1 illustrates an example of an architecture of a computing device 110 in which features described herein may be implemented. This example should not be considered as limiting the scope of the disclosure or usefulness of the features described herein. The computing device 110 may be a server, a personal computer, or other such system. The computing device 110 architecture includes a processor 112, a memory 114, and a hardware accelerator card 118.

[0025] The processor 112 may include one or more general purpose processors, such as a central processing unit (CPU), and / or one or more special purpose processors, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processor 112 may be of any type, including but not limited to one or more microprocessors (uP), one or more microcontrollers (uC), one or more digital signal processors, or any combination thereof. The processor may include one or more levels of caching, one or more processor cores, and one or more registers. Each processor core may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The processor 112 may be configured to execute computer readable program instructions, which may be included in a data storage device, such as instructions 117 stored in the memory 114, and / or other instructions described herein.

[0026] The memory 114 may store information accessible by the processor 112, including instructions 117 that may be executed by the processor 112. The memory may also include data 116 that may be retrieved, manipulated, or stored by the processor 112. The memory 114 may be any type of non-transitory computer-readable medium capable of storing information accessible by the processor 112, such as a hard drive, solid-state drive, tape drive, optical storage device, memory card, ROM, RAM, DVD, CD-ROM, writable memory, read-only memory, etc.

[0027] The instructions 117 may be a set of instructions, such as machine code, that are executed directly by the processor 112, or a set of instructions, such as a script, that are executed indirectly by the processor 112. In this regard, the terms "instructions," "steps," and "program" may be used interchangeably herein. The instructions 117 may be stored in object code format for direct processing by the processor 112, or in other types of computer languages, including scripts or collections of separate source code modules that are interpreted on demand or pre-compiled.

[0028] Data 116 may be obtained, stored, or modified by processor 112 in accordance with instructions 117 or other such instructions. For example, data 116 may be stored in computer registers in a distributed storage system as a structure having multiple different fields and records, or documents, or as a buffer, although the systems and methods are not limited by a particular data structure. Data 116 may also be formatted in a computer readable format, such as, but not limited to, binary values, ASCII or Unicode. Additionally, data 116 may include sufficient information to identify associated information, such as numbers, descriptive text, unique codes, pointers, references to data stored in other memory, including other network locations, or information used by a function to compute the associated data.

[0029] The computing device may further include a hardware (HW) accelerator card 118. The hardware accelerator card 118 may be any device configured to efficiently process a particular type of task. Some examples of HW accelerator cards include network accelerator cards, video transcoding accelerator cards, security function accelerator cards, cryptography accelerator cards, audio processing accelerator cards, artificial intelligence accelerator cards, etc. Each of these HW accelerator cards may be configured to provide a particular acceleration service, such as compression, encryption, transcoding, hash generation, graphics processing, simulation, etc. Some HW accelerator cards may be configured to provide multiple acceleration services, such as compression and encryption, or any other combination of acceleration services.

[0030] Computing device 110 may also include a network interface card 119. A network interface card may be any device capable of communicating directly and indirectly with other nodes of a network, such as network 470 described herein with reference to FIG.

[0031] Although FIG. 1 functionally describes the processor 112, memory 114, HW accelerator card 118, and network interface 119 as being in the same block, the processor 112, memory 114, HW accelerator card 118, and network interface 119 may or may not be stored in the same physical housing. For example, some of the instructions 117 and data 116 may be stored on a removable CD-ROM, and others in read-only DRAM chips. Some or all of the instructions and data may be stored in a location that is physically remote from the processor 112, but still accessible by the processor 112. Additionally, while FIG. 1 describes the computing device 110 as including only one processor 112, memory 114, network interface 119, and HW accelerator card 118, the computing device 110 may include any number of processors, memories, network interfaces, and HW accelerator cards. Similarly, the processor 112 may actually include a collection of processors, which may or may not operate in parallel.

[0032] 2, the HW accelerator card 118 may include a compute complex 212, a memory 214, and accelerators 228a, 228b, and 228c. The compute complex may include one or more compute units 213. The compute complex may control the general operation of other components of the hardware accelerator, for example, by distributing processing tasks to the accelerators 228a-228c, communicating with other devices within the computing device 110, such as the processor 112, etc. In some cases, the compute complex may coordinate or otherwise assist in service aggregation, as described herein.

[0033] The one or more computing units of the computing complex 212 may comprise one or more general-purpose and / or special-purpose processors. Typically, the computing units 213 of a hardware accelerator card may be one or more special-purpose processors, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs) capable of executing ARM or MIPS-based instruction sets, although other instruction sets may be used. In some cases, the computing units 213 may be commercially available processors.

[0034] Each of the accelerators 228a-228c may be comprised of one or more processors capable of providing a particular acceleration service. For example, each accelerator may be configured to provide a particular acceleration service, such as compression, encryption, transcoding, hash generation, graphics processing, simulation, etc. Some HW accelerator cards may be configured to provide multiple acceleration services, such as compression and encryption, or any other combination of acceleration services. The one or more processors of an accelerator may be one or more dedicated processors used, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a dedicated processor, etc. Although three accelerators, including the accelerators 228a-228c, are shown in FIG. 2, a HW accelerator card may include any number of accelerators. As previously described, each individual accelerator may be configured to provide multiple acceleration services (e.g., multiple functions and / or capabilities).

[0035] Referring again to FIG. 2, the HW accelerator card includes a memory 214. The memory 214 is configured to 112The memory 214 may be compared to the memory 114 in that it may be any type of non-transitory computer-readable medium capable of storing information accessible by the computing complex 212 and / or the accelerators 228a-228c, such as a hard drive, solid state drive, tape drive, optical storage device, memory card, ROM, RAM, DVD, CD-ROM, writable memory, read-only memory, etc. The memory 214 may store information accessible by the computing complex 212 and / or the accelerators 228a-228c, including instructions 217 that may be executed by the computing units 213 of the computing complex 212 and / or the accelerators 228a-228c. Although not shown, each accelerator 228-228c may have its own memory and / or a pool of shared memory for storing data and instructions for performing tasks assigned by the computing complex 212.

[0036] The instructions 217 may include an accelerated services manager (ASM) program 219. As described further herein, the ASM 219 may be executed by one or more computing units 213 of a computing complex to control or otherwise assist in service aggregation of the HW accelerator card 118.

[0037] The data 216 in the memory 214 may be obtained, stored, or modified by the computational complex 212 and / or the accelerators 228a-228c in accordance with the instructions 217 or other such instructions. As further shown in FIG. 2, the data 216 may include one or more acceleration service listings 218. The acceleration service listings 218 may include a list of acceleration services offered by each accelerator 228a-228c. The acceleration service listings 218 may be in a standardized form. In this regard, each particular acceleration service may be assigned a particular unique identifier. All accelerators that have an acceleration service will include a unique identifier associated with the particular acceleration service in their listing of acceleration services.

[0038] FIG. 3 illustrates example listings 328a-328c as stored in memory 214, where example listings 328a-328c correspond to accelerators 228a-228c, respectively. In this regard, memory 214 includes a listing for each accelerator on HW accelerator card 118. As shown, listing 328a identifies acceleration services provided by accelerator 228a, as identified by unique identifiers including function 1, capability 1, capability 3, and capability 5. Similarly, accelerator 228b may provide three acceleration services, each identified by a unique identifier in listing 328b, including capability 1, capability 5, and capability 9. Accelerator 228c may provide two acceleration services, each identified by a unique identifier in listing 328c, including capability 9 and capability 5.

[0039] As further illustrated in FIG. 3, accelerators that provide a common acceleration service may be associated with the same unique identifier within their respective listings for the accelerators. For example, function 1 is a unique identifier associated with a particular function that accelerators 228a and 228b can perform. Thus, listings 328a and 328b include function 1 with the same unique identifier. Similarly, capability 9 is a unique identifier associated with a particular capability of accelerators 228b and 228c. Thus, listings 328b and 328c include function 1 with the same unique identifier. Similarly, capability 9 is a unique identifier associated with a particular capability of accelerators 228b and 228c. Thus, listings 328c and 328d include function 1 with the same unique identifier. 8 c includes the same unique identifier of capability 9. The unique identifiers in Figure 3 are only examples of possible identifiers. The identifiers may include any value or other such indicators, including numbers, letters, symbols, etc.

[0040] Listings 328a-328c are examples of possible formats for listing unique identifiers associated with accelerators of HW accelerator card 118. In some examples, the listing of accelerators may be stored in a composite listing, such as a spreadsheet or database. For example, the composite listing may identify each accelerator and a unique identifier associated with the acceleration service provided by the accelerator. Similarly, the listings may be categorized according to accelerator. For example, a first listing may include a composite listing for a first set of accelerators, and a second listing may include a composite listing for a second set of accelerators. Other data may be included in the listings. FIGS. 2 and 3 show the listings stored in memory 21. 4 Although shown as being stored in the memory of one or more accelerators, the listing may be stored in the memory of one or more accelerators.

[0041] Although not shown, an administrator may maintain a repository of acceleration services and associated unique identifiers for the acceleration services. The administrator may be an individual(s), a company, a collection of companies, a standard organization(s), etc. In addition to maintaining the repository, the administrator may assign a unique identifier to each acceleration service and may also add additional acceleration services and corresponding unique identifiers as they are developed, received, or otherwise required. By providing a repository of acceleration services and associated unique identifiers, the identifiers used to indicate acceleration services may be consistent across multiple HW accelerator cards, even when the HW accelerator cards are manufactured by different suppliers.

[0042] Referring again to FIG. 2, the processor(s) 112 can communicate directly with the hardware accelerator card(s) 118 using a communication interface and protocol. For example, the processor(s) 112 can communicate with the hardware accelerator card(s) 118 using a PCIe interface 260. Although FIG. 2 shows a PCIe interface 260, other communication interfaces and protocols may be used. For example, the processor(s) 112 can communicate with the HW accelerator card(s) 118 using one or more of a CAN interface and protocol, an SPI interface and protocol, a USB interface and protocol, an eSPI interface and protocol, an Ethernet interface and protocol, an IDE interface and protocol, or other such interfaces and protocols.

[0043] Communications between devices via a communication interface, such as communications between processor 112 and HW accelerator card 118 via PCIe interface 260, may be controlled via an operating system running on computing device 110. In this regard, the operating system may set up handles to provide communication channels between devices attached to PCIe interface 260. In some cases, the operating system may also close communication channels between different devices connected to PCIe interface 260.

[0044] 1 or 2, computing device 110 may include other components typically found in a personal computer and / or server, such as a display device, e.g., a monitor having a screen, a projector, a touch screen, a small LCD screen, a television, or another device, such as an electrical device, that may be operable to display information processed by processor 112. Computing device 110 may also include speakers. Computing device 110 may also include one or more user input devices, such as a mouse, keyboard, touch screen, microphone, etc. Computing device 110 may also include hardware for connecting some or all of the aforementioned components together.

[0045] Figure 4 shows the network 4 4 shows a networked computing system 400 including computing devices 410, 420, 430, and 440 connected together via a network 70. 4 10~ 440 may each be compared to computing device 110. For clarity, FIG. 4 illustrates computing devices 410-440 as including HW accelerator cards 418, 428, 438, 448 and network interface cards 419, 429, 439, 449, respectively. However, each computing device 410-440 may include the same or different components as computing device 110, such as one or more processors, memory, HW accelerator cards, and / or network interfaces, as well as other components typically found in or otherwise connected to a computer.

[0046] Network 470 may include various protocols and systems such that the network may be part of the Internet, the World Wide Web, a particular intranet, a wide area network, or a local network. The network may utilize standard communication protocols such as Ethernet, WiFi, and HTTP, one or more proprietary protocols, as well as various combinations of the aforementioned protocols. Although certain advantages are obtained when information is transmitted or received as described above, other aspects of the subject matter described herein are not limited to any particular manner of transmission of information. Each computing device 410-440 may communicate with other computing devices via network 470. In some cases, network 470 may be configured to allow communication only between a subset of computing devices. For example, computing device 410 may be able to communicate with computing devices 420 and 430 via network 470, but not with computing device 440. Additionally, although FIG. 4 shows only four computing devices connected via network 470, a networked computing system may include any number of computing devices and networks.

[0047] FIG. 5 shows the hardware accelerators in the computing devices 410 to 440. card3, which identify the acceleration services provided by the accelerators of computing device 110, acceleration service listings 510-540 include capabilities and functionality provided by accelerators of HW accelerator cards 418-448 in computing devices 410-440, respectively. Each acceleration service provided by an accelerator of a HW accelerator card attached to a communication interface of the computing device is identified with a "(local)" label. For example, HW accelerator card 418 in computing device 410 may provide acceleration services for networking and compression. Similarly, HW accelerator card 428 in computing device 420 may provide networking and encoding acceleration services, HW accelerator card 438 in computing device 430 may provide networking and hashing acceleration services, and HW accelerator card 448 in computing device 440 may provide networking and encryption services.

[0048] Listings 510-540 also include accelerated services of other computing devices on network 470 identified through service aggregation. These remotely available accelerated services are identified in FIG. 5 with a "(remote)" label. For example, listing 510 lists accelerated services of computing device 4 Coding and calculation devices provided by HW accelerator cards in 20 4 Hashing and computation provided by HW accelerator cards in the 30 4 The encryption provided by the HW accelerator card in the 40 is 410 as the aggregated acceleration services available to it. Although FIG. 5 illustrates each computing device as being able to use all acceleration services offered by all connected computing devices, in some cases, acceleration services offered by remote devices may be blocked from remote access. For example, computing device 340 may prevent its local encryption acceleration services from being made available through service aggregation.

[0049] The acceleration service listing for each computing device may be generated by ASM software running on one or more HW accelerator cards. For example, during initialization of a HW accelerator card, such as HW accelerator card 418 of computing device 410, the computing complex of the HW accelerator card may execute the ASM. The ASM may prepare a listing of acceleration services that may be provided through the HW accelerator card, including local acceleration services and remote acceleration services. This acceleration service listing may be generated by an operating system running on computing device 410 or by the HW accelerator card 418, calculation This may be provided from a configuration file stored in memory on the device 410 or elsewhere.

[0050] In some cases, the acceleration service listings may be dynamically discovered. For example, an ASM executing on the HW accelerator card 418 may communicate with the HW accelerator card locally installed on the computing device 410 and / or other HW accelerator cards of the computing device connected to the network 470. During the communication, the HW accelerator card 418 may request a listing of acceleration services provided by other locally or remotely connected HW accelerators. The HW accelerator card 418 may aggregate these other acceleration services into an acceleration listing 510. In some cases, the HW accelerator card may maintain a separate acceleration listing for each other locally or remotely connected HW accelerator card. 5 shows only a single listing 510 for computing device 410, computing device 410 may have a listing for acceleration services provided by HW accelerator card 418 and separate listings for acceleration services provided by other remotely connected HW accelerator cards. While the above example describes discovery and collection of acceleration services by an ASM program running on HW accelerator card 418, other HW accelerator cards, such as accelerator cards 428, 438, and / or 448, may also provide listings of acceleration services available to their respective computing devices 420-440.

[0051] An ASM executing on the HW accelerator card 418 may manage the acceleration service listing. In this regard, the ASM may determine which acceleration services are healthy (e.g., operational, available to process workloads, have sufficient processing power for the workload, etc.). The ASM may monitor the operation of the acceleration services in the acceleration service listing to determine the state of the acceleration service (e.g., healthy, unhealthy / busy, unavailable, etc.). For example, the ASM may request status updates from other ASMs to determine the state of acceleration services provided by other HW accelerator cards. The ASM may prune the acceleration service listing to remove acceleration services identified by other ASMs as unavailable (e.g., unreachable). In another example, the ASM may mark accelerated services identified as being unhealthy (e.g., operating inefficiently / slowly) or busy (e.g., processing other workloads, reserved for other workloads, etc.) such that the ASM does not send workloads to these busy / unhealthy accelerated services.

[0052] In another example, the ASM can monitor the workload sent for completion or the response indicator. If no response is received or the workload completion is not identified by the ASM, the ASM can determine that the workload was not received by the remote HW accelerator and / or was not processed by the intended acceleration service provided by the remote HW accelerator. In such a case, the ASM can request another acceleration service to execute the workload.

[0053] In some cases, the ASM can distribute a workload to multiple acceleration services for load balancing, failure handling, performance or other such considerations. For example, a workload may be large and the ASM can leverage the acceleration services of many HW accelerator cards to efficiently process the workload. In another example, the ASM can prioritize several remote HW accelerator cards. For example, the ASM can direct a workload to be processed by a first remote HW accelerator. If the first remote HW accelerator cannot process the workload, a second remote HW accelerator can process the workload. This process can be repeated until the workload is processed, with additional fallback HW accelerators being directed to process the workload.

[0054] As part of the initialization of the HW accelerator card, the ASM executing on the compute complex may prepare the service to send and receive service calls both locally and remotely. In this regard, the ASM may initialize and / or ensure that end service code is enabled to process local service calls (e.g., calls to be processed by the accelerator of the HW accelerator card on which the ASM is executing).

[0055] An ASM may also initialize and / or verify that its proxy code for a remote service is enabled. The proxy code may be used when an intermediary for invoking a remote service (e.g., a call sent from an ASM to another ASM for processing by an accelerator on another HW accelerator card) is enabled. In this regard, proxy code may refer to code that enables ASM-to-ASM communication. Proxy code may also refer to code that enables an ASM on a local HW accelerator card to invoke acceleration services from accelerators on other remote HW accelerator cards that may not have an ASM running for them. That is, the ASM with proxy code on a local HW accelerator card may be configured to route requests for acceleration services from the local computing device to a remote HW accelerator card when the remote HW accelerator card does not include an ASM.

[0056] The ASM can publish the acceleration service listing to a processor of the computing device. For example, the ASM executing on the HW accelerator card 418 can provide the listing 510 to one or more processors of the computing device 410. In some cases, an operating system executing on the computing device can request the listing from the ASM. For example, the operating system executing on the computing device 410 can request the listing 510 of the acceleration services from the HW accelerator card 418.

[0057] Example of how to 6 is a flow diagram illustrating a process for discovering acceleration services provided by a HW accelerator card, such as HW accelerator card 118, connected to a processor, such as processor 112, via a communication interface, such as PCIe bus 260. Processor 112 may request to communicate with HW accelerator card 118 (shown in dashed lines) via the PCIe interface. An operating system executing on the computing device may provide a communication channel via the PCIe bus between the HW accelerator card and processor 112.

[0058] Using the communication channel, the processor 112 can send a request for a listing of acceleration services provided by accelerators on the HW accelerator card 118, as indicated by line 623. In response to receiving the request from the processor 118, the computational complex 212 of the HW accelerator card 118 can query and receive a listing of acceleration services from the memory 214 of the HW accelerator card (or the memory of the accelerator), as indicated by arrows 625 and 627, respectively. In this regard, the HW accelerator card can aggregate the acceleration services of all accelerators. In some cases, the HW accelerator card 118 can query only some accelerators.

[0059] In some cases, the HW accelerator card 118 may aggregate the accelerator's acceleration services in a hierarchical manner. In this regard, the acceleration services may be hierarchical in that one acceleration service may rely on or depend on another acceleration service. This hierarchical relationship between the acceleration services may be identified and stored in this listing. In some cases, each level in the hierarchical relationship may identify the capabilities and functionality of the level below.

[0060] The compute complex 212 may provide the listing of the acceleration service to the processor 112 via the PCIe bus 260, as shown by line 629. Once the processor receives the listing of the acceleration service, the communication channel may be closed.

[0061] If the processor can leverage one or more acceleration services, the processor 112 can request the HW accelerator card to complete one or more tasks using one of the offered acceleration services provided by the accelerator on the HW accelerator card 118. Figure 7 illustrates the processor 112 requesting information about the acceleration services of the HW accelerator card 118 connected via the PCIe bus 260. In this regard, steps 723-729 correspond to steps 623-629 described above.

[0062] As shown by arrow 729, the HW accelerator indicates that the HW accelerator can provide compression services. Upon receiving the acceleration service, the processor 112 can provide the HW accelerator card 118 with workload instructions, including instructions for where to store the data and instructions, to compress the data, as shown by arrow 731. The HW accelerator card's computation complex 212 can then verify the instructions, as shown by arrow 733, and provide an ID with which the processor 112 can communicate to get status update information about the compression by the HW accelerator card 118. The processor 112 can then request and receive the status of the compression, as shown by arrows 735 and 737, respectively. Once the polling request indicates that the compression is complete, the communication between the processor 112 and the HW accelerator card 118 can end, or further tasks can be sent from the processor 112 to the HW accelerator card. Although FIG. 7 shows a compression service, the processing performed by the HW accelerator can be any type of operation or combination of operations.

[0063] 8 shows a flow diagram of a service aggregation operation. Software running on a processor 812 of a computing device uses HW Accelerators One or more acceleration services may be requested from the card 818. The request, as indicated by line 823, may be sent by the processor 812 to the local HW accelerator card 818 via a communication link established by a communication interface, such as the PCE interface 860. For purposes of illustration, the acceleration service requested by the processor 812 is a "compression" service for processing a workload. However, any other acceleration service may be requested.

[0064] Prior to sending the request 823, software executing on the processor 812 may be provided with a listing of acceleration services available locally (i.e., by the accelerator of the HW accelerator card 818) or remotely (i.e., by other remote accelerators of the HW accelerator connected via a network) from the HW accelerator card 818. For example, this listing may be provided during initialization of the HW accelerator card 818. In another example, software such as an operating system or another program executing on the computing device may request the listing. In yet another example, the listing of acceleration services may be received by the software from a configuration system without communication with the HW accelerator card 118. The configuration system may be a centralized listing from which the software executing on the computing device may obtain the listing. For example, the configuration system may be located remotely, with storage accessible via a network or other connection.

[0065] The request 823 may include a name (e.g., a unique identifier) ​​of the acceleration service being requested, inline parameters, and pinned memory location(s) where input (e.g., data for processing a workload) for the requested acceleration service(s) can be found.

[0066] After receiving the request, the ASM of the HW accelerator card 818 can determine whether the requested acceleration service is local, as shown by line 825. If the acceleration service is local, the ASM can service the request using the inline parameters and access the passed memory address using DMA (direct memory access). A local accelerator capable of performing the requested acceleration service can then process the input (e.g., data). Referring again to FIG. 8, the requested compression acceleration service can be performed by an accelerator of the HW accelerator card 818. The results (e.g., output) of the completed compression request can then be passed from the HW accelerator card 818 to the processor 812, as shown by line 827.

[0067] If the requested acceleration service is not available locally, the ASM can act as a client of the request to a remote service HW accelerator card. In this regard, the ASM of the HW accelerator card can pass the request to the remote HW accelerator card that provides the requested acceleration service. The HW accelerator card can also pass the pinned memory to the remote HW accelerator card via remote direct memory access (RDMA).

[0068] 8 further illustrates the flow of a remote service aggregation operation. For example, the processor 812 may request an encryption acceleration service provided by a remote HW accelerator card 838 from the local HW accelerator card 818, as shown by line 829. After receiving the request, the local HW accelerator card 818 may determine, as shown by line 831, that encryption is not an acceleration service that the card provides.

[0069] When the local HW accelerator card cannot complete the acceleration service, the ASM of the local HW accelerator card 818 can determine which remote HW accelerator card can provide the service. card The ASM of 818 may determine that a remote HW accelerator card 838 can provide cryptographic acceleration services. The ASM may forward the request, as shown by line 833.

[0070] In some cases, the request may include an identifier of a HW accelerator card that can provide the acceleration service. In such a case, the local HW accelerator card 818 may skip determining whether it performs the requested acceleration service and proceed to forward the request to the identified remote HW accelerator card.

[0071] The remote HW accelerator card 838 may then perform encryption acceleration services, as shown in Figure 8 by line 835. The result (e.g., output) of the completed encryption may then be passed from the remote HW accelerator card 838 to the local HW accelerator card 818, as shown by line 837, and from the local HW accelerator card 818 to the processor 812, as shown by line 839.

[0072] Communication between HW accelerator cards can be accomplished by ASM programs executing on each HW accelerator card. In this regard, each ASM can communicate directly. Alternatively, the ASM can encapsulate communication data (e.g., requests, outputs, etc.) into data packets in outer headers that can route the packet to a remote routable destination, the address of which can be discovered via regular routing. When a packet reaches its intended ASM, the ASM can decapsulate the packet and direct it to the correct accelerator on that HW accelerator card, as identified in the packet header.

[0073] Unless otherwise stated, the aforementioned alternatives are not mutually exclusive, but rather may be implemented in various combinations to achieve unique advantages. These and other variations and combinations of the above-mentioned features may be utilized without departing from the subject matter defined by the claims, and therefore the foregoing description of the embodiments should be taken as illustrative, and not as limiting, of the subject matter defined by the claims. In addition, the provision of examples described herein, as well as clauses that use phrases such as, including, and the like, should not be construed as limiting the subject matter of the claims to any particular example, but rather, the example is intended to illustrate only one of many possible embodiments. Furthermore, the same reference numbers in different drawings may identify the same or similar elements.

Claims

1. providing, by one or more processors of a local hardware (HW) accelerator card via a communications interface, a listing of acceleration services from the local HW accelerator card, the listing of acceleration services including a first set of acceleration services provided by one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by one or more accelerators of a remote HW accelerator card; receiving, by the one or more processors, a workload instruction from a processor of a computing device, the workload instruction defining a workload for processing by at least one of the acceleration services of the second set of acceleration services; forwarding, by the one or more processors, the workload instructions to the remote HW accelerator card; A method comprising:

2. receiving, by the one or more processors, a processed workload from the remote HW accelerator card, the processed workload being the workload after processing by the at least one of the acceleration services of the second set of acceleration services; The method of claim 1 further comprising:

3. transferring, by the one or more processors, the processed workload to the processor of the computing device. The method of claim 2 , further comprising:

4. The method of claim 1 , wherein the listing of accelerated services is generated by an Accelerated Services Manager (ASM) executing on the one or more processors.

5. The method of claim 4 , wherein the ASM executing on the one or more processors communicates with another ASM executing on the remote HW accelerator card.

6. The method of claim 5 , wherein the communication includes the ASM forwarding the workload instructions to the other ASM.

7. The method of claim 5 , wherein the communication includes the ASM requesting a listing of the second set of acceleration services from the other ASM.

8. The method of claim 5 , wherein the ASM identifies and removes unhealthy acceleration services from the listing of acceleration services.

9. 10. The method of claim 8, wherein identifying the unhealthy acceleration service includes determining, by the ASM, a failure of processing the workload instruction by the at least one of the acceleration services of the second set of acceleration services.

10. Removing the unhealthy acceleration service, marking the at least one of the acceleration services of the second set of acceleration services as unhealthy; or 10. The method of claim 9, further comprising removing the at least one of the acceleration services of the second set of acceleration services from the listing of acceleration services.

11. sending, by the ASM after determining the failure to process the workload instruction, an updated workload instruction to a different HW accelerator card for processing by at least one of the acceleration services of the different HW accelerator. The method of claim 10 further comprising:

12. The method of claim 1 , wherein the workload instructions further define processing by at least one acceleration service of at least one other remote HW accelerator card.

13. A communication interface; A local hardware (HW) accelerator card including one or more processors and one or more accelerators, the one or more processors comprising: receiving, via the communication interface, from the local HW accelerator card, a listing of acceleration services, the listing of acceleration services including a first set of acceleration services provided by one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by one or more accelerators of a remote HW accelerator card; receiving a workload instruction from a processor of a computing device, the workload instruction defining a workload for processing by at least one of the acceleration services of the second set of acceleration services; and transferring the workload instructions to the remote HW accelerator card. A system comprising:

14. the one or more processors: receiving a processed workload from the remote HW accelerator card, the processed workload being the workload after processing by the at least one of the acceleration services of the second set of acceleration services; The system of claim 13 , further configured to:

15. the one or more processors transferring the processed workload to the processor of the computing device. The system of claim 14 , further configured to:

16. The system of any one of claims 13 to 15, wherein the listing of accelerated services is generated by an Accelerated Services Manager (ASM) executing on the one or more processors.

17. 20. The system of claim 16, wherein the ASM executing on the one or more processors communicates with another ASM executing on the remote HW accelerator card.

18. 20. The system of claim 17, wherein the communication includes the ASM forwarding the workload instructions to the other ASM.

19. 20. The system of claim 17, wherein the communication includes the ASM requesting a listing of the second set of acceleration services from the other ASM.

20. 20. The system of claim 18, wherein the ASM identifies and removes unhealthy acceleration services from the listing of acceleration services.

21. A program causing one or more processors of a local hardware (HW) accelerator card to carry out the method according to any one of claims 1 to 12.

Images