Techniques for reducing accelerator-memory access costs in a platform using multiple memory channels

Abstract

Techniques for reducing accelerator-memory access costs in a platform using multiple memory channels. An apparatus includes a computing platform having multiple accelerators and multiple storage devices accessed via multiple memory channels. A job is submitted via software running on the computing platform to access a function to be offloaded to the accelerators. Under the offloaded function, the accelerators access one or more buffers that all require access via multiple memory channels among the multiple memory channels. The accelerators are identified that have available instances of the function, and a total cost is computed for each of the accelerators of accessing the one or more buffers via the multiple memory channels. The accelerator with the least total cost is then selected to offload the function. New instruction set architecture (ISA) instructions are also disclosed for identifying memory pages and memory channels for buffers.

Description

BACKGROUND

[0001] In recent years, data analytics and "big data" processing have become increasingly important. Data analytics and big data workloads require processing large amounts of data. One approach to processing such large amounts of data is to distribute processing tasks across a large number of servers and process the workloads in parallel. For example, the Apache Hadoop software framework enables distribution of tasks across a large number of commodity servers, and the use of MapReduce to process the workloads. While Hadoop and MapReduce provide excellent scalability, they require a very large amount of intra-server communication (when implemented at a large scale), and do not efficiently use processor and memory resources.

[0002] Some compute and memory bandwidth intensive workloads, such as for data analytics and big data, are difficult to achieve the required performance levels using processor cores. To address this problem, so-called "accelerators" have been developed. Accelerators were initially implemented as components coupled to CPUs (central processing units) and managed as IO (input output) devices with their own address space, which required a significant level of IO communication to transfer data between the accelerator address space and applications running in the system memory address space. More recently, CPUs employing a system-on-chip (SoC) architecture with embedded accelerators have been proposed. BRIEF DESCRIPTION OF DRAWINGS

[0003] The foregoing aspects and many of the attendant advantages of this application will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:

[0004] FIG. 1 is a schematic diagram of a first multi-socket non-uniform memory access (NUMA) platform architecture including a pair of processors with two on-chip accelerators;

[0005] FIG. 2 is a schematic diagram of a second multi-socket NUMA platform architecture in which each NUMA node includes a processor coupled to two off-chip accelerators;

[0006] Figure 3 is a flowchart illustrating operations performed to select an accelerator for offloading a function according to one embodiment;

[0007] Figure 4 is a flowchart illustrating operations for determining a total cost of an accelerator accessing one or more buffers from available instances of the platform having a function to be offloaded;

[0008] Figure 5is an example cost matrix;

[0009] Figure 6 is a simplified NUMA platform architecture, illustrating various costs of accessing corresponding components including accelerators and memory devices;

[0010] Figure 7 is a schematic block diagram illustrating an example of an Arm-based microarchitecture suitable for implementing the instruction set architecture (ISA) instructions disclosed herein; and

[0011] Figure 8 is a schematic diagram illustrating further details of an example NUMA node, in accordance with one embodiment. DETAILED DESCRIPTION

[0012] Embodiments of methods and apparatuses for reducing accelerator-memory access costs in a platform using multiple memory channels are described herein. In the following description, numerous specific details are set forth to provide a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations have not been shown or described in detail to avoid obscuring aspects of the application.

[0013] Throughout this specification, reference can be made to "one embodiment" or "an embodiment." This does not necessarily refer to the same embodiment, although it may. Different embodiments of the application can be described or claimed in different dependent claims. Furthermore, the terms "comprises an," "comprises," or "comprising," and the like, can be used herein. This, and similar terms, are intended to be interpreted as specifying examples of something within the terms. This and similar terms do not necessarily exclude other similar items that can also be used, in

[0014] For clarity, various components in the drawings can be labeled with labels other than specific reference numerals. Furthermore, reference numerals that are followed by an asterisk (typ) are intended to indicate reference numerals that refer to components of a particular type (as opposed to specific components). It will be understood that the configuration of these components will be typical of the following components: similar components that can be present but are not shown in the drawings for purposes of simplicity and clarity; or otherwise similar components that are not labeled with a separate reference numeral. Conversely, (typ) is not to be interpreted to mean that the component, element, etc. is typical for its disclosed function, implementation, purpose, etc.

[0015] Workloads such as data analytics and large mathematical processing often need to process large amounts of data from memory and present interesting challenges for efficient accelerator design. Accelerator capabilities are steadily increasing, and one of the most notable recent trends is accelerators with "shared virtual memory" (SVM) capability. Traditional accelerators need to be managed as IO (input output) devices in their own private address space; this is implemented using an expensive kernel mode driver (KMD) that requires applications to cross back and forth between user and kernel space, pin pages in memory, or copy user buffers to and from dedicated buffers managed by the OS / kernel mode driver. With SVM, an accelerator or IO device can work directly on the address space of a user application thread, as it shares the same virtual -> physical address translation capabilities as a CPU (central processing unit, aka processor) thread. This is a key improvement in accelerator efficiency, among other performance benefits, enabling user mode submission directly to the accelerator via a "user mode driver" or UMD.

[0016] However, SVM also presents some challenges for applications that require many acceleration engines to operate in a data streaming fashion at very high bandwidth. When an application assigns a job to an accelerator, it allocates input data buffers and output data buffers to the accelerator in virtual memory space. When using multiple accelerators in a multi-socket platform, these input and output data buffers can include physical memory pages spread across different sockets, and spread across many memory channels (e.g., 6 to 8) within each socket. Since these accelerators operate as high bandwidth streaming engines, depending on which engine is working on the job, can result in large differences in performance and amount of data streaming through all internal structures of the CPU SoC (system on a chip) and intra-socket links.

[0017] This problem is exacerbated under a multi-socket non-uniform memory access (NUMA) architecture, such as the architecture shown by NUMA architecture 100 in FIG. 1. NUMA architecture 100 includes two NUMA nodes 102a and 102b, also labeled 'A' and 'B'. For simplicity and convenience, references to nodes herein can be references to their letter labels rather than their reference numbers. Also, references to components that are common across nodes will be references to the base reference number, with the letters 'a', 'b' added to the reference numbers in the figures to identify the component on a particular NUMA node. For example, each of nodes A and B includes a processor 104, with the processor of node A labeled 104a and the processor of node B labeled 104b.

[0018] Each processor 104 has a core portion that includes multiple processor cores 106, each of which includes local level 1 (LI) and level 2 (L2) caches (depicted as LI / L2 caches 108). The remaining portion of the processor is referred to as the non-core, and includes various interconnect circuitry and interfaces for connecting the various functional blocks in communication on the processor. For simplicity, this interconnect circuitry is depicted as interconnect 110; however, it will be understood that the interconnect 110 can represent one or more interconnect structures, such as a bus, and single-lane or multi-lane serial point-to-point or mesh interconnect structures. Moreover, the interconnect structures can be implemented in a hierarchy, and / or employ different protocols and structures, with bridges used to interface these different protocols and structures.

[0019] A number of components are coupled to the interconnect 110, including a level 3 (L3) cache 112 (also referred to as a last level cache or LLC), a socket-to-socket interface 114, a pair of accelerators as depicted by accelerators 116, 118, 120, and 122 (also described or referred to as accelerator 1, accelerator 2, accelerator 3, and accelerator 4), and memory controllers (MCs) 124. Accelerators 1 and 2 are further connected by a link 117, while accelerators 120 and 122 are further connected by a link 119. In one embodiment, one or both of the accelerators comprises a field programmable gate array (FPGA) that includes programmable circuitry embedded on the processor die. The accelerators can also include pre-defined circuitry embedded on the processor die. In addition to these illustrated components, each processor 104 can include many other functional blocks that are not shown for clarity.

[0020] Each of the memory controllers 124a and 124b includes one or more channels connected to one or more DRAM (dynamic random access memory) devices 126, such as dual in-line memory modules (DIMMs) 128. In Figure 1, these DRAM memory devices are depicted as DRAMs 1-4 for node A and DRAMs 5-8 for node B. In the illustrated embodiment, a respective memory channel (as indicated by Ch. 1, Ch. 2, Ch. 3, etc., where “Ch.” means channel) is connected to each DRAM memory device 126. However, this is merely exemplary, as a memory controller can have more than one memory channel connected to the same memory device.

[0021] Each of the DRAM memory devices 126 has a physical address space. Generally, the physical address space is divided into "page" units, which are further divided into cache line units, although other addressing schemes can be used. The physical address space of the DRAM memory devices is mapped to a virtual address space, as shown by virtual address spaces 128a and 128b. The virtual address space is typically divided into a plurality of virtual memory "pages," and is accessed at the page level. Mapping of the virtual and physical address spaces is generally handled by the memory controllers and / or other platform components (not shown). Depending on the particular platform, the operating system can provide further aspects of this mapping.

[0022] In one embodiment, each of the processors 104 is operatively coupled to a printed circuit board referred to as a motherboard 132 via a socket, or otherwise coupled to the motherboard via direct coupling techniques such as flip-chip bonding. In either case, it is common practice to refer to the processor itself as the socket, and thus the term socket-to-socket link and interface is used for coupling the processor nodes in communication. The motherboard 132 includes electrical wiring (e.g., traces and vias) for facilitating electrical connections corresponding to the physical structures of the various interconnections depicted in FIG. 1. These interconnections include a socket-to-socket link 134 coupled between the socket-to-socket interfaces 114a and 114b. In one embodiment, the socket-to-socket interfaces 114a and 114b employ the QuickPath Interconnect (also referred to as the QuickPath Interconnect (QuickPath Interconnect (QuickPath Interconnect

[0023] Under the NUMA architecture, processors (and processor cores) are able to access different memory resources distributed across the platform. The memory resources can be viewed as local memory resources (e.g., memory resources on the same node as the processor or core) or non-local memory resources (e.g., memory resources on other nodes). For example, from the perspective of the node 102a, each of the DRAMs 1-4 comprises a local memory resource, and each of the DRAMs 5-8 comprises a non-local memory resource. Under another type of NUMA architecture (not depicted herein), non-local memory resources can also be shared among processors, and not associated with a particular processor or node. Thus, for a given node, the local memory resources are operatively coupled to the processor, and the non-local memory resources are not, and access to the local memory resources is not the same as access to the non-local memory resources (e.g., access is non-uniform). Moreover, it is preferable to employ the local memory resources when they are available.

[0024] Under NUMA architecture 100, software running on a processor core in one node can be allocated memory address space in system memory nominally associated with another node. Thus, under some embodiments, address space for a portion of the memory resources in a platform can be managed as a single global pool, with a unique address for each memory address location within the pool. Further, a portion of the address space can be treated as local address space (e.g., address space allocated to an operating system) that employs local addressing, while another portion can be addressed using global addressing.

[0025] In addition to on-chip accelerators (meaning as part of the SoC die on the processor SoC), accelerators can be separate off-chip components that are communicatively coupled with the processor. An example of this is shown by NUMA architecture 200 in FIG. 2, which includes nodes 202a and 202b with respective processors 204a and 204b. Components in NUMA architectures 100 and 200 with like reference numbers are similar in the two embodiments. Further, it will be appreciated that the NUMA architecture employs a similar virtual memory mapping scheme as NUMA architecture 100 (not shown in FIG. 2 due to lack of space).

[0026] To facilitate communication between the processor cores, accelerators, and memory, the processor will have one or more input-output (IO) interfaces. Under the illustrated embodiment, the IO interfaces are implemented using Peripheral Component Interconnect Express (PCIe) components, which are depicted as PCIe root complexes 202a and 202b and PCIe interfaces 204, 206, 208, and 210. On node 102a, PCIe interface 204 is coupled to accelerator 212 via PCIe link 214, while PCIe interface 206 is coupled to accelerator 216 via PCIe link 218. On node 102b, PCIe interface 208 is coupled to accelerator 220 via PCIe link 222, while PCIe interface 210 is coupled to accelerator 224 via PCIe link 226. PCIe supports various lane widths and speeds, including lx, 2x, 4x, 8x, and 16x configurations, where 'x' represents the number of lanes. Existing and future PCIe links and interfaces can be used for the PCIe links and interfaces described in FIG. 2 and discussed herein.

[0027] In general, off-chip accelerators can include chips (e.g., FPGA chips) that are populated to a motherboard, or can reside on accelerator boards or cards that are installed in PCIe expansion slots. There can also be multiple accelerator chips on the same board or card.

[0028] In addition to PCIe IO infrastructure, other IO technologies can also be employed in a similar manner. These include, but are not limited to OmniPath and RapidIO interconnect, Open Systems Interconnect protocols, and Advanced Microcontroller Bus Architecture (AMBA) bus protocols. Further examples of IO infrastructure that can be implemented are described below with reference to Figure 8

[0029] Under aspects of the embodiments described herein, workloads for stream accelerator jobs are spread across multiple accelerators spread across multiple sockets, and many locations within each socket (e.g., many locations in a CPU die, and / or on an attached MCP (multi-processor compute) die). When a new job is submitted, accelerators with available instances of functionality to perform one or more tasks are identified. Memory channels to access memory buffers for the job are also identified, and a total cost of accessing the memory buffers via the memory channels is computed for each accelerator with available instances of functionality. An accelerator with the lowest total cost is then selected for the functionality.

[0030] Figure 3 The flowchart 300 in FIG. 3 illustrates an overview of one embodiment of a process of selecting an accelerator to perform and offload functionality. In block 302, a job is submitted to access functionality to be offloaded to an accelerator. As described in further detail below, in one embodiment, the job is submitted via a driver, and includes information identifying one or more buffers to be used by the functionality. Depending on the allocation of memory pages for the buffers (e.g., by an operating system running on the platform), access to the one or more buffers can require use of multiple memory channels.

[0031] In block 304, accelerators with available instances of the functionality are identified. As shown in block 306, for each of the accelerators identified with available instances of the functionality, a total cost of accessing the one or more buffers via multiple memory channels is computed. The process is completed in block 308, where an accelerator with the minimum total cost is selected to offload the functionality onto.

[0032] Figure 4 ​The flowchart 400 in FIG. 4 illustrates further operations for implementing one embodiment of the process. In block 402, the interconnect topology between accelerators on the platform and the memory channels is determined. In one embodiment, during the initialization phase, when accelerator devices are discovered and enumerated, they also record their "location" relative to the memory subsystem under the given system topology. In conjunction with this initialization phase, a cost matrix is constructed, where the rows are the different accelerator instances, and the columns are the memory channels in the system. Each cell in the matrix is populated with the "cost" of engine (e.g., accelerator) x to read / write (i.e., access) data to channel y. Generally, we can expect that NUMA remote channels will be assigned a higher cost (for a given instance) compared to local channels. The construction of the cost matrix is illustrated in block 404.

[0033] In block 406, the number of memory pages associated with the one or more buffers accessed by each memory channel is determined. The cost of accessing memory pages via this memory channel is then computed in block 408 by multiplying the cost of the accelerator to access the memory channel (in the cost matrix) by the number of memory pages associated with the one or more memory buffers accessed via this memory channel.

[0034] In one embodiment, when a CPU thread submits a job to the accelerator pool, the driver (which can be user-mode) performs its customary checks to find available instances, also computing the cost of each available instance based on the cost matrix. This is done by obtaining the list of physical pages for each data buffer in the command descriptor and its memory channel, and summing them to obtain a list of the number of pages for each channel that this job will reference (called L). In one embodiment, this is aided by a new instruction set architecture (ISA) instruction for the processor, as described below. For each available engine (e.g., accelerator), select the row Ri in the cost matrix, and compute the dot product Ri*L. The engine with the lowest dot product cost is then chosen for use with the CPU thread.

[0035] Constructing the cost matrix

[0036] As discussed above, in one embodiment, a cost matrix is used that defines the cost for using various combinations of accelerators and memory channels. To establish the matrix, a cost is assigned from each accelerator instance to each memory channel in the system. This will generally be a function of the topology of the platform, as described in further detail below. In one embodiment, the cost will take into account the interconnect latency from the accelerator to the channel, as well as factors in the way (e.g., across a congested inter-socket link, or accessing an intermediary node that can share bandwidth with other valuable resources such as a PCIe link or network interface controller (NIC)). In general, various types of heuristics can be used to calculate the cost, but the simple idea is to assign a cost to different aspects of the system bandwidth that will be affected when a memory reference is made from an accelerator to this memory channel.

[0037] Figure 5 An example of a cost matrix 500 for the simplified platform architecture 600 shown in Figure 6 An example of a cost matrix 500 for the simplified platform architecture 600 shown in

[0038] New ISA instruction Get_phys_mem_from_VA

[0039] Listing 1 shows pseudo code for implementing a new instruction called Get_phys_mem_from_VA that can be added to the instruction set architecture (ISA) of the processor illustrated herein.

[0040] LISTING 1 (Listing 1)

[0041]

[0042] (Listing 1

[0043] )

[0045] Basically, the Get_phys_mem_from_VA instruction takes a data buffer in virtual space and provides it with a physical page table (address as well as associated information such as exact memory channel / controller). The instruction includes three parameters (operands) dest, src, and len, which are stored in the first and second m64 64-bit registers and the r64 64-bit register, respectively. The m64 notation means that the operand is a pointer to memory, so its content can be used in load / store instructions. The r64 notation means that the operand is used as a data value. len is the length of the data buffer (in bytes) while src contains a 64-bit pointer to the data buffer. In C-like notation, src[i] would be the address of the ith byte of src (also the same as src+i). The core hardware will execute the function lookup-TLB-for-virtual-address, so we can get the physical address corresponding to the virtual page table for the byte src[i]. The loop increments i by 4K (i.e., the page size) because bytes that are apart by a page or a page apart by 4K will map to the current page.

[0046] The m64 dest register contains a pointer to a memory buffer that includes an array of structs (dest[i / 4K]), the fields of which will be {PA (physical address), channel#,... other attributes}. Get-physical-memory-attributes (PA) is a hardware circuit in the processor similar to the TLB lookup logic that takes a physical address and broadcasts it like a virtual "load", but the non-core / memory controller drops the "load" and simply returns an ID that is the ID of the device that would have serviced the load if the "load" were a real load.

[0047] Job submission

[0048] Suppose we have a function that is offloaded to the accelerator that adds two arrays of numbers. The descriptor will have 2 input buffers of the same size S, and an output buffer of size S. This is a very simple example, however real usage will have many combinations of input / output buffers, and each buffer has its own size.

[0049] When the job is submitted to the driver, the driver code examines the descriptor and looks for a list of available instances of the function. It also calls the aforementioned Get_phys_mem_from_VA instruction to get a per-buffer list of channels used. Note that the Get_phys_mem_from_VA instruction will be called 3 times, ignoring the PA result, because at this point we are only interested in getting the channel#. For example, we get a table of channels as shown in Table 1.

[0050] Table 1

[0051] Buffer Pg1 Pg2 Pg3 Src-1 1 2 5 Src-2 6 8 - Dest 3 4 6

[0052] Note that since the 2 source buffers have the same size S (e.g., 8KB), they can use up to 2 or 3 pages, depending on the address assignment.

[0053] In Table 1, we know which channel is used by each page, but we need to find the total of channel usage by summing across buffers. An exemplary total of channel usage is shown in Table 2.

[0054] Table 2

[0055] Ch1 Ch2 Ch3 Ch4 Ch5 Ch6 Ch7 Ch8 1 1 1 1 1 2 0 1

[0056] Continuing with this example, assume that accelerators 1 and 4 are available. The cost calculated for each is shown in Table 3 and Table 4, respectively, where the cost for a given channel is determined by multiplying the channel usage across buffers for this channel, and multiplying it by the accelerator cost matrix row value from Figure 5 .

[0057] Table 3

[0058]

[0059] Table 4

[0060]

[0061] The driver will pick accelerator 4, since its cost (27) is less than that of accelerator 1 (29).

[0062] Variations

[0063] Systems without SVM

[0064] This scheme can even be done in a traditional CPU system, where the IO device has a physical address space. For efficiency, to service those uses, define a slightly different instruction Get_phys_mem_vl that provides a mapping from physical address to channel. One embodiment of this instruction is shown in Listing 2:

[0065] LISTING 2

[0066]

[0067] (Listing 2

[0068] )

[0070] Sometimes, a buffer is represented as a set of physical pages rather than a virtual memory address and length. In this case, the Get_phys_mem_v2 instruction in Listing 3 can be used:

[0071] LISTING 3

[0072]

[0073] ( Listing 3

[0074] )

[0076] For Get_phys_mem_v2, the src parameter is now a pointer to a data structure that includes a set of one or more physical (memory) pages used by the buffer, and the number parameter corresponds to the number of physical pages in the set of physical pages to replace the length (len) of the buffer.

[0077] Systems where the memory allocator does this during malloc():

[0078] During memory allocation, a user application can request a buffer with affinity to certain memory channels. An improved malloc call can then optimize this after the buffer is created, helping to make the post / map process more efficient.

[0079] Dynamic cost function:

[0080] This scheme can be extended to the case where we do not have a static cost matrix, but have one that changes with system load and possibly with QOS (quality of service) considerations. In one embodiment, memory bandwidth and latency on memory channels, as well as slot-to-slot interconnect links, are monitored. Periodically, the cost matrix can be adjusted based on heavily used regions. More generally, the cost matrix can be updated by monitoring memory bandwidth and latency on memory channels and by heuristic analysis of the observed information by the application.

[0081] Complex function:

[0082] The approach can be extended to uses such as compression where the size is unknown beforehand. In this case, we rely on the software framework to make reasonable guesses about the input and output buffer sizes. Generally, these guesses will not be perfect—for example, when compressing a 128KB buffer, we can allocate a 64KB output buffer, predicting a compression ratio of at least 2:1. However, sometimes we can achieve a 4:1 ratio and only use the first 32KB. Mapping is not ideal in this case. To improve such systems, the driver can take note that for compression-like workloads, there are provided buffer sizes, but there are also more typical sizes that are actually used (which can be tracked by the driver without application intervention) and can be used for cost calculations.

[0083] In one embodiment, each of the aforementioned Get_phys_mem_v2, Get_phys_mem_v1, and Get_phys_mem_v2 instructions can be used on a processor employing an x86 ISA (i.e., a processor based on...). The implementation is for x86 microarchitecture processors. However, this is merely exemplary and not limiting, as variations of the aforementioned instructions can be implemented on various processor architectures. For example, consider RISC-type Arm processors. ARM instructions generally allow three operands. They have integer scalar instructions that operate on general-purpose registers (GPRs) (e.g., 16 or 32 registers), and vector / floating-point instructions that operate on 128-bit SIMD (called Neon) registers.

[0059] Figure 7 An example of an embodiment of an Arm processor microarchitecture 700 is shown. Microarchitecture 700 includes: a branch prediction unit (BPU) 702; a fetch unit 704; an instruction translation back buffer (ITLB) 706; a 64KB instruction store 708; a fetch queue 710; multiple decoders (DECs) 712; a register renaming block 714; a reordering buffer (ROB) 716; reservation station units (RSUs) 718, 720, and 722; a branch arithmetic logic unit (BR / ALU) 724; an ALU / MUL (multiplier) / BR 726; shift / ALUs 728 and 730; and load / store blocks 732 and 734. The microarchitecture 700 further includes vector / floating-point (VFP) Neon blocks 736 and 738, a VFP Neon cryptographic block 740, an L2 control block 742, an integer register 744, a 128-bit VFP and Neon register 746, an ITLB 748, and a 64KB instruction store 750.

[0084] In general, each of the aforementioned Get_phys_mem_v2, Get_phys_mem_vl, and Get_phys_mem_v2 instructions described herein can be implemented using embedded logic (e.g., via circuitry), microcode, or a combination of both. Under the Arm microarchitecture, general purpose registers can be used for 64-bit m64 and r64 operands.

[0085] Due to space limitations, the NUMA nodes in Figures 1 and 2 are shown in simplified form. Figure 8 A NUMA platform architecture 800 is shown, which shows further details of a NUMA node 802a that employs a processor 804 that includes further components that are generally included in the NUMA nodes 102 and 202 in Figures 1 and 2. The NUMA node 802a includes a processor 804a that is assembled to a motherboard 805. As before, the processor 804a includes a core that includes a plurality of cores 106 that are coupled to a L1 / L2 cache 108 that in turn is coupled to an interconnect 110a. Also connected to the interconnect 110a are a PCIe root complex, and a L3 cache 112a, accelerators 116 and 118, an IO interface 808, and memory controllers 810 and 812 (also denoted as MC1 and MC2).

[0086] The PCIe root complex 806 will generally be coupled to one or more PCIe interfaces, such as the PCIe interfaces depicted by PCIe interfaces 814, 816, and 818. In general, all or a portion of the PCIe interfaces and PCIe links can be connected to a PCIe expansion slot (not shown) that is assembled on the motherboard 805. The PCIe interface 814 is depicted as being connected to an optional off-chip accelerator 820 via a PCIe link 822. As discussed above, the off-chip accelerator can include an accelerator chip that is assembled on the motherboard of the platform, or is mounted on an accelerator board or card that is assembled in a PCIe expansion slot.

[0087] The PCIe interface 816 is connected (via a PCIe link 824) to a NIC 826 that provides access to a network 828. In general, the NIC 826 denotes various types of network interface adapters and fabric adapters, including but not limited to Ethernet adapters, InfiniBand Host Controller Adapters (HCAs), and OmniPath Host Fabric Interface (HFI).

[0088] The PCIe interface 818 is connected to a solid state drive (SSD) 830 via a PCIe link 832. Optionally, other types of IO interfaces can be used to enable the processor to communicate with the SSD. As shown, system software 834 is stored in the SSD 830. The system software can generally include an operating system and one or more applications running on the operating system. The system software can also support various types of virtualization embodiments, including virtualization platforms implementing type 1 and type 2 hypervisors, as well as container-based virtualization environments. All or part of the system software can be loaded during platform boot on the network 828 as further depicted by the software 836.

[0089] The IO interface 808 is connected to a firmware storage device, such as a flash memory device 838, via an IO link 840. The flash memory device 838 stores system firmware 842 that is loaded as part of platform initialization. Generally, depending on the platform, various types of firmware can be used, including firmware that employs the Unified Extensible Firmware Interface (UEFI) architecture. For historical reasons, all or part of the firmware can also be referred to as BIOS (Basic Input Output System).

[0090] Generally, the processor can include one or more memory controllers, each including one or more memory channels connected to one or more memory devices, as discussed above. Figure 8 Embodiments include two memory controllers 810 and 812. The memory controller 810 is coupled to memory devices DRAM 1 and DRAM 2 via respective memory channels 1 and 2. The memory controller 812 is coupled to memory devices DRAM 3 and DRAM 4 via respective memory channels 3 and 4.

[0091] As discussed above, in some embodiments, the topology of the platform interconnect fabric is determined during initialization. For PCIe components, this information is automatically determined during PCIe enumeration and can be accessed via PCIe firmware or software drivers. Similarly, various firmware and / or software drivers can be used to enumerate various IO and memory interfaces, as well as other interconnect links, including any socket-to-socket links, on the platform.

[0092] Further aspects of the subject matter described herein are set forth in the following numerous clauses:

[0093] 1. A method performed on a computing platform, the computing platform including a plurality of accelerators and a plurality of memory devices accessed through a plurality of memory channels, the method comprising:

[0094] in response to a job submission to access a function to be offloaded to an accelerator, the function to access one or more buffers, the one or more buffers collectively requiring access via a plurality of memory channels among the plurality of memory channels,

[0095] identifying accelerators having available instances of the function;

[0096] for each identified accelerator, computing a total cost of accessing the one or more buffers via the plurality of memory channels; and

[0097] selecting an accelerator having a minimum total cost to offload the function.

[0098] 2. The method of clause 1, wherein the computing platform is a multi-socket computing platform having a non-uniform memory access (NUMA) architecture, the NUMA architecture including a shared virtual memory (SVM), the one or more buffers being logically stored in the SVM, and wherein at least a portion of the plurality of accelerators are SVM-capable accelerators.

[0099] 3. The method of clause 1 or 2, further comprising:

[0100] determining an interconnect topology between the plurality of accelerators and the plurality of memory channels.

[0101] constructing a cost matrix comprising a plurality of cells based on the interconnect topology, wherein each cell is populated with a cost of an accelerator of the plurality of memory channels accessing a memory channel of the plurality of memory channels using the plurality of memory channels; and

[0102] computing the total cost for each of the identified accelerators using the cost matrix.

[0103] 4. The method of clause 3, further comprising:

[0104] determining a plurality of memory pages associated with the one or more buffers, the one or more buffers accessed through each memory channel of the plurality of memory channels; and

[0105] computing the cost of accessing the memory pages via a memory channel of the plurality of memory channels by multiplying the cost of the accelerator accessing the memory channel in the cost matrix by a number of memory pages associated with the one or more buffers accessed through this memory channel.

[0106] 5. The method of clause 3, further comprising:

[0107] updating one or more cells using dynamically determined updated costs during ongoing runtime operations of the computing platform.

[0108] 6. The method of clause 5, wherein the cost of at least one unit or the updated cost of one or more units is determined by monitoring bandwidth along one or more links in the interconnect topology and updating the cost based on the monitored bandwidth applying heuristics.

[0109] 7. The method of any of the preceding clauses, further comprising:

[0110] for each buffer of the one or more buffers, identifying one or more memory pages allocated to the buffer;

[0111] for each memory page of the one or more memory pages, identifying a memory channel used to access the memory page; and

[0112] for each accelerator of the one or more accelerators, determining a respective cost of accessing the memory pages via each of the memory channels,

[0113] wherein the aggregate cost for a given accelerator is determined by,

[0114] summing, for each memory page identified in i, the cost of the accelerator to access the memory page.

[0115] 8. The method of any of the preceding clauses, wherein the computing platform comprises a processor comprising an instruction set architecture (ISA) having an instruction that is executed once for each buffer of the one or more buffers to identify memory pages used by the buffer and a memory channel used to access each memory page.

[0116] 9. The method of clause 8, wherein the instruction has a first operand in a first register to store a pointer to a data structure, a second operand in a second register to store an address of the buffer, and a third operand in a third register to store a length of the buffer or a number of memory pages spanned by the buffer, wherein the instruction, when executed, is configured to populate the data structure with data identifying, for each memory page of the one or more memory pages, a memory channel used to access the memory page.

[0117] 10. The method of clause 8, wherein the processor employs an ARM-based architecture.

[0118] 11. A multi-socket computing platform having a non-uniform memory access (NUMA) architecture, comprising:

[0119] a plurality of NUMA nodes, each NUMA node comprising:

[0120] a processor comprising a plurality of components, the plurality of components comprising:

[0121] a plurality of processor cores;

[0122] at least one memory controller having a plurality of memory channels;

[0123] socket-to-socket interfaces;

[0124] at least one accelerator; and

[0125] internal interconnect circuitry coupling a plurality of components in communication;

[0126] a plurality of memory devices communicatively coupled to the plurality of memory channels on the processor; and

[0127] socket-to-socket interconnect circuitry for enabling communication between pairs of processors;

[0128] wherein, in operation, the multi-socket computing platform is configured to,

[0129] in response to a job submission via software running on the multi-socket computing platform for accessing a function to be offloaded to an accelerator, the function for accessing one or more buffers, the one or more buffers collectively requiring access via a plurality of memory channels among the plurality of memory channels,

[0130] identifying accelerators having available instances of the function;

[0131] for each identified accelerator, computing a total cost of accessing the identified one or more buffers via the plurality of memory channels; and

[0132] selecting an accelerator having a minimum total cost to offload the function.

[0133] 12. The multi-socket computing platform of clause 11, wherein the NUMA architecture includes a shared virtual memory (SVM), the one or more buffers are logically stored in the SVM, and wherein at least a portion of the plurality of accelerators are SVM-capable accelerators.

[0134] 13. The multi-socket computing platform of clause 11 or 12, wherein, in operation, the multi-socket computing platform is further configured to:

[0135] determine an interconnect topology between each of the plurality of accelerators and each of the plurality of memory channels;

[0136] construct a cost matrix comprising a plurality of cells based on the interconnect topology, wherein each cell is populated with a cost of an accelerator in the plurality of accelerators accessing a memory channel in the plurality of memory channels; and

[0137] The cost matrix is used to compute a total cost for each of the identified accelerators.

[0138] 14. The multi-socket computing platform of clause 13, wherein, in operation, the multi-socket computing platform is further configured to:

[0139] determine a plurality of memory pages associated with one or more buffers accessed by each of the plurality of memory channels; and

[0140] compute a cost of accessing the memory pages via a memory channel of the plurality of memory channels by multiplying a cost of the corresponding accelerator accessing the memory channel in the cost matrix by a number of memory pages associated with one or more buffers accessed through this memory channel.

[0141] 15. The multi-socket computing platform of clause 13, wherein, in operation, the multi-socket computing platform is further configured to:

[0142] update one or more cells using the dynamically determined updated costs during ongoing runtime operation of the computing platform.

[0143] 16. The multi-socket computing platform of clause 15, wherein the updated costs of at least one or the one or more cells are determined by monitoring bandwidths along one or more links in the interconnect topology and updating the costs based on the monitored bandwidths using a heuristic.

[0144] 17. The multi-socket computing platform of any one of clauses 11-16, wherein, in operation, the multi-socket computing platform is further configured to:

[0145] for each buffer of the one or more buffers, identify one or more memory pages allocated to this buffer;

[0146] for each memory page of the one or more memory pages, identify a memory channel used to access this memory page; and

[0147] for each accelerator of the accelerators, determine a respective cost of accessing the memory pages via each of the memory channels,

[0148] wherein the total cost for a given accelerator is determined by summing the cost of this accelerator accessing each of the memory pages identified in i.

[0149] 18. The multi-socket computing platform of any of clauses 11-17, wherein each processor comprises an instruction set architecture (ISA) having instructions that, upon execution, are capable of identifying memory pages used by a buffer and a memory channel used to access each memory page.

[0150] 19. The multi-socket computing platform of clause 18, wherein the instructions have a first operand in a first register to store a pointer to a data structure, a second operand in a second register to store an address of the buffer, and a third operand in a third register to store a length of the buffer or a number of memory pages spanned by the buffer, wherein the instructions, upon execution, are configured to populate the data structure with data that identifies, for each of the one or more memory pages, a memory channel used to access the memory page.

[0151] 20. The multi-socket computing platform of clause 18, wherein the processors employ an ARM-based architecture.

[0152] 21. A processor configured to be installed in a computing platform having a plurality of memory devices that are accessed by a plurality of memory channels and having an instruction set architecture (ISA) comprising instructions having a first operand in a first register to store a pointer to a data structure, a second operand in a second register to store an address of a buffer that spans one or more memory pages, and a third operand in a third register to store a length of the buffer or a number of memory pages spanned by the buffer, wherein the instructions, upon execution, are configured to populate the data structure with data that identifies, for each of the one or more memory pages, a memory channel used to access the memory page.

[0153] 22. The processor of clause 21, wherein the computing platform employs a multi-socket non-uniform memory access (NUMA) architecture comprising a plurality of sockets, the processor further comprising at least one socket-to-socket interface, and wherein when the processor is installed in one socket of the plurality of sockets, the processor is communicatively coupled to a second processor via a socket-to-socket link.

[0154] 23. The processor of clause 22, wherein a plurality of memory devices from among the plurality of memory devices are coupled to each of the processor and the second processor via the plurality of memory channels, and wherein upon being populated via execution of the instructions, the data structure comprises data that identifies at least one memory channel for each of the processor and the second processor.

[0155] 24. The processor of any of clauses 21-23, wherein the processor comprises at least one accelerator comprising circuitry embedded on the processor.

[0156] 25. The processor of any of clauses 21-24, wherein the address of the buffer is a virtual memory address.

[0157] 26. The processor of clause 25, wherein the data structure further comprises data identifying a physical address of the memory for each of the one or more memory pages.

[0158] 27. The processor of clause 25, wherein the processor further comprises a translation lookaside buffer (TLB), and the instruction, upon execution, accesses the TLB to find the physical address for each of the one or more memory pages.

[0159] 28. The processor of any of clauses 21-27, wherein the first operand is stored in a first 64-bit register, the second operand is stored in a second 64-bit register, and the third operand is stored in a third 64-bit register, and wherein each of the first and second 64-bit registers is configured to store a pointer to a memory to be used with the load / store instruction.

[0160] 29. The processor of any of clauses 21-28, wherein the processor employs an Arm-based microarchitecture.

[0161] 30. The processor of any of clauses 21-28, wherein the processor employs an Intel x86-based microarchitecture.

[0162] 31. A multi-socket computing platform having a non-uniform memory access (NUMA) architecture, comprising:

[0163] a plurality of NUMA nodes, each NUMA node comprising:

[0164] a processor comprising a plurality of components, the plurality of components comprising:

[0165] a plurality of processor cores;

[0166] at least one memory controller having a plurality of memory channels;

[0167] a socket-to-socket interface;

[0168] at least one accelerator; and

[0169] internal interconnect circuitry coupling the plurality of components in communication;

[0170] a plurality of memory devices communicatively coupled to the plurality of memory channels on the processor; and

[0171] socket-to-socket interconnect circuitry to enable communication between pairs of processors;

[0172] wherein the multi-socket computing platform further comprises means for,

[0173] in response to a job submission via software running on the multi-socket computing platform for accessing a function to be offloaded to an accelerator, the function for accessing one or more buffers, the one or more buffers collectively requiring access via a plurality of the memory channels,

[0174] identifying accelerators having available instances of the function;

[0175] for each identified accelerator, computing a total cost of accessing the identified one or more buffers via the plurality of memory channels; and

[0176] selecting an accelerator having a minimum total cost to offload the function.

[0177] 32. The multi-socket computing platform of clause 31, wherein the NUMA architecture comprises a shared virtual memory (SVM), the one or more buffers being logically stored in the SVM, and wherein at least a portion of the plurality of accelerators are SVM-capable accelerators.

[0178] 33. The multi-socket computing platform of clause 31 or 32, wherein the multi- socket computing platform further comprises means for:

[0179] determining an interconnect topology between each of the plurality of accelerators and each of the plurality of memory channels;

[0180] constructing a cost matrix comprising a plurality of cells based on the interconnect topology, wherein each cell is populated with a cost of an accelerator of the plurality of accelerators accessing a memory channel of the plurality of memory channels; and

[0181] computing a total cost for each of the identified accelerators using the cost matrix.

[0182] 34. The multi-socket computing platform of clause 33, wherein the multi-socket computing platform further comprises means for:

[0183] determining a plurality of memory pages associated with the one or more buffers, the one or more buffers being accessed by each of the plurality of memory channels; and

[0184] The cost of accessing a memory page via a memory channel is computed by multiplying the cost of the accelerator accessing the memory channel in the cost matrix by the number of memory pages associated with one or more buffers accessed through this memory channel.

[0185] 35. The multi-socket computing platform of clause 33, wherein the multi-socket computing platform further comprises means for:

[0186] updating the one or more units using the updated costs determined dynamically during ongoing runtime operations of the computing platform.

[0187] 36. The multi-socket computing platform of clause 35, wherein the updated costs of at least one or the one or more units are determined by monitoring bandwidth along one or more links in the interconnect topology and updating the costs based on the monitored bandwidth using heuristics.

[0188] 37. The multi-socket computing platform of any one of clauses 31-36, wherein the multi-socket computing platform further comprises means for:

[0189] identifying, for each buffer in the one or more buffers, one or more memory pages allocated to the buffer;

[0190] identifying, for each memory page in the one or more memory pages, a memory channel used to access the memory page; and

[0191] determining, for each accelerator in the accelerators, a respective cost of accessing the memory pages via each memory channel in the memory channels,

[0192] wherein the aggregate cost for a given accelerator is determined by summing the cost of the accelerator accessing each memory page identified in i to the memory page.

[0193] 38. The multi-socket computing platform of any one of clauses 31-37, wherein each processor comprises an instruction set architecture (ISA) having instructions that, upon execution, are capable of identifying memory pages used by a buffer and a memory channel used to access each memory page.

[0194] 39. The multi-socket computing platform of clause 38, wherein the instructions have a first operand in a first register to store a pointer to a data structure, a second operand in a second register to store an address of the buffer, and a third operand in a third register to store a length of the buffer or a number of memory pages spanned by the buffer, wherein the instructions, upon execution, are configured to populate the data structure with data identifying, for each memory page in the one or more memory pages, a memory channel used to access the memory page.

[0195] 40. The multi-socket computing platform of clause 38, wherein the processors employ an ARM-based architecture.

[0196] 41. A method comprising:

[0197] determining an interconnect topology between a plurality of accelerators and a plurality of memory channels in a computing platform;

[0198] constructing a cost matrix comprising a plurality of cells, wherein each cell is populated with a cost of an accelerator x reading / writing data to a memory channel y;

[0199] in response to a job submission to access a function to be offloaded to an accelerator,

[0200] identifying accelerators having available instances of the function;

[0201] using the cost matrix to compute an accelerator having an available instance of the function with a lowest aggregate cost; and

[0202] selecting this accelerator to offload the function.

[0203] 42. The method of clause 41, wherein the computing platform is a multi-socket computing platform having a non-uniform memory access (NUMA) architecture.

[0204] 43. The method of clause 41 or 42, wherein the job submission identifies one or more buffers to be accessed by the accelerator, the method further comprising:

[0205] for each of the one or more buffers, identifying a memory page for this buffer; and

[0206] for each identified memory page, identifying a memory channel for accessing this memory page.

[0207] 44. The method of clause 43, further comprising:

[0208] determining a number of memory pages associated with the one or more buffers that are accessed through each memory channel; and

[0209] computing a cost of accessing the memory pages via a memory channel by multiplying a cost of an accelerator accessing the memory channel in the cost matrix by a number of memory pages associated with the one or more buffers that are accessed through this memory channel.

[0210] 45. The method of clause 43, wherein the computing platform comprises a processor that comprises an instruction set architecture (ISA) having an instruction that is executed once for each of the one or more buffers to identify a memory page used by the buffer and a memory channel used to access each memory page.

[0211] 46. The method of clause 45, wherein the instruction has a first operand in a first register to store a pointer to a data structure, a second operand in a second register to store an address of the buffer, and a third operand in a third register to store a length of the buffer or a number of memory pages spanned by the buffer, wherein the instruction, upon execution, is configured to populate the data structure with data that identifies, for each of the one or more memory pages, a memory channel used to access the memory page.

[0212] 47. The method of clause 45, wherein the processor employs an ARM-based architecture.

[0213] 48. The method of any of clauses 41-47, further comprising:

[0214] updating the one or more units using the updated cost that is dynamically determined during an ongoing runtime operation of the computing platform.

[0215] 49. The method of clause 45, wherein the updated cost of at least one or the one or more units is determined by monitoring bandwidth along one or more links in the interconnect topology and updating the cost based on the monitored bandwidth using a heuristic.

[0216] 50. The method of any of clauses 41-49, wherein the computing platform comprises a processor that has at least one accelerator integrated on the processor.

[0217] 51. The method of any of clauses 41-50, wherein the computing platform comprises a processor and the computing platform comprises at least one accelerator that is external to the processor.

[0218] 52. A processor configured to be installed in a computing platform that has a plurality of memory devices that are accessed by a plurality of memory channels, and the processor has an instruction set architecture (ISA) that includes an instruction that has a first operand in a first register to store a pointer to a data structure, a second operand to store a pointer to a data structure that comprises a set of physical pages, and a third operand in a third register to store a length of the buffer or a number of physical pages in the set of physical pages, wherein the instruction, upon execution, is configured to populate the data structure with data that identifies, for each of the physical pages, a memory channel used to access the physical page.

[0219] 53. The processor of clause 52, wherein the computing platform employs a multi- socket non-uniform memory access (NUMA) architecture comprising a plurality of sockets, the processor further comprising at least one socket-to-socket interface, and wherein the processor is communicatively coupled to a second processor via a socket-to-socket link when the processor is installed in one socket of the plurality of sockets.

[0220] 54. The processor of clause 53, wherein a plurality of memory devices from among the plurality of memory devices are coupled to each of the processor and the second processor via a plurality of memory channels, and wherein the data structure comprises data identifying at least one memory channel for each of the processor and the second processor upon being populated via execution of the instruction.

[0221] 55. The processor of any one of clauses 52-54, wherein the processor comprises at least one accelerator comprising circuitry embedded on the processor.

[0222] 56. The processor of any one of clauses 52-55, wherein the processor employs an Arm-based microarchitecture.

[0223] 57. The processor of any one of clauses 52-55, wherein the processor employs an Intel x86-based microarchitecture.

[0224] While some embodiments have been described with reference to particular implementations, other implementations are possible according to some embodiments. Additionally, the arrangement and / or order of elements or other features illustrated in the figures and / or described herein need not be arranged or ordered as illustrated and / or described. Many other arrangements are possible according to some embodiments.

[0225] In each of the apparatuses or systems illustrated in the figures, in some cases, elements can each have the same or different reference numbers to indicate that the represented elements can be different and / or similar. However, the elements can be flexible enough to have different implementations and operate with some or all of the systems illustrated or described herein. The various elements illustrated in the figures can be the same or different. It is arbitrary which is called a first element and which is called a second element.

[0226] In the description and claims, the terms "coupled" and "connected," along with their derivatives, can be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, "connected" can be used to indicate that two or more elements are in direct physical and / or electrical contact with each other. "Coupled" can mean that two or more elements are in direct physical or electrical contact with each other, but can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

[0227] Embodiments are implementations or examples of the application. References in the specification to "an embodiment," "one embodiment," "some embodiments," or "other embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments of the application, but not necessarily all embodiments of the application. The various appearances of "embodiment," "one embodiment,” or "some embodiments" in the specification do not necessarily all refer to the same embodiments.

[0228] Not all components, features, structures, characteristics, etc. described and shown in the specification are necessary for all embodiments of the application. For example, some embodiments can not include an element discussed and / or shown in a particular figure. For example, if an embodiment includes a component, feature, structure, and / or characteristic, but does not include a specific one of those, then the specification should not be regarded as suggesting that the specific one is a required feature or component. If the specification or claim states that an element can be included, then it can be included in some embodiments, but not necessarily in others. Where the specification states that a component, feature, structure, or characteristic "may" be included, this means that an embodiment can or can not include the component, feature, structure, or characteristic. Where the specification states that a component, feature, structure, or characteristic "is" included, this means that an embodiment can or can not include the component, feature, structure, or characteristic.

[0229] As discussed above, aspects of the embodiments herein can be facilitated by corresponding software and / or firmware components and applications, such as software and / or firmware executed by embedded processors or the like. Thus, embodiments of the application can be used as or to support software programs, software modules, firmware, and / or virtual machines that execute on some form of processor, processing core, or embedded logic, a virtual machine running on or otherwise realized or achieved within or by a non-transitory computer- or machine-readable storage medium. Non-transitory computer- or machine-readable storage media include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, non-transitory computer- or machine-readable storage media include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and the like. Content can be directly executable ("object" or "executable" form), source code, or difference code ("delta" or "patch" code). Non-transitory computer- or machine-readable storage media can also include memories stores or databases that content can be downloaded from. Non-transitory computer- or machine-readable storage media can also include devices or products that store content thereon when sold or transported. Thus, transporting a device with stored content or providing content for download over a communication medium can be understood to provide an article of manufacture including a non-transitory computer- or machine-readable storage medium having such content as described herein.

[0230] The various components described herein above as processes, servers, or tools can be apparatuses that perform the described functions. The operations and functions performed by the various components described herein can be implemented by software running on processing elements, via embedded hardware, or any combination of hardware and software. Such components can be implemented as software modules, hardware modules, special-purpose hardware (e.g., application specific hardware, ASICs, DSPs, etc.), embedded controllers, hardwired circuitry, hardware logic, etc. Software content (e.g., data, instructions, configuration information, etc.) can be provided via an article of manufacture including a non-transitory computer- or machine-readable storage medium that provides content representing instructions that can be executed. The content can cause a computer to perform various functions / operations described herein.

[0231] As used herein, a list of items connected by a term such as "at least one of" can mean any combination of the listed terms. For example, the phrases "at least one of A, B, or C" can mean A; B; C; A and B; A and C; B and C; or A, B, and C.

[0232] In the following claims, the term one or more buffers that need to be accessed via multiple memory channels includes the following. Each buffer will generally be stored in such a way that all buffers can be accessed using a single memory channel, and two separate buffers can use the same or different memory channels. In addition, the contents of a single buffer can be split such that it is stored in more than one memory device or in a single memory device with multiple channels between the memory controller and the memory device. Thus, in these scenarios, it is possible that accessing the contents of a single buffer can require the use of more than one memory channel.

[0233] Italicized terms such as 'x' and 'y' in the above DETAILED DESCRIPTION are used to depict integers, and the use of a particular letter is not limited to a particular embodiment. In addition, the same letter can be used in different claims to represent different integers or different letters can be used. Further, the use of a particular letter in the DETAILED DESCRIPTION can or can not match the letter used in the claims referring to the same subject matter in the DETAILED DESCRIPTION.

[0234] The above description of illustrated embodiments of the application, including what is described in the abstract, is not intended to be exhaustive or to be limited to the precise forms disclosed. While specific embodiments of, and examples for, the application are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the application, as those skilled in the relevant art will recognize.

[0235] These modifications can be made in light of the above DETAILED DESCRIPTION. The terms used in the following claims should not be construed to limit the application to the specific embodiments disclosed in the specification and drawings. Rather, the scope of the application is to be determined entirely by the following claims, which are to be construed in accordance with the principles of claim interpretation.

Claims

1. A method executed on a computing platform, the computing platform including a plurality of accelerators and a plurality of memory devices accessed through a plurality of memory channels, the method comprising: Determine the interconnection topology between the plurality of accelerators and the plurality of memory channels; Based on the interconnect topology, a cost matrix comprising multiple units is constructed, wherein each unit is filled with the cost of accessing a memory channel among the multiple memory channels by one of the multiple accelerators; In response to a job submission for accessing a function to be offloaded to the accelerator, the function accessing one or more buffers, all of which require access via a plurality of memory channels. An accelerator that identifies available instances of the described functionality; For each identified accelerator, the cost matrix is ​​used to calculate the total cost of accessing the one or more buffers via the plurality of memory channels; as well as Select the accelerator with the lowest total cost to offload the function to.

2. The method as described in claim 1, characterized in that, The computing platform is a multi-slot computing platform with a non-uniform memory access (NUMA) architecture. The NUMA architecture includes a shared virtual memory (SVM), and the one or more buffers are logically stored in the SVM. At least a portion of the multiple accelerators are accelerators with SVM capabilities.

3. The method as described in claim 1, characterized in that, Further includes: Determine the number of memory pages associated with the one or more buffers, which are accessed through each of the plurality of memory channels; as well as The cost of accessing memory pages via this memory channel is calculated by multiplying the cost of the accelerator for accessing the memory channel in the cost matrix by the number of memory pages associated with the one or more buffers accessed through this memory channel.

4. The method as described in claim 1, characterized in that, Further includes: During the ongoing operation of the computing platform, one or more units are updated using dynamically determined updated costs.

5. The method as described in claim 4, characterized in that, By monitoring the bandwidth of one or more links in the interconnect topology and applying a heuristic method based on the monitored bandwidth to update the cost, the cost of at least one unit whose cost has been updated, or the updated cost of the one or more units, is determined.

6. The method as described in claim 1, characterized in that, Further includes: i. For each of the one or more buffers, identify one or more memory pages allocated to that buffer; ii. For each of the one or more memory pages, identify the memory channel used to access this memory page; as well as iii. For each of the accelerators, determine the corresponding cost of accessing a memory page via each of the memory channels. The total cost of a given accelerator is determined in the following way. For each memory page identified in i, sum the cost of the accelerator accessing this memory page.

7. The method as described in claim 1, characterized in that, The computing platform includes a processor, the processor including an instruction set architecture (ISA) having instructions that are executed once for each of the one or more buffers to identify the memory page used by that buffer and the memory channel for accessing each memory page.

8. The method as described in claim 7, characterized in that, The instruction has: a first operand in a first register for storing a pointer to a data structure; and a second operand in a second register for storing the address of a buffer; And a third operand in a third register for storing the length of the buffer or the number of memory pages spanned by the buffer, wherein the instruction is configured to populate the data structure with data at execution time, the data identifying a memory channel for accessing this memory page for each of the one or more memory pages.

9. The method as described in claim 7, characterized in that, The processor uses an ARM-based architecture.

10. A multi-socket computing platform with a non-uniform memory access (NUMA) architecture, comprising: Multiple NUMA nodes, each NUMA node includes, The processor includes multiple components, the multiple components including, Multiple processor cores; At least one memory controller, the at least one memory controller having multiple memory channels; Slot-to-slot interface; At least one accelerator; and Internal interconnect circuitry, said internal interconnect circuitry coupling the plurality of components in communication; Multiple memory devices, the multiple memory devices being communicatively coupled to multiple memory channels on the processor; and A slot-to-slot interconnect circuit system for enabling communication between pairs of processors; During operation, the multi-slot computing platform is configured to, Determine the interconnection topology between each of the plurality of accelerators and each of the plurality of memory channels; Based on the interconnect topology, a cost matrix comprising multiple units is constructed, wherein each unit is filled with the cost of accessing a memory channel among the multiple memory channels by one of the multiple accelerators; In response to a job submission via software running on the multi-socket computing platform for accessing a function to be offloaded to the accelerator, the function accessing one or more buffers, all of which require access via a plurality of memory channels among the plurality of memory channels, An accelerator that identifies available instances of the described functionality; For each identified accelerator, the cost matrix is ​​used to calculate the total cost of accessing the identified one or more buffers via the plurality of memory channels; and Select the accelerator with the lowest total cost to offload the function to.

11. The multi-slot computing platform as described in claim 10, characterized in that, The NUMA architecture includes a shared virtual memory (SVM), the one or more buffers are logically stored in the SVM, and at least a portion of the plurality of accelerators are accelerators with SVM capabilities.

12. The multi-slot computing platform as described in claim 10, characterized in that, During operation, the multi-slot computing platform is further configured to: Identify a plurality of memory pages associated with the one or more buffers, which are accessed through each of the plurality of memory channels; as well as The cost of accessing a memory page via this memory channel is calculated by multiplying the cost of the corresponding accelerator accessing the memory channel in the cost matrix by the number of memory pages associated with the one or more buffers accessed through this memory channel.

13. The multi-slot computing platform as described in claim 10, characterized in that, During operation, the multi-slot computing platform is further configured to: During the ongoing operation of the computing platform, one or more units are updated using dynamically determined updated costs.

14. The multi-slot computing platform as described in claim 13, characterized in that, The updated cost of one or more units or at least one unit whose cost has been updated is determined by monitoring the bandwidth of one or more links along the interconnect topology and applying a heuristic method based on the monitored bandwidth to update the cost.

15. The multi-slot computing platform as described in claim 10, characterized in that, During operation, the multi-slot computing platform is further configured to: i. For each of the one or more buffers, identify one or more memory pages allocated to that buffer; i. For each of the one or more memory pages, identify the memory channel used to access this memory page; as well as iii. For each of the accelerators, determine the corresponding cost of accessing a memory page via each of the memory channels. The total cost of a given accelerator is determined by summing the cost of accessing this memory page for each memory page identified in i.

16. The multi-slot computing platform as described in claim 10, characterized in that, Each processor includes an instruction set architecture (ISA) having instructions that, when executed, are implemented to identify memory pages used by buffers and memory channels for accessing each memory page.

17. The multi-slot computing platform as described in claim 16, characterized in that, The instruction has: a first operand in a first register for storing a pointer to a data structure; and a second operand in a second register for storing the address of a buffer; And a third operand in a third register for storing the length of the buffer or the number of memory pages spanned by the buffer, wherein the instruction is configured at execution to populate the data structure with data that identifies a memory channel for accessing each of the one or more memory pages.

18. The multi-slot computing platform as described in claim 16, characterized in that, The processor uses an ARM-based architecture.

19. The multi-slot computing platform as described in claim 10, characterized in that, The at least one accelerator includes a circuit system embedded in the processor.

20. The multi-slot computing platform as described in claim 17, characterized in that, The address of the buffer is a virtual memory address.

Images