Provides direct data access between accelerators and storage devices in the computing environment.

Abstract

A method for providing direct access to non-volatile memory by a processor in a computing environment includes: providing direct access to the non-volatile memory device to one or more accelerators via an application programming interface (“API”) independent of the host central processing unit (“CPU”) on the control or data path, thereby performing read and write operations on the data.

Description

Technical Field

[0001] This invention generally relates to computing systems, and more particularly to providing direct access between accelerators and non-volatile memories and storage devices in a computing environment. Background Technology

[0002] A popular type of large-scale computing is cloud computing, where resources can interact and / or be accessed via communication systems such as computer networks. Resources can be computing devices, storage devices, applications, and / or software-rendered simulations and / or emulations of other computer-related devices and / or services running on one or more computing devices (such as servers). For example, multiple servers can transfer and / or share information that can scale across servers based on processing power, storage space, and / or the amount of other computing resources required to complete a requested task. The term "cloud" refers to the cloud-like appearance of the interconnectivity diagram between computing devices, computer networks, and / or other computer-related devices interacting in this arrangement. Summary of the Invention

[0003] Various aspects are provided for providing direct access to non-volatile memory in a computing environment. In one aspect, a method is provided for again providing direct access between an accelerator and non-volatile memory in a computing environment by a processor. Direct access to the non-volatile memory device can be provided to one or more accelerators (e.g., a graphics processing unit "GPU") via an application programming interface ("API"), independent of the host central processing unit ("CPU") on the control or data path, thereby performing data read and write operations. Attached Figure Description

[0004] To facilitate understanding of the advantages of the invention, a more specific description of the invention, which has been briefly described above, will be presented with reference to specific embodiments shown in the accompanying drawings. It should be understood that these drawings depict only exemplary embodiments of the invention and should not be considered as limiting its scope. Additional features and details of the invention will be described and explained using the drawings, in which:

[0005] Figure 1 This is a block diagram depicting an exemplary computing node according to an embodiment of the present invention;

[0006] Figure 2 This is an additional block diagram depicting an exemplary cloud computing environment according to an embodiment of the present invention;

[0007] Figure 3 This is an additional block diagram depicting the abstract model layer according to an embodiment of the present invention;

[0008] Figure 4A cloud computing network in which various aspects of the present invention can be implemented is shown;

[0009] Figures 5A-5C This is a block diagram illustrating exemplary operation for providing direct access between an accelerator and a non-volatile storage device in a computing environment, according to various aspects of the present invention.

[0010] Figure 6 This is a block diagram depicting an exemplary accelerator page cache organization and application programming interface (“API”) in which various aspects of the present invention can be implemented;

[0011] Figures 7A-7C It is a table describing the application programming interface (“API”) calls for an accelerator in which various aspects of the present invention can be implemented; and

[0012] Figure 8 This is a flowchart depicting an exemplary method for providing direct access between an accelerator and a non-volatile storage device in a computing environment, in which aspects of the invention can also be implemented. Detailed Implementation

[0013] Currently, many computer systems employ accelerators (e.g., coprocessors such as graphics processing units (GPUs) to enhance the performance of such systems, where programs run on both the central processing unit (“CPU”) and the accelerator. For example, GPUs can be involved in performing complex mathematical calculations. Similarly, the parallel architecture of accelerators makes them more efficient than CPUs for applications / operations that can handle large blocks of data.

[0014] Furthermore, application data is currently stored on non-volatile memory because the data is often too large to be stored in GPU / CPU memory. Since non-volatile memory is managed by the CPU, the CPU manages GPU memory (through multiple memory copies and kernel calls), or the GPU kernel must request data from the CPU, requiring synchronization with the CPU and queuing of requests. Assuming the GPU is a high-throughput device, a large number of threads execute concurrently. Since all parallel threads may be waiting for data, large latency exists and occurs. Once a block is scheduled on the GPU's streaming multiprocessor ("SM"), that block is held until it completes. If a thread is forced to wait for data, execution resources are held, and the block for which its data is available cannot be executed. Therefore, GPU resources are being inefficiently managed, resulting in wasted synchronization with the host CPU and waiting for data.

[0015] Therefore, feeding or providing data to accelerators such as GPUs with low latency and high bandwidth has become a bottleneck in data-intensive applications. Currently, as mentioned earlier, accelerators (e.g., GPUs) rely on the host central processing unit (“CPU”) to manage the accelerator’s memory and on non-volatile memory to provide data to the accelerator. This not only increases data copying overhead but also increases control overhead in the form of synchronization between the high-throughput accelerator and the CPU. Furthermore, this not only requires additional host resources but can also add an order of magnitude of latency and significantly reduce data access bandwidth, given the sub-microsecond access latency of modern non-volatile memory fast (“NVMe”) devices, as shown below. Figure 5A As shown in the image.

[0016] To fully utilize the low latency / high bandwidth of high-capacity NVMe devices, various embodiments of the present invention allow accelerators (e.g., GPUs) to access them directly without involving the host CPU in the control or data plane. In one aspect, direct access to non-volatile storage devices can be provided to one or more accelerators (e.g., graphics processing units "GPUs") via an application programming interface ("API"), independent of the host central processing unit ("CPU") on the control or data path, thereby performing data read and write operations.

[0017] Therefore, the various embodiments described herein increase computational efficiency and utilization of computing units available on the GPU, as GPU memory capacity is currently mismatched with GPU computing power. Furthermore, the present invention frees up the CPU to perform other useful work in parallel, rather than simply managing data transfer between the GPU and storage devices. The present invention further provides direct GPU thread access to terabytes of non-volatile memory, while also allowing GPU threads to manage GPU memory instead of relying on the CPU for GPU memory management, so that GPU threads can now directly access the data they need, thereby providing cost savings and reduced computational power consumption.

[0018] Furthermore, the GPU can be used even when the data is not suitable for the GPU's memory, enabling the GPU to be used for more irregular applications with large amounts of data (e.g., large graphics processing). This invention further eliminates synchronization between the CPU and GPU to obtain more data for the GPU. Additionally, the prefetching operation described herein allows GPU computing resources to be released until the data for the prefetched block is ready in the GPU memory (e.g., the data for the requested data block exists in the GPU memory in a state where they can be used). Moreover, due to the fact that it is now less dependent on the CPU, this invention allows for the scaling up of more GPUs without incurring the additional cost of increasing the number of host CPUs. This invention further improves fault tolerance and management, while also enabling the co-scheduling of tasks and I / O on one or more GPUs to avoid congestion and utilize locality.

[0019] This document will further describe other examples and corresponding benefits of various aspects of the illustrated embodiments.

[0020] It should be understood in advance that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings recorded herein is not limited to a cloud computing environment or an Internet of Things (IoT) network environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter. It should be noted that IoT is an emerging concept involving computing devices that can be embedded in objects, such as appliances, and connected via a network. An IoT network may include one or more IoT devices or “smart devices,” which are physical objects such as appliances in which computing devices are embedded. Many IoT devices are independent of the mainframe, but they may also be paired with a control system or with a distributed control system, such as one running in a cloud computing environment. The control system may include an end-to-end flow monitoring mechanism similar to that described herein.

[0021] Cloud computing is a service delivery model that enables convenient, on-demand network access to a pool of shared, configurable computing resources (e.g., networks, network bandwidth, servers, processing power, storage, applications, VMs, and services) that can be rapidly provisioned and released with minimal management costs or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0022] The characteristics are as follows:

[0023] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power (such as server time and network storage) on demand without human interaction with the service provider.

[0024] Wide network access: Capabilities are available on the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0025] Resource pooling: A provider's computing resources are grouped into resource pools to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. Typically, consumers cannot control or know the exact location of the resources provided, but can specify the location at a higher level of abstraction (e.g., country, state, or data center), thus exhibiting location independence.

[0026] Rapid flexibility: Capabilities can be rapidly and flexibly (in some cases automatically) provided to expand outward quickly and be rapidly released to shrink back down. For consumers, the available capacity often appears unlimited and can be purchased at any time and in any quantity.

[0027] Measurable services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0028] The service model is as follows:

[0029] Software as a Service (SaaS): The capability offered to consumers is the ability to use applications running on a provider's cloud infrastructure. These applications can be accessed from various client devices via thin client interfaces such as web browsers (e.g., web-based email). Aside from limited user-specific application configuration settings, consumers neither manage nor control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities.

[0030] Platform as a Service (PaaS): This provides consumers with the ability to deploy consumer-created or acquired applications on cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers neither manage nor control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the applications they deploy and may also have control over the configuration of the application hosting environment.

[0031] Infrastructure as a Service (IaaS): This provides consumers with the capability to deploy and run any software, including operating systems and applications, on the cloud, providing them with processing, storage, networking, and other basic computing resources. Consumers neither manage nor control the underlying cloud infrastructure, but they have control over the operating system, storage, and deployed applications, and may have limited control over chosen network components (e.g., host firewalls).

[0032] The deployment model is as follows:

[0033] Private cloud: A cloud infrastructure that runs exclusively for a single organization. It can be managed by that organization or a third party, and can exist inside or outside the organization.

[0034] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with common interests (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist inside or outside the organization.

[0035] Public cloud: Cloud infrastructure available to the general public or large industrial groups and owned by organizations that sell cloud services.

[0036] Hybrid cloud: A cloud infrastructure consisting of two or more clouds (private, community, or public) that remain distinct entities but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (e.g., cloud bursts for load balancing between clouds).

[0037] Cloud computing environments are service-oriented, characterized by statelessness, loose coupling, modularity, and semantic interoperability. The core of computing is the infrastructure comprising a network of interconnected nodes.

[0038] Now for reference Figure 1 The diagram illustrates an example of a cloud computing node. Cloud computing node 10 is merely one example of a suitable cloud computing node and is not intended to impose any limitation on the scope or functionality of the embodiments of the invention described herein. In any case, cloud computing node 10 can be implemented and / or perform any of the functions set forth above.

[0039] Within cloud computing node 10, there exists a computer system / server 12 that can operate alongside many other general-purpose or special-purpose computing system environments or configurations. Examples of known computing systems, environments, and / or configurations suitable for computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the aforementioned systems or devices.

[0040] Computer system / server 12 can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Typically, program modules can include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer system / server 12 can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via communication networks or IoT networks. In a distributed cloud computing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0041] like Figure 1 As shown, the computer system / server 12 in cloud computing node 10 is shown in the form of a general-purpose computing device. The components of the computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that couples various system components, including system memory 28, to the processor 16.

[0042] Bus 18 represents one or more of several types of bus architectures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. By way of example and not limitation, these architectures include the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0043] Computer system / server 12 typically includes various computer system readable media. Such media can be any available media accessible to computer system / server 12, and it includes volatile and non-volatile media, removable and non-removable media.

[0044] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be provided for reading from and writing to non-removable non-volatile magnetic media (not shown, commonly referred to as a "hard disk drive"). Although not shown, however, disk drives for reading from and writing to removable non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs, or other optical media) may be provided. In this case, each may be connected to bus 18 via one or more data media interfaces. As will be further described below, system memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the present invention.

[0045] A program / utility 40 having at least one set of program modules 42, along with (by way of example and not limitation) an operating system, one or more applications, other program modules, and program data, may be stored in system memory 28. Each of the operating system, one or more applications, other program modules, and program data, or some combination thereof, may include an implementation of a networking environment. Program modules 42 typically perform the functions and / or methods of the embodiments of the invention described herein.

[0046] Computer system / server 12 can also communicate with one or more external devices 14 (such as a keyboard, indicating device, display 24, etc.); one or more devices that enable a user to interact with computer system / server 12; and / or any device that enables computer system / server 12 to communicate with one or more other computing devices (e.g., network interface card, modem, etc.). This communication can occur via input / output (I / O) interface 22. However, computer system / server 12 can communicate with one or more networks (such as local area networks (LANs), general-purpose wide area networks (WANs), IoT networks, and / or public networks (e.g., the Internet)) via network adapter 20. As shown, network adapter 20 communicates with other components of computer system / server 12 via bus 18. It should be understood that, although not shown, other hardware and / or software components can be used in conjunction with computer system / server 12. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0047] Now for reference Figure 2 The illustration depicts a cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 to which local computing devices used by cloud consumers can communicate, such as personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, smart mattresses 54D, and / or automotive computer systems 54N. As used herein, a mattress (e.g., mattress 5D) can be a pad, mat, foam, or an object designed to support all or part of the body or to allow the body to recline in a reclining position, such as a bed (or part of a bed), sofa, recliner, chair, or seat.

[0048] Nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks (such as private clouds, community clouds, public clouds, or hybrid clouds or combinations thereof as described above). This allows cloud computing environment 50 to provide Infrastructure as a Service, Platform as a Service, and / or Software as a Service, without requiring cloud consumers to maintain resources for them on their local computing devices. It should be understood that... Figure 2 The various computing devices 54A-N shown are merely illustrative. The computing node 10 and the cloud computing environment 50 can communicate with any type of computing device over any type of network and / or network-addressable connection (e.g., using a web browser).

[0049] Now for reference Figure 3 This demonstrates a cloud computing environment of 50 ( Figure 2 This provides a set of functional abstraction layers. First, it should be understood that... Figure 3 The components, layers, and functions shown are merely illustrative, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0050] Device layer 55 includes physical and / or virtual devices embedded with individual electronics, sensors, actuators, and other objects to perform various tasks within the cloud computing environment 50. Each device in device layer 55 integrates networking capabilities with other functional abstraction layers, enabling information obtained from the device to be provided to these layers, and / or information from other abstraction layers to be provided to the device. In one embodiment, the various devices, including device layer 55, may be incorporated into a physical network collectively referred to as the “Internet of Things” (IoT). As those skilled in the art will understand, such a physical network allows data to communicate, be collected, and disseminated to achieve various purposes.

[0051] As shown in the figure, device layer 55 includes sensors 52, actuators 53, a "learning" thermostat 56 with integrated processing, sensors, and networked electronics, a camera 57, a controllable household socket / outlet 58, and a controllable electrical switch 59. Other possible devices may include, but are not limited to, various additional sensor devices, networked devices, electronic devices (such as remote control devices), additional actuator devices, so-called "smart" appliances (such as refrigerators or washing machines / dryers), and a wide variety of other possible interconnected objects.

[0052] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) based server 62; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0053] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0054] In one example, management layer 80 may provide the following functionalities: Resource provisioning function 81 provides dynamic acquisition of computing resources and other resources for performing tasks in the cloud computing environment. Metering and pricing function 82 provides cost tracking for the use of resources in the cloud computing environment and provides bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security function provides authentication for cloud consumers and tasks and protection for data and other resources. User portal function 83 provides access to the cloud computing environment for consumers and system administrators. Service level management function 84 provides cloud resource allocation and management to meet required service levels. Service level agreement (SLA) planning and enforcement function 85 provides pre-scheduling and procurement of cloud resources according to the SLA for its projected future needs.

[0055] Workload layer 90 provides examples of functions that can leverage a cloud computing environment. Examples of workloads and functions that can be provided in this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom instruction provision 93; data analytics processing 94; transaction processing 95; and, in the context of the illustrated embodiments of the invention, various workloads and functions 96 for providing direct access between the accelerator and non-volatile memory. Furthermore, workloads and functions 96 for providing direct access between the accelerator and non-volatile memory may include operations such as data analytics (including data collection and processing from various environmental sensors), networking, sending / receiving data, providing virtualization / virtual computing, cloud computing communications, and / or management functions. Those skilled in the art will understand that workloads and functions 96 for providing direct access between the accelerator and non-volatile memory can also work in conjunction with other parts of various abstraction layers (such as hardware and software 60, virtualization 70, management 80, and other workloads 90 such as data analytics processing 94) to achieve the various objectives of the illustrated embodiments of the invention.

[0056] As previously described, the mechanism of this invention provides a novel method for again providing direct access between accelerators and non-volatile memory by a processor in a computing environment. Direct access to non-volatile memory can be provided to one or more accelerators (e.g., graphics processing units "GPUs") via application programming interfaces ("APIs"), independent of the host central processing unit ("CPU") on the control or data path, thereby performing data read and write operations.

[0057] Turn now Figure 4 A block diagram depicting exemplary functional components 400 according to various mechanisms of the illustrated embodiments is shown. In one aspect, in Figures 1 to 3 One or more of the components, modules, services, applications, and / or functions described herein may be available. Figure 4 The direct data access service 410 is shown, which, in conjunction with a processing unit 420 (“processor”), performs various computational, data processing, and other functions according to aspects of the invention. In one aspect, the processor 420 and memory 430 may be internal to and / or external to the direct data access service 410, and internal to and / or external to the computing system / server 12. The direct data access service 410 may be included in the computer system / server 12, such as... Figure 1 As described in [the text].

[0058] In one aspect, the computer system / server 12 may communicate with one or more accelerators (e.g., GPU 492 (shown by way of example only)) and non-volatile memory 490 (e.g., a solid-state drive "SDD"). In one aspect, GPU 492 may be included on the computer system / server 12 (e.g., located inside and / or outside the computer system / server 12). Additionally, non-volatile memory 490 may be included on the computer system / server 12 (e.g., located inside and / or outside the computer system / server 12), and / or non-volatile memory 490 may be located outside the computer system / server 12.

[0059] Processing unit 420 can communicate with memory 430. Direct data access service 410 may include direct data access component 440, page caching component 450, prefetch component 460 and / or object management component 470.

[0060] As those skilled in the art will understand, the depiction of the various functional units in the Direct Data Access Service 410 is for illustrative purposes, as the functional units may be located within the Direct Data Access Service 410 or elsewhere within and / or between distributed computing components.

[0061] In one aspect, the Direct Data Access Service 410 can provide direct access to a non-volatile storage device to one or more accelerators via an application programming interface (“API”), independent of the host central processing unit (“CPU”) on the control or data path, thereby performing data read and write operations.

[0062] Direct Data Access Service 410 can map one or more non-volatile memory fast (“NVMe”) queues and registers of non-volatile memory 490 in the memory address space of one or more accelerators.

[0063] Page caching component 450 may use page caches managed by one or more accelerators in the memory of one or more accelerators to take advantage of locality of data. In the presence of multiple accelerators (e.g., multiple GPUs), page caching component 450 may use distributed page caches managed by one or more accelerators in the memory of multiple accelerators to take advantage of locality of data.

[0064] Page caching component 450 can copy one or more pages from non-volatile memory 490 to the page cache and update the page mapping table. Direct data access service 410 can remove the host CPU from the control path or data path when copying data from non-volatile memory 490 to one or more accelerators (such as GPU 492).

[0065] The prefetch component 460 can prefetch data using the API and save each computation cycle to overlap computation and data movement.

[0066] The Direct Data Access Service 410 associated with the page cache component 450 and / or the prefetch component can determine the occurrence of a page cache miss upon receiving one or more API calls initiated by one or more accelerators for data read or write operations, copy one or more pages from one or more non-volatile memory fast (“NVMe”) data buffers to the page cache, and / or update the page mapping table.

[0067] The object management component 470, located in the host CPU (e.g., computer system / server 12, and may be internal and / or external to GPU 492), can manage the object address of each object in non-volatile memory 490 at application startup. A mapping of each object identifier (“ID”), object starting block address, and object size can be managed by the host CPU and can be persistently maintained in a reserved area of ​​non-volatile memory 490. Operations at 800 can use one or more API calls to allocate page subranges of each object to one or more accelerators.

[0068] For example, at the start of the application, the host CPU (e.g., computer system / server 12) uses an API call (e.g., the "get_object_map" API call) to read the map. During kernel calls, instead of passing pointers, the host CPU (e.g., computer system / server 12) passes the starting block address of each object, along with its size. The GPU 492 can then determine the location of the desired data structure. The host CPU (e.g., computer system / server 12) can use another API call (e.g., the "pcreate" API call) to create the space and the mapping of the new objects. A GPU (e.g., one of GPUs 492) can be assigned a page subrange to each object. A GPU (e.g., one of GPUs 492) can be referred to as the home of these pages (e.g., if a GPU owns a page, it is the home GPU for that page).

[0069] Now go to Figures 5A to 5C A block diagram depicts an exemplary function 500 involving providing direct access between an accelerator and non-volatile memory. As shown, various functional blocks are depicted with arrows, indicating the relationships between blocks 500 and illustrating the processing flow. Additionally, descriptive information associated with each functional block 500 (or “action step” or simply “step”) can be seen. It will be observed that, in conjunction with the previously... Figures 1 to 4In the same descriptive sense as described herein, many functional blocks can also be considered as “modules” of functionality. In light of the foregoing, module 500 can also be incorporated into various hardware and software components of the system according to the invention for providing direct access between the accelerator and non-volatile memory. Many functional blocks 500 can execute as background processes on various components (in distributed computing components, on user devices, or elsewhere, but generally not known to the user). For the sake of brevity, repetitive descriptions of similar elements, components, modules, services, applications, and / or functions employed in other embodiments described herein are omitted.

[0070] As mentioned above, Figure 5A The description describes how GPU 510 relies on a host file system "FS" (e.g., host CPU 530 and CPU dynamic random access memory "DRAM") to manage its memory and on non-volatile memory device 520 to provide data to GPU 510. This not only increases data copying overhead but also increases control overhead in the form of synchronization between the high-throughput accelerator and CPU 530. The host CPU 530 causes memory management overhead between GPU 510 and non-volatile memory 520 in steps 1 and 2. Multiple copies of data can be transferred / fed between non-volatile memory device 520 and CPU 530 in step 3 and between GPU 510 and CPU 530 in step 4. Furthermore, GPU threads, blocks, and grids can be stopped until data is transferred to GPU 510 as in step 4. It should be noted that there are attempts to reduce data copying overhead by performing direct memory access (“DMA”) directly between the GPU 510 and the non-volatile storage device 520, but this still results in CPU control / synchronization overhead from the CPU 530.

[0071] However, using various embodiments of the present invention, Figure 5B Operations for providing direct access between the accelerator and non-volatile memory using a single accelerator (e.g., a single GPU510) are described. That is, Figure 5B This describes a method to reduce access overhead / latency and allow GPU threads of GPU 510 to directly access non-volatile storage device 520 independently of CPU 530. In one aspect, GPU 510 manages non-volatile storage device 520 using a file system “FS” (similar to simple object storage) and utilizes an NVMe driver running on GPU 510 (e.g., using GPU Direct). CPU 530 is removed from the control path, as in step 1, and from the data path, as in step 2. GPU 510 then has access to the entire non-volatile storage device 520.

[0072] More specifically, Figure 5B Operations are described that issue read and write requests from a single accelerator (such as GPU 510) to reduce access overhead / latency and allow GPU threads of GPU 510 to directly access non-volatile storage device 520 independently of CPU 530. In one aspect, the API enables GPU threads of GPU 510 to read and write data on non-volatile storage device 520 using NVMe without a host CPU 530 on the control or data path, as shown in the following steps.

[0073] In step 1, the host CPU 530 may initialize the interface between the GPU and the non-volatile memory device 520 (e.g., queues and DMA) (via NVMe and DMA mappings such as GPUDirect / DirectGMA). In step 2, the host CPU 530 may initiate the transfer of object mappings from the non-volatile memory device 520 to the GPU memory of the GPU 510, so that the GPU thread knows the location of each object on the non-volatile memory device 520. In step 2(1), entries in the mapping may be key-value pairs, where the object identifier (“ID”) is the key and the starting address on the non-volatile memory device 520 is the value. In step 2(2), the application of the host CPU 530 may indicate to the GPU 510 which object to use by passing the appropriate object ID as a parameter to the GPU kernel startup. In step 3, the host CPU 530 may start the GPU kernel in a normal application other than passing the object ID as a parameter (from step 2(2)).

[0074] In step 4, the GPU kernel thread can use one or more API calls (such as the provided "pread" and "pwrite" API calls) to read and / or write data on the non-volatile memory device 520.

[0075] In step 5, when the GPU thread issues a pread API call, the GPU thread (e.g., the thread that calls the API and is checked by the GPU thread) can check the GPU page cache to determine whether the requested page is in the GPU memory of GPU 510. In step 5(1), if the page is in the memory of GPU 510, the data is copied from the page cache to a user-allocated destination buffer (e.g., the destination buffer may be a user-allocated buffer in GPU memory that can be read and written by the user and can be allocated using one or more APIs). In step 5(2), if the page (or a subset of the page) is not in the GPU page cache, the GPU thread making the call can prepare an NVMe command for the page that was not hit, as in step 5(2)(1); atomically add the NVMe command to the NVMe submission queue and ring the NVMe doorbell, as in step 5(2)(2), while not using the CPU 530 in the control path, as in step 5(2)(2)(1); poll the NVMe completion queue until the request is completed, as in step 5(2)(3), where data between the GPU 510 and the NVMe device is copied directly between them without the intervention of the CPU 520, as in step 5(2)(3)(1); and copy data from the NVMe IO buffer to the user-provided buffer, as in step 5(2)(4), and update the page cache state, as in step 5(2)(5).

[0076] Additionally, in step 6, when the GPU thread issues the pwrite API, the calling thread can: update the page cache state, as in step (6)(1); prepare an NVMe command to write the page to the non-volatile storage device, as in step (6)(2); atomically add the command to the NVMe commit queue and ring the NVMe doorbell, as in step (6)(3) (e.g., without using CPU 530 in the control path), as in step 5(2)(2)(1); poll the NVMe completion queue until the request is completed, as in step (6)(4) (e.g., data between GPU 510 and the NVMe device is copied directly between them without the intervention of CPU 520), as in step 5(2)(3)(1); and release its buffer, as in step (6)(7).

[0077] In step 7, the GPU thread can issue read and write API calls, however it wants to do so multiple times and can repeat steps 1(5) and 1(6) respectively.

[0078] In the case of multiple GPUs, Figure 5C Depicts the provision of multiple GPUs 510 (e.g., Figure 5C 510 in the diagram describes the operation of direct access between a group of GPUs and non-volatile memory 520. Before accessing NVMe, the GPUs 510 exchange data with each other to verify whether the data already exists in different GPU caches.

[0079] In one aspect, the object page is hashed to one or more specific GPUs 510. A GPU 510 may own the object page, in which case the object page may reside in the cache of one or more specific GPUs 510; or the GPU 510 may be responsible for retrieving the object page. Furthermore, a GPU 510 may request the page from the owning GPU within a group of GPUs 510.

[0080] Now go to Figure 6 Block diagram 600 depicts exemplary operations for organizing and using the accelerator page cache and the application programming interface (“API”). In one aspect, in Figures 1 to 5A - Figure 5C Each of the devices, components, modules, operations, and / or functions described herein may also be applied to or performed. Figure 6 One or more operations or actions.

[0081] In step 1, API calls (e.g., pread API calls and / or pwrite API calls) may be published to the GPU page cache (“gpcache”) in GPU memory 620. In step 2, upon receiving one or more API calls (e.g., pread API calls and / or pwrite API calls) initiated by the one or more accelerators (e.g., GPU thread 610) for reading or writing data, a page cache miss is determined to have occurred, and an NVMe request is submitted to NVMe submission queue 624 for access to NVMe device 630. In step 3, polling for NVMe requests using NVMe device 630 is completed and provided to NVMe completion queue 626.

[0082] In step 4, the page can be copied from the NVMe data buffer 628 to the page cache 622, and the page / object mapping table can be updated.

[0083] When using a single accelerator (e.g., Figure 5B In cases where a single GPU (510) provides direct access between the accelerator and non-volatile memory, page caches in GPU memory (e.g., application / software managed page caches) can be used to perform the following steps to take advantage of locality in the data.

[0084] In step 1), a page table may be maintained in GPU memory 620 (e.g., GPU global memory), which stores page mappings from object data to GPU global memory addresses in GPU memory 620 (e.g., GPU global memory). In steps (1) and (1), the mapping exists only if the page is located in GPU memory 620.

[0085] In step 2, page caches can be maintained in the Least Recently Used (“LRU”) list. In step 2(1), if there are insufficient free pages (e.g., not enough free pages) when a read or write API call is made, an LRU page can be written back in the case of being dirty, unmapped from the previous mapping, and remapped for the new mapping.

[0086] In step 3(1), during the pread API call, each requested page can be checked in the page cache. If the page is in the page cache, the data can be copied from there to a user-provided buffer, as in step 3(2). In step 3(2), an input / output (“I / O”) request can be created for a page not located in page cache 622. In step 3(2)(1), a mapping can be created in the page table for the newly requested page.

[0087] In step 4(1), upon the pwrite API call, pages from the LRU list of the page cache are allocated, data from the user-specified buffer is copied to these pages, and the page mapping for the destination page in the object on the non-volatile storage device 630 is updated to point to the appropriate page. In step 4(2), when the write I / O request completes, a bit can be set in the page table entry for (one or more) pages. In step 4(2)(1), an additional write-back operation (from step 2(2)(4)(1)) can be performed in the event of recycling. In step 5, one or more read / write operations can be performed.

[0088] It should be noted that when using multiple accelerators (e.g., using distributed page caching for multi-GPU computing environments), Figure 5C In cases where a group of GPUs (510) provides direct access between the accelerator and non-volatile memory, page caches in GPU memory (e.g., application / software managed page caches) can be used to perform the following steps to take advantage of locality in the data.

[0089] In step 1, a GPU (e.g., GPU 510 in Figure 510) may be assigned to a page subrange of an object. The GPU may be referred to as the home of those pages. In step 1(1), each GPU is also assigned a unique ID.

[0090] In step 2, the home GPU 510 maintains / preserves the mapping of those pages designated as home for its GPU 510. In step 2(1), the mapping operation may map which other GPUs cache the pages. In step 2(2), if the page is in the GPU's page cache, the GPU ID and memory address may be maintained within the page cache of that specific GPU in the group of GPUs. In step 2(3), alternatively, if the page is not in the GPU's page cache, the page is located in the non-volatile memory device 520.

[0091] In step 3, when the pread API call of GPU memory 620 cannot locate the requested page in the local page cache, GPU memory 620 submits a request to one or more home GPUs containing those pages. In step 3(1), the home GPU may check whether the page is located in the page cache of another GPU. In step 3(1)(1), if the page is located in the page cache of another GPU, GPU memory 620 responds with the GPU ID and page cache address so that the requesting GPU may copy the page(s) from that GPU's page cache; otherwise, the home GPU allows the requesting GPU to know that the page is in non-volatile memory 620 and updates its mapping table as in step 3(1)(2), and the requesting GPU submits an I / O request as in step 3(2)(1).

[0092] In step 4, when GPU thread 610 performs a pwrite operation on some pages, GPU thread 610 submits a request to one or more home GPUs for those pages. In step 4(1), the home GPU may check whether other GPUs have the pages to be overwritten in their page caches. In step 4(1)(1), if other GPUs have the pages to be overwritten in their page caches, the home GPU may invalidate all those pages in the page caches of those alternative GPUs and update its own mapping. In step 4(1)(2), if other GPUs do not have the pages to be overwritten in their page caches, the home GPU informs the requesting GPU that the pages are in non-volatile memory 630 and updates the mapping table after the requesting GPU completes its I / O request. In step 4(1)(2)(1), the requesting GPU may submit an I / O request.

[0093] In step 5, each thread on a GPU can be dedicated to handling requests from other GPUs.

[0094] It should be noted that one or more operations can be used to perform the prefetch operation as described herein (e.g., Figures 4 to 5B - Figure 5C For example, consider the following two operations used to perform a prefetch operation.

[0095] In prefetch operation 1, the following operations for prefetching data and transferring data between thread blocks can be performed as follows. In step 1, the GPU (such as GPU 510) can launch one or more thread blocks (hereinafter referred to as I / O blocks), which are only responsible for issuing corresponding I / O requests for those blocks using the pread_prefetch API call.

[0096] In step 2, the thread in the I / O block can create and submit NVMe requests. In step 2(1), all the same actions performed in steps 1(5)(2)(1) up to 1(5)(2)(2) for read and write requests from a single GPU can be performed. In step 2(2), the thread does not poll for completion (e.g., asynchronously). In step 2(3), the thread can create a mapping for I / O requests to NVMe command IDs. This mapping is later used by the GPU thread when polling the completion queue to find a completion entry for a command submitted for prefetching.

[0097] In step 3, when the actual compute thread block is scheduled and the GPU thread issues a prefetch API call, the mapping can first be checked to see if a request has already been made, as in step 3(1). If a request has been made / made, the GPU thread uses the consume_prefetch_pread API call to poll the NVMe command ID on NVMe completion queue 626, as in step 3(2); otherwise, it continues, as in step 3(3) (similar to steps 1(5)(2)(4) for executing read and write requests from a single GPU). In this way, one or more cycles of the GPU streaming multiprocessor (“SM”) are not wasted waiting for I / O completion.

[0098] In prefetch operation 2, the following operations for prefetching data and overlapping computation and data movement can be performed as follows. In step 1, each GPU thread block can be multiplexed to perform computations for multiple thread blocks.

[0099] In step 2, the GPU thread block can process multiple thread blocks mapped to it by pipelined these blocks.

[0100] In step 3, the GPU thread block can use the pread_prefetch API call to initiate an I / O request for a block.

[0101] In step 4, a GPU thread block can use the data in its block's buffer to perform computations against another thread block.

[0102] In step 5, the GPU thread block can use the consume_prefetch_pread API call to consume data in the block's buffer for the I / O request submitted for the third block. In step 5(1), the I / O request for steps 3(2)(3) described herein may overlap with the computation in steps 3(2)(4).

[0103] In step 6, each GPU thread block may execute steps 3(2)(2) up to 3(2)(5) to use the distributed page cache for the multi-GPU computing environment until all thread blocks are exhausted.

[0104] Now go to Figures 7A-7C Table 700 depicts a description of application programming interface (“API”) calls used for the GPU. In one aspect, in Figures 1 to 6 Each of the devices, components, modules, operations, and / or functions described herein can also be used. Figures 7A to 7C One or more API calls.

[0105] The “initialize()” API call can be used to establish GPUDirect / DirectGMA, create NVMe queues, map NVMe queues in GPU memory, and retrieve / obtain pointers and statuses of NVMe queues to be passed to the GPU.

[0106] The “get_object_map()” API call can read an object map from non-volatile memory, allowing the GPU thread to know the starting location of each object on that memory. The returned map can then be passed to each GPU core that made the call.

[0107] The API call "int pcreate(size_t n_bytes)" creates a new object of size "n_bytes" bytes on a non-volatile storage device and adds the new object to the object map. It can return the object ID.

[0108] The `size_t pread(int obj_id, size_t offset, size_t n_bytes, char* buffer)` API call can read `n_bytes` bytes from an object with the object ID `obj_id` on non-volatile storage, starting at the specified `offset`, and store the read data in a user-provided buffer. That is, the object ID (e.g., `obj_id`) can be a unique identifier for the object. Each object stored in non-volatile memory can have a unique identifier so that it can be used to specify the object when using the generic API class `pread`. The object ID (e.g., `obj_id`) can be a file descriptor or a file path.

[0109] The “int pread_prefetch(int obj_id, size_t offset, size_t n_bytes)” API call submits an I / O request for pread and sets the mapping between the I / O request and the NVMe command ID.

[0110] The API call “size_tconsume_prefetch_pread(int ID,char*buffer,size_t n_bytes)” waits until the specified command is completed and the result is consumed in the specified buffer.

[0111] The API call “size_t pwrite(int obj_id, size_t offset, size_t n_bytes, char*buffer)” can write “n_bytes” bytes, starting at the offset, to a non-volatile storage device from the object with the object id “obj_id”.

[0112] The "flush()" API call can clear outdated entries in the NVMe queue and wait until all NVMe write commands are completed.

[0113] Now go to Figure 8 This describes a method 800 for providing direct access between an accelerator (e.g., a graphics processing unit "GPU") and non-volatile memory in a computing environment. In one aspect, in... Figure 1 Each of the devices, components, modules, operations, and / or functions described in Figure 7 may also be applied to or performed. Figure 8One or more operations or actions. Function 800 may be implemented as a method of execution as instructions on a machine, wherein these instructions are included on at least one computer-readable medium or a non-transitory machine-readable storage medium. Function 800 may begin in block 802.

[0114] As in box 804, one or more accelerators can be provided with direct access to non-volatile memory via an application programming interface (“API”) independent of the host central processing unit (“CPU”) on the control or data path, thereby performing data read and write operations. Function 800 may end, as in box 806.

[0115] In one respect, combination Figure 8 At least one box and / or as part of it, operation 800 may include one or more of each of the following: Operation 800 may map one or more non-volatile memory fast (“NVMe”) queues and registers in the memory address space of one or more accelerators. Operation 800 may utilize page caches managed by one or more accelerators in the memory of one or more accelerators to take advantage of locality in data, and / or utilize distributed page caches managed by one or more accelerators in the memory of multiple accelerators to take advantage of locality in data.

[0116] Operation 800 can copy one or more pages from non-volatile storage to the page cache and update the page mapping table. Operation 800 can remove the host CPU from the control path or data path while copying data from non-volatile storage to one or more accelerators. Operation 800 can prefetch data using an API and save computation cycles to overlap computation and data movement. Operation 800 can determine the occurrence of a page cache miss upon receiving one or more API calls initiated by the one or more accelerators for read or write operations on data, copy one or more pages from one or more non-volatile memory fast (“NVMe”) data buffers to the page cache, and / or update the page mapping table.

[0117] The 800 operation can receive the object address of each object in non-volatile storage when the application starts. The mapping of each object identifier (“ID”), object starting block address, and object size can be managed by the host CPU and persisted in a reserved area of ​​the non-volatile storage. The 800 operation can use one or more API calls to allocate page subranges of each object to one or more accelerators.

[0118] This invention can be a system, method, and / or computer program product. A computer program product may include one or more computer-readable storage media having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0119] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or recessed structures on which instructions are recorded, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0120] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the respective computing / processing device.

[0121] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuits in order to perform aspects of this invention.

[0122] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0124] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus or other device perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions indicated in the blocks may occur in a non-consecutive order as shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

Claims

1. A method for providing direct access to non-volatile memory by a processor in a computing environment, comprising: providing direct access to non-volatile storage to one or more accelerators via an application programming interface (API) to perform read and write operations of data, wherein the direct access is independent of a host CPU on a control path or a data path; and wherein the host CPU initiates object mapping of a transfer from the non-volatile storage to the one or more accelerators, wherein the object mapping identifies objects on the non-volatile storage.

2. The method of claim 1, further comprising: mapping one or more non-volatile memory queues and registers in a memory address space of the one or more accelerators.

3. The method of claim 1, further comprising: using a page cache in memory of the one or more accelerators to exploit locality in the data, wherein the page cache is managed by the one or more accelerators.

4. The method of claim 1, further comprising: using a distributed page cache in memory of the one or more accelerators to exploit locality in the data, wherein the distributed page cache is managed by the one or more accelerators.

5. The method of claim 1, further comprising: copying one or more pages from the non-volatile storage to the page cache and updating a page mapping table; or removing the host CPU from the control path or the data path while copying the data from the non-volatile storage to the one or more accelerators.

6. The method of claim 1, further comprising: prefetching the data with the API and saving compute cycles to overlap computation and movement of the data.

7. The method of claim 1, further comprising: determining an occurrence of a page cache miss upon receiving one or more API calls initiated by the one or more accelerators for the read or write operations of the data; copying one or more pages from one or more non-volatile memory data buffers to the page cache; and updating a page mapping table.

8. A system for providing direct access to non-volatile memory, comprising: one or more computers having executable instructions that, when executed, cause the system to: provide direct access to non-volatile storage to one or more accelerators via an application programming interface (API) to perform read and write operations of data, wherein the direct access is independent of a host CPU on a control path or a data path; and wherein the host CPU initiates object mapping of a transfer from the non-volatile storage to the one or more accelerators, wherein the object mapping identifies objects on the non-volatile storage.

9. The system of claim 8, wherein, the executable instructions map one or more non-volatile memory queues and registers in a memory address space of the one or more accelerators.

10. The system of claim 8, wherein, the executable instructions use a page cache in memory of the one or more accelerators to exploit locality in the data, wherein the page cache is managed by the one or more accelerators.

11. The system of claim 8, wherein, The executable instructions use a distributed page cache in memory of the one or more accelerators to exploit locality in the data, where the distributed page cache is managed by the one or more accelerators.

12. The system of claim 8, wherein, The executable instructions: copy one or more pages from the non-volatile storage to a page cache and update a page map table; or remove the host CPU from the control path or the data path while copying the data from the non-volatile storage to the one or more accelerators.

13. The system of claim 8, wherein, The executable instructions prefetch the data with the API and save per compute cycle to overlap computation and movement of the data.

14. The system of claim 8, wherein, The executable instructions: determine occurrence of a page cache miss upon receiving one or more API calls initiated by the one or more accelerators for the read or write operation of the data; copy one or more pages from one or more non-volatile memory data buffers to a page cache; and update a page map table.

15. A computer program product for providing direct access to non-volatile memory by a processor, the computer program product comprising computer readable program code portions, the computer readable program code portions comprising: an executable portion that provides direct access to non-volatile storage to one or more accelerators via an application programming interface (API) to perform read and write operations of data, where the direct access is independent of a host CPU on a control path or a data path; and where the host CPU initiates object mapping of a transfer from the non-volatile storage to the one or more accelerators, where the object mapping identifies an object on the non-volatile storage.

16. The computer program product of claim 15, further comprising: an executable portion that maps one or more non-volatile memory queues and registers in a memory address space of the one or more accelerators.

17. The computer program product of claim 15, further comprising: an executable portion that: uses a page cache in memory of the one or more accelerators to exploit locality in the data, where the page cache is managed by the one or more accelerators; or uses a distributed page cache in memory of a plurality of accelerators to exploit locality in the data, where the distributed page cache is managed by the one or more accelerators.

18. The computer program product of claim 15, further comprising: an executable portion that: copies one or more pages from the non-volatile storage to a page cache and updates a page map table; or removes the host CPU from the control path or the data path while copying the data from the non-volatile storage to the one or more accelerators.

19. The computer program product of claim 15, further comprising: an executable portion that prefetches the data with the API and saves per compute cycle to overlap computation and movement of the data.

20. The computer program product of claim 15, further comprising: an executable portion that: determines occurrence of a page cache miss upon receiving one or more API calls initiated by the one or more accelerators for the read or write operation of the data; copies one or more pages from one or more non-volatile memory data buffers to a page cache; and updates a page map table.

Images