Method, apparatus, and computer readable storage medium for distributed systems

Abstract

According to example implementations, a method includes receiving, by a connector associated with a compute node and associated with a fabric-attached memory (FAM), an application programming interface (API) invoked to perform an operation associated with a hierarchical data format (HDF) object of an HDF file. The API call includes an HDF object identifier corresponding to the HDF object. The method includes, in response to the request, accessing, by the connector, mapping information stored in the FAM based on the HDF object identifier, and identifying, by the connector using the mapping information, a FAM descriptor corresponding to a first data item stored in the FAM and corresponding to the HDF object. The method includes, in response to the request, servicing, by the connector, the API call in response to the identification of the FAM descriptor.

Description

Background Technology

[0001] Distributed systems comprise networked computing nodes that coordinate processing activities to achieve a common goal. As an example, a high-performance computing (HPC) cluster is a distributed system that performs parallel processing to solve computationally complex problems. Other examples of distributed systems include database management systems, file storage systems, content delivery systems, software-defined servers (SDS), and cloud computing systems. Attached Figure Description

[0002] Figure 1 This is a block diagram of a distributed system with a hierarchical data format (HDF) file input / output (I / O) interface according to an example implementation, the interface serving application programming interface (API) calls to HDF files stored in a structured attached memory (FAM) pool.

[0003] Figure 2 This is a diagram illustrating the relationship between HDF file objects and FAM data items according to an example implementation.

[0004] Figure 3 This is a diagram of a trie structure that maps HDF object identifiers to FAM data item descriptors according to an example implementation.

[0005] Figure 4 It is a flowchart depicting the message passing and actions taken by the HDF file I / O interface according to an example implementation.

[0006] Figure 5 It is a flowchart depicting the process performed by the FAM connector to provide services for HDF file API calls according to an example implementation.

[0007] Figure 6 It is a flowchart depicting a technique, according to an example implementation, for servicing API calls to perform operations associated with HDF objects in an HDF file.

[0008] Figure 7 This is a block diagram of an apparatus having a processor according to an example embodiment, the processor being configured to service requests to perform operations on HDF files stored in a region of FAM.

[0009] Figure 8 This is an illustration of machine-readable instructions according to an example implementation, which enable a machine to access a data item corresponding to an HDF object identifier contained in a request to perform operations associated with an HDF file. Detailed Implementation

[0010] The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. However, it should be clearly understood that these drawings are for illustrative and descriptive purposes only. While several examples are described in this document, modifications, adaptations, and other embodiments are possible. Therefore, the following detailed description does not limit the disclosed examples. Rather, the appropriate scope of the disclosed examples may be defined by the appended claims.

[0011] The terminology used herein is for the purpose of describing particular examples only and is not intended to be restrictive. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “a plurality” is defined as two or more. As used herein, the term “another” is defined as at least a second or more. As used herein, the term “connection” is defined as a direct connection without any intermediate elements, or an indirect connection with at least one intermediate element, unless otherwise stated. Two elements may be mechanically coupled, electrically coupled, or linked by communication channels, paths, networks, or systems. As used herein, the term “and / or” refers to and covers any and all possible combinations of the associated enumeration. It will also be understood that although the terms first, second, third, etc., may be used herein to describe various elements, these elements should not be limited by these terms, as these terms are used only to distinguish one element from another, unless otherwise stated or indicated by the context. As used herein, the term “includes” means including but not limited to, and the term “including” means including but not limited to. The term "based on" means at least partially based on.

[0012] Computational nodes in a distributed system can generate and manipulate large amounts of complex, heterogeneous data. For example, a distributed system performing parallel processing can manipulate and generate data representing arrays of numbers, which can have various data types and formats. Distributed systems can store their data as files in persistent storage. Hierarchical Data Format (HDF) files, such as HDF version 4 (HDF4) or HDF version 5 (HDF5) files, support large, complex, portable, and heterogeneous data. The HDF5 file format specification is available from the HDF Group and is described in HDF5 file format specification version 3.0, which can be found at: https: / / docs.hdfgroup.org / hdf5 / develop / _f_m_t3.html.

[0013] HDF files can be viewed as file systems contained within and described within a single file. More specifically, HDF files contain an object hierarchy (referred to herein as the "HDF hierarchy") that includes two main object types: group objects and dataset objects. Group objects can be viewed as directories (e.g., UNIX directories) that can host one or more HDF objects (e.g., other group objects, dataset objects, and other objects) that are descendants of group objects. The starting point or root of the HDF hierarchy is a group object, which represents a file. Dataset objects store data elements.

[0014] HDF files can contain objects other than group objects and dataset objects, such as data type objects describing the data element types of dataset objects, and data space objects defining the logical layout or organization of the data elements in a dataset object. Attribute objects contain metadata describing the properties of group objects, dataset objects, or data type objects. For example, an attribute object might contain metadata describing the association of a specific group object with a specific project, or metadata describing a specific dataset as sensor data collected at a specific time and location.

[0015] In one approach, the distributed system may store HDF files in a hard disk drive (HDD) based storage device, such as a storage device that includes HDDs attached to compute nodes or HDDs in a distributed file storage system (e.g., a storage area network (SAN)). In another approach, the distributed system may store HDF files in a shared memory pool (e.g., a persistent memory pool) within a compute node-centric architecture. More specifically, depending on the compute node-centric architecture, the compute nodes have their own local memory, and the shared memory pool is enabled by the replication of data and messages passed between compute nodes. The relatively slow access times and limited bandwidth of the aforementioned methods for storing HDF files may limit the performance of HDF file input / output (I / O) operations.

[0016] According to the example implementation described herein, the HDF files of a distributed system can be stored in a Structured Attached Memory (FAM) pool (e.g., a FAM pool providing persistent storage). The FAM pool corresponds to a memory-centric architecture in which shared memory is decoupled or separate from compute nodes, allowing shared memory to scale independently of compute nodes. Compute nodes share a common view of the FAM pool, and the memory-centric architecture avoids message-based coordination of memory accesses and other performance limitations inherent in compute node-centric architectures. The FAM pool may include a memory server or node containing memory devices (e.g., non-volatile or persistent memory devices).

[0017] FAM pools can be organized into memory regions (referred to herein as "FAM regions") that function as memory blocks. A given FAM region always possesses the same properties (e.g., resilience, security, or other memory-related properties). Data within a FAM region is allocated to cells called "data items" (also referred to herein as "FAM data items"). FAM data items inherit the properties of the FAM memory region in which they reside. For access purposes, FAM memory regions and FAM data items are identified by corresponding FAM descriptors. For example, a region's FAM descriptor may specify a region identity (ID), while a specific data item's FAM descriptor may specify the region ID (of the region containing the data item) and an offset within the region corresponding to the data item's location within that region. According to some implementations, the FAM pool can be accessed via application programming interface (API) calls, such as the set of API calls defined by the OpenFAM library, which is described in "OpenFAM: A library for programming Fabric-Attached Memory" (https: / / openfam.github.io / index.html) and is available at https: / / github.com / OpenFAM / API.

[0018] According to an example implementation, the data of an HDF file can be stored in a single FAM region, which serves as a container for the HDF file. HDF objects of the HDF file can be stored as corresponding FAM data items in the FAM memory region. According to an example implementation, a given compute node in a distributed system may include an HDF file I / O interface that serves HDF API calls generated by executing an application on the compute node and target HDF files stored in a FAM pool. For this purpose, the HDF file I / O interface includes an HDF file FAM connector (referred to herein as the "FAM connector").

[0019] According to the example implementation, the FAM connector is a terminally pluggable connector operatively connected to the Virtual Object Layer (VOL) of the HDF file I / O interface. The FAM connector handles FAM pool-specific operations for creating, accessing, modifying, and deleting HDF files stored in the FAM pool. According to the example implementation, the VOL is one level lower in hierarchy than the API layer of the HDF I / O interface. The API layer, the VOL, and one or more connectors other than the FAM connector may belong to an HDF library (e.g., the HDF5 library, such as the one available at https: / / github.com / HDFGroup / hdf5).

[0020] In response to an HDF API call, the API layer checks the integrity of the API call (e.g., checks semantics), and then (assuming the integrity check passes) forwards the HDF API call to the VOL, which effectively intercepts all HDF API calls. An HDF API call that opens an HDF file stored in the FAM pool or creates an HDF file in the FAM pool selects an HDF file connector to handle the I / O operations servicing the API call. The VOL can identify the appropriate HDF file connector for a given HDF API call based on the HDF file connector selected when the specific HDF file referenced by the HDF API call is opened or created. According to an example implementation, for an HDF API call targeting an HDF file associated with the FAM pool, the VOL call resolves to a VOL callback from the FAM connector.

[0021] According to an example implementation, in response to a VOL callback, the FAM connector performs one or more Remote Memory Access (RMA) operations to service an HDF API call. An RMA operation is an example of a one-sided operation in which the source process (e.g., the process entity corresponding to the FAM connector) controls all aspects of the data transfer specified to the target process (the process entity corresponding to the FAM memory node). In the context used herein, a FAM connector “serving” an HDF API call refers to a FAM connector that performs one or more operations in response to one or more functions requested by the HDF API call. As an example, to service an HDF API call to create a dataset object in a specific HDF file, the FAM connector may generate one or more RMA operations (e.g., one or more Put operations, one or more Get operations, and / or one or more Allocate operations) before returning control to the VOL.

[0022] Generally, HDF API calls can specify or reference an identifier (referred to herein as an "HDF object identifier") of a specific HDF object (e.g., an HDF object to be created, read, modified, or deleted). As further described herein, the HDF object identifier can (according to the HDF hierarchy of the HDF file) state a specific hierarchical path of the HDF object. According to an example implementation, the data and metadata of the objects in the HDF file are contained in corresponding data items in the FAM region (e.g., a data item corresponding to the HDF object), and the FAM region acts as a container for the HDF file. According to an example implementation, the FAM data item also stores data representing distributed mapping information, which allows the FAM connector to map the HDF object identifier to the corresponding FAM descriptor of the FAM data item containing the metadata and data of the HDF object. More specifically, according to some implementations, the distributed mapping information is arranged in a distributed trie structure, which includes trie nodes stored in the corresponding data items of the FAM region.

[0023] According to some implementations, each trie node corresponds to a key-value store (KVS). A KVS is associated with a specific HDF object (referred to as the "parent HDF object" in the following discussion) and contains one or more key-value pairs. Each key-value pair is then associated with an HDF object that is a child object of the parent HDF object. In this way, the key-value pair stores the identifier (key) of the child HDF object and the descriptor (value) of a FAM data item that stores the data and metadata of the child HDF object (and possibly another KVS). For cases where the parent HDF object has multiple child objects, a given KVS can contain multiple key-value pairs. The FAM connector can traverse the trie structure, which involves accessing multiple data items (and multiple KVSs) to ultimately determine the descriptor of the data item corresponding to the HDF object referenced in the HDF API call.

[0024] refer to Figure 1 As a more specific example, according to some implementations, the distributed system 100 may include one or more computing nodes 104. Figure 1 The specific implementation described herein includes a distributed system 100 comprising N computing nodes 104 ( Figure 1The text describes N compute nodes 104-1 and 104-N. As an example, distributed system 100 can be a distributed system in which compute nodes 104 coordinate processing activities to achieve a common goal. As an example, distributed system 100 can be a high-performance computing (HPC) cluster, a parallel processing system, a database management system, a file storage system, a content delivery system, a software-defined server (SDS), a cloud-based computing system, or other systems. In the context used herein, a "compute node" can be a logical or physical entity associated with a single instance of an operating system.

[0025] As an example, compute node 104 may be an entire computer platform or a sub-part thereof. In this context, a "computer platform" refers to an electronic device having processing resources capable of executing machine-readable instructions ("software"). As an example, a computer platform may be a server computer (e.g., a blade server, rack server, or standalone server), a desktop computer, a laptop computer, a tablet computer, a smartphone, a storage array, a network switch, a wearable computer, a gateway, or another electronic device with processing resources. As an example, compute node 104 may correspond to one or more central processing unit (CPU) cores or GPU cores of the computer platform. As a more specific example, the computer platform may be a blade server, and the CPU cores of the blade server may be partitioned across multiple compute nodes 104.

[0026] As another example, compute node 104 may be a host abstraction of a single computer platform or a sub-part thereof (e.g., a container, a virtual machine, or another host abstraction). As another example, compute node 104 may be a host abstraction spanning multiple computer platforms (e.g., a single virtual machine hosted across multiple computer platforms, such as a software-defined server).

[0027] Regardless of its specific form, compute node 104 has associated hardware and software. Figure 1 The specific hardware and software components of the particular computing node 104-1 are depicted and described in detail herein. According to further embodiments, other computing nodes 104 may have similar or different architectures and / or components as computing node 104-1. Figure 1In the illustrated example implementation, the hardware may include a collection of physical processors 108 (a single processor or multiple processors). As used herein, a "collection" of items (such as a collection of physical processors 108) may refer to a single item or multiple items. Processor 108 may include a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry. As a more specific example, according to some implementations, processor 108 may be a central processing unit (CPU) core, a collection of CPU cores, a CPU semiconductor package (or "socket"), a graphics processing unit (GPU) core, a collection of GPU cores, or a GPU semiconductor package.

[0028] Computation node 104 may further include physical memory 112, which may be implemented using a collection of physical memory devices. Generally, the memory devices forming physical memory 112, as well as other memories and storage media described and mentioned herein, are examples of non-transitory machine-readable storage media. According to example embodiments, machine-readable storage media can be used for various storage-related and computation-related functions of computation node 104. As examples, memory devices may include semiconductor memory devices, flash memory devices, memristors, phase-change memory devices, magnetic memory devices, combinations of one or more of the foregoing memory technologies, and memory devices based on other technologies. Furthermore, unless otherwise stated herein, memory devices may be volatile memory devices (e.g., dynamic random access memory (DRAM) devices, static random access memory (SRAM) devices, etc.) or non-volatile memory devices (e.g., flash memory devices, read-only memory (ROM) devices, etc.).

[0029] According to an example implementation, memory 112 may store machine-readable instructions 114, which can be executed by one or more processors 108 to provide software components for computing node 104. For example, according to some implementations, instructions 114, when executed by processor(s) 108, may cause processor(s) 108 to form one or more application instances 117 (also referred to herein as "application 117"). Furthermore, instructions 114, when executed by processor(s) 108, may cause processor(s) 108 to form an operating system instance 118 and one or more Hierarchical Data Format (HDF) file input / output (I / O) interface instances (referred herein as "HDF file I / O interface 120").

[0030] More specifically, according to the example implementation, a given application 117 (at execution) can provide one or more application programming interface (API) requests or "API calls" that point to an HDF file and are processed or serviced by an HDF file I / O interface 120. For this purpose, the HDF file I / O interface 120 includes an API layer 124 that performs initial integrity processing of the HDF API calls (e.g., checking the semantics of the HDF API calls). A virtual object layer (VOL) 128 of the HDF file I / O interface 120 intercepts the HDF API calls and routes them (via callbacks) to the appropriate connector. Figure 1 As depicted, according to an example implementation, the HDF file I / O interface 120 includes a Structure Attached Memory (FAM) connector 150 and one or more other connectors 130. The other connectors 130 may be local connectors, including pass-through connectors and terminal connectors. As described herein, the FAM connector 150 is a terminal connector that serves HDF file API calls directed to HDF files to be created or already stored in a FAM pool 156 of the distributed system 100.

[0031] like Figure 1 As depicted, according to the example implementation, computing node 104 can be connected to FAM pool 156 via FAM interconnect structure 154. FAM interconnect structure 154 can be associated with one or more types of communication networks, such as Infiniband or Slingshot structures, as well as other network structures or any combination thereof.

[0032] According to the example implementation, FAM pool 156 includes M memory servers or memory nodes 168 (example memory nodes 168-1 and 168-M in...). Figure 1 (Depicted as an example). Compute node 104 can access FAM pool 156 using remote memory access (RMA) operation 140. As depicted at 159, HDF file 160 can be stored as data 157 in memory node 168. Figure 1 The text describes G HDF files 160 stored in FAM pool 156. Figure 1 HDF files 160-1 and 160-G are depicted. A given HDF file 160 may be distributed across multiple memory nodes 168, or may be stored in a specific memory node 168. According to the example implementation, FAM pool 156 may be a persistent storage device, and correspondingly, the memory devices of memory nodes 168 may be persistent or non-volatile memory devices.

[0033] Generally, HDF file 160 has HDF object 164, which has an HDF hierarchy defined by hierarchical data model 170. Hierarchical data model 170 includes objects corresponding to the root HDF object 164 (in...). Figure 1 The root HDF object (marked with "R") is the starting or root level. The root HDF object 164 corresponds to the entire HDF file 160. The root object 164 can be connected to one or more child HDF objects 164, as in the example HDF objects 164-2 and 164-3. The root object 164-1 is connected to the child HDF object via the corresponding link 166. In this way, as... Figure 1 As depicted, root object 164-1 can be connected to HDF object 164-2 via link 166, and can be connected to HDF object 164-3 via another link 166. Child HDF objects 164 of root object 164-1 can be connected to one or more other HDF objects 164 via link 166, and these HDF objects 164 can be connected to other HDF objects via link 166, and so on.

[0034] As an example, a non-root object in an HDF file (160) can include groups, datasets, data types, and data spaces. A group object can be viewed as a directory that can host any other HDF object. A dataset object stores the actual data. The storage of the actual data in a dataset can be contiguous or chunked, with chunking meaning the data is stored in multiple smaller blocks. For a chunked layout, each block maps to a data item, and block information is stored in the dataset metadata.

[0035] Generally, each object 164 can have associated metadata describing the attributes of object 164. For example, a specific object 164 can be a dataset object, and the associated metadata can describe aspects of the dataset, such as whether the dataset was acquired using a specific type of sensor, was acquired on a specific date, or descriptors for dataset blocks.

[0036] As further described herein, the data model 170 of HDF file 160 does not directly correspond to the organization or layout of FAM pool 156. To accommodate this difference, mapping information 180 is stored in FAM pool 156. More specifically, according to an example implementation, FAM connector 150 creates, manages, and uses mapping information 180 to create, modify, and delete data corresponding to HDF file 160 and HDF object 164. More specifically, according to an example implementation, mapping information 180 for a specific HDF file 160 describes the mapping between HDF file data model 170 and descriptors of the corresponding FAM data item set storing the data and metadata of HDF file 160. Thus, according to an example implementation, when FAM connector 150 creates a specific HDF file 160, FAM connector 150 creates corresponding mapping information 180 and stores mapping information 180 together with the data corresponding to HDF file 160 in FAM pool 156. FAM connector 150 uses mapping information 180 to access FAM data items corresponding to the respective HDF object 164. Furthermore, the FAM connector 150 can appropriately modify the mapping information 180 (e.g., when the HDF data object 164 is created or deleted, or when the HDF file is created or deleted).

[0037] According to an example implementation, for a given HDF API call pointing to a specific HDF file 160, the API call may reference a specific HDF object 164 of file 160. To perform one or more functions of the API call pointing to HDF object 164 (serving the API call), FAM connector 150 uses mapping information 180 to identify the FAM descriptor corresponding to the referenced HDF object 164. Generally, a FAM descriptor is a handle that identifies a specific data item in FAM memory pool 156. The FAM descriptor(s) identified by mapping information 180 further allows FAM connector 150 to access (e.g., via one or more RMA operations 140) the FAM data item corresponding to HDF object 164.

[0038] Figure 2 An example organization of data corresponding to HDF files in FAM pool 156, according to some implementation methods, is depicted. More specifically, Figure 2 It describes the objects in the HDF file 160 and their relationships according to the HDF hierarchy, and Figure 2The corresponding data item 208 of FAM pool 156 is depicted. For this specific example, HDF file 160 is mapped (as depicted by reference numeral 201) to a specific region 204-1 of FAM pool 156. According to the example embodiment, HDF file 160 is stored as a collection of data items 208 contained in region 204-1 of FAM pool 156. In this way, according to the example embodiment, region 204-1 serves as a container for HDF file 160. Figure 2 As depicted, FAM pool 156 may include one or more other areas ( Figure 2 The diagram illustrates region P 204, and specifically depicts regions 204-1 and 204-P. A particular region 204 possesses a set of properties (e.g., security properties and other properties).

[0039] For example region 204-1, HDF file 160 is mapped to a set of data items 208, which includes data item 208-1 corresponding to the root group object 212 of HDF file 160. Furthermore, for this example, the root group object 212 is connected to data type object 230, dataset object 231, and group object 232 via corresponding links 218-1, 218-2, and 218-3, respectively, through the HDF hierarchy of file 160. In other words, for this example, data type object 230, dataset object 231, and group object 232 are located at a hierarchical level above or below the root level containing root group object 212. Furthermore, for Figure 2 In the described example, group object 232 is connected to data type object 245, dataset object 246 and group object 247 via corresponding links 218-4, 218-5 and 218-6, respectively, which are located in a hierarchy below the hierarchy containing data type object 230, dataset object 231 and group object 232.

[0040] According to an example implementation, the mapping information is stored in region 204-1 in the form of key-value store (KVS) 214. According to an example implementation, each data item 208 stores a KVS 214 corresponding to a node in a trie data structure, the node mapping the identifier of the data item 208 to a corresponding FAM descriptor. As an example, according to some implementations, the trie structure may be a base-number trie structure.

[0041] Generally, the KVS 214 of a specific data item 208 contains mapping information for the data item 208 at the next HDF hierarchical level. As a more specific example, the KVS 214 of data item 208-1 stores mapping information for the FAM descriptors of data items 208-2 (which stores data of data type object 230), 208-3 (which stores data of data item 231), and 208-4 (which stores data of group object 232).

[0042] According to an example implementation, KVS 214 associates or maps specific FAM descriptors to identifiers of corresponding links 218. For example, a given HDF API call can reference a specific HDF object in an HDF file via an HDF object identifier. The HDF object identifier can then be used by the FAM connector to identify a specific next HDF object (from the current HDF object), and based on this identification, KVS 214 can be used to identify a specific FAM descriptor for a data item containing data of that HDF object. According to an example implementation, based on the HDF object identifier, the FAM connector can traverse KVS 214 to reach a specific FAM descriptor for the data item 208 corresponding to the HDF object referenced by the HDF API call. For example, a specific HDF API call may contain an HDF object identifier that specifically identifies dataset object 246. Figure 2 As depicted, data item 208-6 contains data from dataset object 246. For the purpose of obtaining a specific FAM descriptor for data item 208-6, the FAM connector can first determine the FAM identifier corresponding to data item 208-1 containing root group object 212 via a name service. The FAM connector can then access the KVS 214 of data item 208-1 and identify data item 208-4 (containing group object 232) as the next data item to be accessed based on the HDF object identifier.

[0043] Continuing the example, the access KVS 214 for data item 208-1 identifies data item 208-4. The FAM connector can then access the KVS 214 for data item 208-4 to determine the FAM descriptor corresponding to the data item containing dataset object 246 identified by the HDF object identifier. In this example, data item 208-6 contains data for dataset object 246, and the KVS 214 for data item 208-4 associates dataset object 246 with the FAM descriptor for data item 208-6. Furthermore, in this example, dataset object 246 is an HDF object referenced by an HDF API call. Using the FAM descriptor for data item 208-6, the FAM connector can then access data item 208-6 to access the corresponding data for dataset object 246.

[0044] for Figure 3 In the specific example depicted, each data item 208 contains two KVSs. As an example, a particular KVS 214 may correspond to an HDF identifier containing the link name, while another KVS 214 may correspond to a link identifier containing the link index.

[0045] Figure 3A more detailed illustration 300 depicts a trie-based mapping according to an example implementation. More specifically, illustration 300 depicts an example data item 310 storing information (e.g., data and metadata) of the corresponding HDF object 312. Figure 3 Specific HDF objects 312 are depicted, referred to as "HDF object A", "HDF object B", "HDF object C", "HDF object D" and "HDF object E". The HDF objects are arranged in the depicted HDF hierarchy such that HDF object A is at the first highest level of the depicted hierarchy; HDF objects B, C and D are at the second level below the first level of the hierarchy; and HDF object E is at the third level below the second level of the hierarchy.

[0046] A given HDF object 312 can be referenced by an HDF object identifier, which is a concatenation of identifiers of the links that are traversed (according to the HDF hierarchy) to reach the HDF object 312. Figure 3 The links 313 between HDF object A and HDF objects B, C, and D are depicted. For Figure 2 In the illustrated example implementation, the links are referenced by corresponding names. For example, HDF object A is connected to HDF object D via link 313, which has the link name "AD". If HDF object A is the root object of the HDF file, the link identifier for that object can be " / ", and as an example, the HDF object identifier for HDF object D is " / A / D", as depicted at 344. Using similar notation, HDF object A is connected to HDF object B via link 313 "AB", and the HDF object identifier for HDF object B is " / AB". HDF object B is connected to HDF object E via link 343 "BE", and the HDF object identifier for HDF object E is " / AB / BE", as depicted at 340.

[0047] Continuing the example, data item 310, containing data and metadata for HDF object A, also includes a KVS 320 with three entries or key-value pairs 321-1, 321-2, and 321-3, associated with HDF objects B, C, and D, respectively. Each key-value pair 321 contains a key 328 and a value 329. The key 328 is an identifier (e.g., a link name) for a link from HDF object A to another HDF object, and the corresponding value 329 is a FAM descriptor for data item 310, which contains information (e.g., data, metadata, and possibly another KVS store) to another HDF object connected by the link.

[0048] As a more specific example, key-value pair 321-1 contains key 328, which is the link identifier for link "AB". As a more specific example, the link identifier could be " / AB". The corresponding value 329 of key-value pair 321-1 is the FAM data item descriptor for FAM data item 310, which contains information about HDF object B (e.g., data, metadata, and another KVS storage 340).

[0049] KVS 320 also includes key-value pairs 321-2, which contain a key 328, which is a link identifier for the link "AC". As a more specific example, the link identifier could be " / AC". The corresponding value 329 of key-value pair 321-2 is a FAM data item descriptor for a FAM data item 310 containing information (e.g., data and metadata) of HDF object C. In this example, another HDF object is not connected to HDF object C, and accordingly, the FAM data item descriptor 310 of HDF object C does not contain KVS storage.

[0050] KVS 320 also includes key-value pairs 321-3, each containing a key 328, which is a link identifier for the link "AD". As a more specific example, the link identifier could be " / AD". The corresponding value 329 of key-value pairs 321-3 is a FAM data item descriptor for a FAM data item 310 containing information (e.g., data and metadata) of HDF object D. In this example, another HDF object is not connected to HDF object D, and correspondingly, the FAM data item descriptor 310 of HDF object D does not contain KVS storage.

[0051] The KVS storage 340 of data item 310 containing information about HDF object B includes key-value pairs 341, which include a key 343, the key being a link identifier for the link “AD”. The corresponding value 345 of key-value pair 341 is a FAM data item descriptor for FAM data item 310 containing information about HDF object E (e.g., data and metadata).

[0052] As an example, assuming the identifier ( / AB / BE) of HDF object E is referenced by a specific HDF API call, the FAM connector can proceed as follows to identify the FAM descriptor of data item 310 corresponding to HDF object E. For this example, assume HDF object A is the root object of the HDF file. The FAM connector accesses key-value pair 321-1 based on key-value pair 321-1 containing the link identifier (key 328) corresponding to link AB. The value 329 of key-value pair 321-1 identifies the FAM descriptor of data item 310 at the other end of link AB. The FAM connector then accesses the KVS storage, and more specifically, accesses key-value pair 341, since key 343 of key-value pair 341 contains the link identifier corresponding to link BE. More specifically, the FAM connector accesses the corresponding value 345, which is the FAM descriptor of data item 310 corresponding to HDF object 312.

[0053] Although in the foregoing example, the KVS store associates a link name (key) with a FAM data item descriptor (value), according to a further example implementation, a given KVS may associate a link index (key) with a FAM data item descriptor (value). According to some implementations, a given data item may store data representing one KVS store for the link name and another KVS store for the link index. Figure 4 The illustrated example implementation shows two KVS for each data item.

[0054] Figure 1 An HDF file I / O interface for handling HDF API calls, according to an example implementation, is described (e.g., Figure 4 The diagram 400 illustrates the communication and actions taken by the component of the HDF file I / O interface 120. More specifically, refer to... Figure 5 API layer 124 provides HDF API call 404 to VOL 128. Generally, VOL 128 intercepts the HDF API call and forwards it to FAM connector 150 for HDF files targeting the FAM memory pool. As depicted at 408, VOL 128 associates HDF API call 404 with FAM connector 150. As an example, the association of HDF API call 404 with FAM connector 150 can be based on the file access properties of the HDF file, which are created when the HDF file is created or opened. In this way, HDF API call 404 can, for example, reference an HDF file whose file access properties associate the HDF file with FAM connector 150.

[0055] VOL 128 then provides the forwarded HDF API call to the FAM connector 150, as depicted at 412. In response to the forwarded HDF API call 412, the FAM connector 150 begins the process of servicing the call. More specifically, the FAM connector 150 obtains the FAM region descriptor 414 corresponding to the HDF file, and as depicted at 420, the FAM connector 150 reads KVS data from the FAM region corresponding to the HDF file to obtain the FAM object identifier corresponding to the HDF object referenced by the HDF API call. Reading the KVS data may involve one or more RMA accesses 424. In this way, the FAM connector 150 can traverse the trie structure by reading one or more KVSs from one or more data items in the region containing the HDF file. The FAM connector 150 can then continue servicing the HDF API call based on the obtained FAM object identifier, as depicted at 428. Serving the HDF API call may include one or more additional RMA accesses 432.

[0056] Figure 1 It depicts a FAM connector (e.g., according to an example implementation) Figure 5 The FAM connector 150 executes a flowchart 500 to obtain mapping information of HDF objects when processing HDF API calls. (See reference...) Figure 6 Process 500 includes receiving (box 504) an HDF API call pointing to an HDF file associated with the FAM connector and referencing an HDF object. According to box 508, process 500 includes the FAM connector initiating an RMA acquisition operation on the FAM to read KVS data from the root data item corresponding to the HDF file.

[0057] Next, process 500 includes a sub-process 506 for the descriptor of the HDF object. Sub-process 506 includes initiating (box 508) an RMA acquisition operation on the FAM to read KVS data from the root data item corresponding to the HDF file. Sub-process 506 performs one or more iterations to obtain the FAM descriptor of the HDF object.

[0058] According to box 512, each iteration first includes identifying the descriptor of the next data item corresponding to the next link in the link path of the HDF object from the read KVS data. According to box 516, the current data item is then set as the next data item. The current data item may correspond to an HDF object referenced in an API call. In this respect, the iteration includes determining (decision box 520) whether the current data item corresponds to an HDF object. If not, according to box 524, another iteration is initialized by initiating an RMA acquisition operation on the FAM to read KVS data from the current data item, and then control returns to box 512.

[0059] Sub-procedure 506, used to determine the descriptor of the HDF object, terminates in decision box 520 by determining that the current data item corresponds to an HDF object. At this point, the descriptor of the current data item corresponds to the descriptor of the data item containing the data of the HDF object. According to box 530, procedure 500 includes servicing API calls, including initiating one or more RMA operations to access the current data item.

[0060] refer to Figure 7 According to an example implementation, process 600 includes receiving (box 604) an application programming interface (API) call by a connector associated with a compute node and with a structure-attached memory (FAM). According to some implementations, the FAM may be a persistent storage device containing persistent or non-volatile memory devices. The call is a request to perform an operation associated with an HDF object of a hierarchical data format (HDF) file. According to some implementations, the HDF file may be an HDF5 file. As an example, the HDF object may be a group object, a dataset object, or a data type object. Any of these HDF objects may have associated attribute metadata. The API call includes or references an HDF object identifier corresponding to the HDF object. As an example, the HDF object identifier may represent a link path. As an example, a link path may be represented by a reference to one or more link names. As another example, a link path may be represented by one or more indexes.

[0061] Process 600 includes: in response to a request, accessing (box 608) mapping information stored in the FAM based on the HDF object identifier by the connector. According to some embodiments, the mapping information maps the hierarchical data model of the HDF file to data items stored in regions of the FAM. According to some embodiments, the mapping information may be a trie structure. According to some embodiments, nodes of the trie structure may be stored in FAM data items. According to some embodiments, trie nodes may be represented by a key-value store. According to some embodiments, a given key-value pair in the key-value store may contain a key representing a link identifier of the HDF file and a corresponding value representing a descriptor of the FAM data item containing information about the data items referenced by the link.

[0062] Process 600 includes: in response to a request, the connector uses (box 608) mapping information to identify a FAM descriptor corresponding to a data item stored in the FAM and corresponding to an HDF object. According to an example implementation, identifying the FAM descriptor may include traversing a trie structure, where each node of the trie structure references another node, and ultimately identifying the FACM descriptor corresponding to the data item containing information about the HDF object referenced by the API call.

[0063] Process 600 further includes: in response to a request, the connector servicing the API call in response to the identification of the FAM descriptor (block 608). According to an example implementation, servicing the API call includes performing one or more functions requested by the API call. According to some implementations, servicing the API call may include performing one or more remote memory access (RMA) operations using the FAM.

[0064] refer to Figure 8 According to an example implementation, apparatus 700 includes memory 704 and processor 708. Memory 704 stores instructions 706. When executed by processor 708, instructions 706 cause processor 708 to receive a request from a virtual object layer to perform an operation on a Layered Data Format (HDF) file. The virtual object layer may be a component of an HDF file I / O interface. The request may be an API call to perform an operation associated with an HDF object of the HDF file. According to some implementations, the HDF file may be an HDF5 file. As an example, an HDF object may be a group object, a dataset object, or a data type object. Any of these HDF objects may have associated attribute metadata. API calls may include or reference an HDF object identifier corresponding to the HDF object. As an example, the HDF object identifier may represent a link path. As an example, a link path may be represented by a reference to one or more link names. As another example, a link path may be represented by one or more indexes.

[0065] HDF files are stored in an area of ​​the structure-attached memory (FAM). According to some implementations, the FAM may be a persistent storage device that includes persistent or non-volatile memory devices.

[0066] Instruction 706, when executed by processor 708, further causes the processor to respond to the request. Responding to the request includes accessing a descriptor stored in a region. Accessing the descriptor may include performing one or more RMA operations (e.g., a get operation, a place operation, or an accumulate operation). The descriptor may correspond to a FAM data item and includes the FAM region ID corresponding to the region and the offset of the FAM data item within the region. The descriptor corresponds to a mapping from the data model of the HDF file to multiple data items stored in the region. According to some implementations, the descriptor may be the value of a key-value pair stored in a data item in the region. The key in the key-value store may correspond to a link path of the HDF object referenced in the request. Responding to the request further includes accessing the data item corresponding to the descriptor. Accessing the data item may include performing one or more RMA operations (e.g., a get operation, a place operation, or an accumulate operation).

[0067] refer to ​ According to an example implementation, the non-transitory machine-readable storage medium 800 includes machine-readable instructions 804 that, when executed by a machine, cause the machine to receive a request to perform an operation on a Layered Data Format (HDF) file. The HDF file may be an HDF5 file. The request may be an HDF5 API call. The API call may be handled by the OpenFAM API.

[0068] The request references the data model of the HDF file, and the HDF file is stored as multiple data items in a Structure Attached Memory (FAM). The FAM may contain memory nodes that provide a persistent memory pool, and the HDF file may be stored in the persistent memory pool.

[0069] Instruction 706, when executed by the machine, further enables the machine to access mapping information stored in the FAM. Accessing the mapping information may include performing one or more RMA operations using the FAM. RMA operations may be one-way communication and may include get operations, place operations, or accumulate operations.

[0070] The mapping information describes the mapping between the HDF object identifier included in the request and the FAM identifier corresponding to the given data item. The FAM identifier corresponding to the given data item may include a region ID and an offset of the data item. Instruction 804, when executed by the machine, further causes the machine to access the given data item using the FAM identifier in response to the request. Accessing the given data item using the FAM identifier may include one or more RMA operations. Accessing the given data item may include traversing a trie structure stored in the region. The trie structure may include KVSs corresponding to the nodes of the trie structure. A particular KVS may be stored in the data item of the FAM region and contains a key corresponding to the link path of the HDF object referenced in the request and a value corresponding to the FAM descriptor of the data item containing another KVS. The given data item may contain a KVS including a key corresponding to the link name and another KVS corresponding to a key corresponding to the link index.

[0071] According to an example implementation, the connector includes a pass-through Virtual Object Layer (VOL) connector. A particular advantage is that the HDF file can be advantageously stored in the FAM of the computer system, which can be efficiently scaled independently of the computer system's compute nodes.

[0072] According to the example implementation, VOL intercepts API calls and routes them to the connector in response to the file access nature of the HDF file. A particular advantage is that the HDF file can be advantageously stored in the FAM of the computer system, which can be efficiently scaled independently of the computer system's compute nodes.

[0073] According to an example implementation, accessing the mapping information includes accessing a trie data structure distributed across multiple data items stored in the FAM. The trie data structure maps multiple HDF identifiers to multiple FAM descriptors. A particular advantage is that HDF files can be advantageously stored and accessed within and from the FAM of a computer system, which can be efficiently scaled independently of the computer system's compute nodes.

[0074] According to the example implementation, the HDF identifier includes one of the following: multiple names corresponding to the corresponding HDF object or multiple indexes corresponding to the corresponding HDF object. A particular advantage is that HDF files can be advantageously stored and accessed in and from a FAM of a computer system, which can be efficiently scaled independently of the computing nodes of the computer system.

[0075] According to an example implementation, the access mapping information includes accessing a first portion of a trie structure, which is stored in a given data item and corresponds to the root HDF object of the HDF file. The access mapping information further includes identifying and accessing data items storing a second portion of the trie structure based on the first portion. A particular advantage is that the HDF file can be advantageously stored and accessed in and from a FAM of a computer system, which can be efficiently scaled independently of the computer system's compute nodes.

[0076] According to an example implementation, accessing the mapping information includes performing a remote memory access (RMA) operation. A particular advantage is that the HDF file can be advantageously stored and accessed in and from the FAM of the computer system, which can be efficiently scaled independently of the computer system's compute nodes.

[0077] According to an example implementation, servicing an API call in response to the identification of a first FAM descriptor includes accessing data items. A particular advantage is that the HDF file can be advantageously stored and accessed within and from the FAM of the computer system, which can be efficiently scaled independently of the computer system's compute nodes.

[0078] According to an example implementation, accessing the first data item includes initiating a remote memory access (RMA) operation to access the data item. A particular advantage is that the HDF file can be advantageously stored and accessed in and from the FAM of the computer system, which can be efficiently scaled independently of the computer system's compute nodes.

[0079] Although this disclosure has been described with respect to a limited number of embodiments, those skilled in the art who benefit from this disclosure will recognize many modifications and variations. The appended claims are intended to cover all such modifications and variations.

Claims

1. A method comprising: A connector associated with a computing node and with a structure-attached memory (FAM) receives an application programming interface (API) call to perform an operation associated with a first HDF object of a hierarchical data format (HDF) file, wherein the API call includes a first HDF object identifier corresponding to the first HDF object; In response to the request, the following actions will be taken: Based on the first HDF object identifier, the connector accesses the mapping information stored in the FAM; The connector uses the mapping information to identify a first FAM descriptor corresponding to a first data item, the first data item being stored in the FAM and corresponding to the first HDF object; and The connector provides services for the API calls in response to the recognition of the first FAM descriptor.

2. The method as described in claim 1, wherein, The connector includes a pass-through virtual object layer (VOL) connector.

3. The method of claim 1, further comprising intercepting the API call by the Virtual Object Layer (VOL) and routing the API call to the connector in response to the file access nature of the HDF file.

4. The method of claim 1, wherein, Accessing the mapping information includes accessing a trie structure distributed across multiple data items stored in the FAM, wherein the trie structure maps multiple HDF identifiers to multiple FAM descriptors, the multiple HDF identifiers including the first HDF object identifier, and the multiple FAM descriptors including the first FAM descriptor.

5. The method of claim 4, wherein, The plurality of HDF identifiers include one of the following: a plurality of names corresponding to a corresponding HDF object or a plurality of indexes corresponding to a corresponding HDF object.

6. The method of claim 4, wherein, Accessing the mapping information includes: Access the first portion of the given data item in the trie structure that corresponds to the root HDF object of the HDF file; Based on the first part, identify the data item among the plurality of data items that stores the second part of the trie structure; and Access the data item that stores the second part of the trie structure.

7. The method of claim 1, wherein, Accessing the mapping information includes performing a remote memory access (RMA) operation.

8. The method of claim 1, wherein, Providing services for the API call in response to the recognition of the first FAM descriptor includes accessing the first data item.

9. The method of claim 8, wherein, Accessing the first data item includes initiating a Remote Memory Access (RMA) operation to access the first data item.

10. An apparatus comprising: processor; as well as A memory for storing instructions that, when executed by the processor, cause the processor to perform the following operations: Receive a request from the virtual object layer to perform an operation on a hierarchical data format HDF file, wherein the HDF file is stored in a region of the structured attachment memory (FAM); Responding to the request includes: accessing a descriptor stored in the region, wherein the descriptor corresponds to a mapping from the data model of the HDF file to a plurality of data items stored in the region, and accessing the data item corresponding to the descriptor; and Use information to serve the request.

11. The apparatus of claim 10, wherein, The request identifies the HDF object of the HDF file, and the instruction, when executed by the processor, further causes the processor to perform the following operations: Read key-value storage data from the root data item stored in the area; A descriptor corresponding to a second data item of the region is identified based on the key-value storage data read from the root data item, the second data item corresponding to the link path of the HDF object; and The data item in the HDF file corresponding to the HDF object is accessed based on the descriptor corresponding to the second data item.

12. The apparatus of claim 11, wherein, When executed by the processor, the instruction further causes the processor to initiate a remote memory access (RMA) operation to read the key-value storage data from the root data item.

13. The apparatus of claim 11, wherein, The second data item corresponds to the data item in the HDF file that corresponds to the HDF object.

14. The apparatus of claim 11, wherein, When executed by the processor, the instruction further causes the processor to perform the following operations: Key-value storage data is read from the second data item using the descriptor corresponding to the second data item to identify a descriptor corresponding to a third data item of the region, the third data item corresponding to the link path of the HDF object; and The data item in the HDF file corresponding to the HDF object is accessed based on the descriptor corresponding to the third data item.

15. The apparatus of claim 11, wherein, The request to identify the HDF object of the HDF file and to provide services for the request includes reading from or writing to the data item of the HDF file corresponding to the HDF object.

16. A non-transitory computer-readable storage medium for storing machine-readable instructions that, when executed by a machine, cause the machine to perform the following operations: Receive requests to perform operations on HDF files, where, The request references the data model of the HDF file, and the HDF file is stored as multiple data items in the Structure Attachment Memory (FAM). Access mapping information stored in the FAM, the mapping information describing the mapping between the HDF object identifier contained in the request and the FAM identifier corresponding to a given data item among the plurality of data items; as well as The FAM identifier is used to access the given data item in response to the request.

17. The non-transitory computer-readable storage medium of claim 16, wherein, When executed by the machine, the instruction further causes the machine to initiate a Remote Memory Access (RMA) operation to access the mapping information and the given data item.

18. The non-transitory computer-readable storage medium of claim 16, wherein, When executed by the machine, the instructions further enable the machine to access at least one key-value store of at least one of the plurality of data items to access the mapping information.

19. The non-transitory computer-readable storage medium of claim 16, wherein, When executed by the machine, the instruction further causes the machine to initiate multiple Remote Memory Access (RMA) operations on the FAM to read multiple data items stored in the FAM, thereby reading the mapping information.

20. The non-transitory computer-readable storage medium of claim 19, wherein, The mapping information is stored in a trie structure distributed across the multiple data items.

Images