Application durability

Abstract

There is herein provided a method for converting a non-durable computer program for an application into a durable version. The method comprises a transparent wrapper that converts the non-durable computer into a durable version. The method further comprises determining that the operation is read-only or a read-write operation and, upon determining that the operation is a read only operation, processing the request on the volatile memory. A read-write operation is processed over both the volatile and non-volatile memory. The method synchronizes multiple such operations to ensure correctness while maximizing throughput. In particular multiple read-only operations are executed concurrently, and while read-write operations are batched to allow dynamic binary instrumentation and deduplication.

Description

[0001]Application Durability Field Example embodiments may relate to methods, an application programming interface and computer programs for providing application durability. In particular, a method is provided for converting a non-durable computer program executing over volatile memory to a durable program that executes over both volatile and non-volatile memory. The derived durable program has the same behavior as the original non- durable version but additionally supports recoverability from crashes. Background The advent of byte-addressable non-volatile memory (NVM) technologies, as such non-volatile random access memory (NV-RAM), has enabled the development of low- latency high-throughput durable applications, i.e., applications that are capable of recovering from full-system crashes. However, programming such applications is error prone as efficiency gains often require fine-grained (programmer-controlled) management of low-level persistence instructions. The disclosure herein, which may be referred to as “Mangosteen”, provides a high-level programming framework that allows developers to transform an existing linearizable in-memory application to a corresponding durably linearizable version. The framework’s API consists of a set of callback hooks that interpose on an application’s request processing flow with minimal developer effort. Mangosteen executes client operations on DRAM and persists the effects using binary instrumentation and redo logging. Mangosteen’s concurrency control facilitates batching of read-write requests to minimize the cost of persistence, while allowing read-only requests to execute concurrently. A novel intra-batch deduplication disclosed herein provides a mechanism that further reduces persistence overheads for common workloads such as online transaction processing (OLTP) ones. Empirical evaluation results show that Mangosteen-enabled applications outperform state-of-the-art solutions across the entire spectrum of read-write ratios. For example, the Mangosteen-based version of Redis demonstrates throughput gains of between two to five times in comparison to known prior art. Summary The scope of protection sought for various embodiments of the invention is set out by the independent claims. The embodiments and features, if any, described in this specification that do not fall under the scope of the independent claims are to be interpreted as examples useful for understanding various embodiments of the invention. According to a first aspect, there is described a method for managing computer program code at a non-durable version of an application and a durable version of the application. The method comprises determining computer program code for the non- durable version of the application and converting the computer program code from the non-durable version into the durable version. The durable version of the application comprises volatile memory and non-volatile memory. The method further comprises determining that the computer program code comprises a request to be performed by the application, determining that the request is read-only; and upon determining that the request is read-only, processing the request over the volatile memory of the durable version of the application. The method may further comprise concurrently executing multiple read-only requests of the computer program in parallel at the durable version of the application over the volatile memory. The read-only request may comprise allowing data to be accessed at the application but not modified. The method may further comprise converting the request to a format that can be interpreted by the application and performing deserialization, based on a type of the request, wherein the deserialization comprises converting the request from a binary representation to an in-memory representation. The method may further comprise determining that the computer program code comprises a second request to be performed by the application and determining that the second request is read-write. Upon determining that the second request is read- write, the method may further comprise processing the second request over the volatile memory of the durable version of the application, wherein processing the request comprises modifying data stored in the volatile memory and copying the modified data stored in the in the volatile memory to the non-volatile memory. The method may further comprise grouping together multiple requests into a batch of requests, wherein each of the multiple requests are read-write requests and processing the batch of requests sequentially at the durable version of the application. The read-write operation may comprise allowing data to be both accessed and modified. The volatile memory may be characterised by temporary storage of data in memory during processing and the non-volatile memory may be characterised by permanent storage of data in memory that can retain stored information even after power is removed. Converting the computer program code from the non-durable version into the durable version may comprise using dynamic code instrumentation. Alternatively, converting the computer program code from the non-durable version into the durable version may comprise using static code instrumentation. According to a second aspect, there is described a method for managing computer program code at a non-durable version of an application and a durable version of the application. The method comprises determining computer program code for the non- durable version of the application and converting the computer program code from the non-durable version into the durable version. The durable version of the application comprises volatile memory and non-volatile memory. The method further comprises determining that the computer program code comprises a request to be performed by the application, determining that the request is read-write, and upon determining that the request is read-write processing the request over the volatile memory of the durable version of the application, wherein processing the request comprises modifying data stored in the volatile memory and copying the modified data stored in the in the volatile memory to the non-volatile memory. Processing the request at the durable version of the application comprises modifying data of the durable version of the application. The method may further comprise converting the request to a format that can be interpreted by the application and performing deserialization, based on a type of the request, wherein the deserialization comprises converting the request from a binary representation to an in-memory representation. The method may further comprise grouping together multiple requests into a batch of requests, wherein each of the multiple requests are read-write requests and processing the batch of requests sequentially at the durable version of the application. The read-write request may comprise allowing data to be accessed and modified. The non-durable version of the application may be characterised by temporary storage of data in memory during processing and the durable version of the application may be characterised by permanent storage of data in memory that can retain stored information even after power is removed. Converting the computer program code from the non-durable version into the durable version may comprise using dynamic code instrumentation. Alternatively, converting the computer program code from the non-durable version into the durable version may comprise using static code instrumentation. Upon converting the computer program code from the non-durable version into the durable version, logic of the computer program code is not modified. According to a third aspect, there is provided a computer program product comprising a set of instructions which, when executed on an apparatus, is configured to cause the apparatus to carry out the method of any preceding method definition. According to a fourth aspect, there is provided a non-transitory computer readable medium comprising program instructions stored thereon for performing the method of any preceding method definition. According to a fifth aspect, there is provided an application programming interface, API, comprising means for carrying out the method of any preceding method definition. The API may further comprise a front-end comprising a random access memory, RAM, component suitable for storing the computer program code and data of the durable version of an application and a back-end comprising a non-volatile memory, NVM, component suitable for storing the computer program code and data of the durable version of an application. Brief Description of the Drawings Example embodiments will now be described by way of non-limiting example, with reference to the accompanying drawings, in which: FIG.1 shows, by way of example, a conversion of a non-durable computer program code to a durable program. FIG. 2 shows, by way of example, callback functions for use in conversion of a computer program code from a non-durable version to a durable one. FIG.3 shows, by way of example, a method. FIG.4 shows, by way of example, a second method. FIG.5 shows, by way of example, Mangosteen operations. FIG. 6 shows, by way of example, an architecture overview for carrying out the disclosure herein. FIG.7 shows, by way of example, a graph of performance of Mangosteen compared to other known prior art. Detailed Description In-memory storage applications (e.g., Redis, Memcached) are popular alternatives to traditional disk-oriented databases for both analytical and transactional workloads. The advent of non-volatile memory hardware such as Intel OptaneTMand compute express link (CXL) Memory-Semantic SSDs has driven the development of high- performance durable versions of such in-memory applications. NVM is byte- addressable and greatly reduces the cost of persistency. However, achieving the purported benefits of NVM for complex existing applications is challenging. First, the latency and memory bandwidth of existing NVM hardware is not yet comparable to dynamic random-access memory (DRAM). NVM reads measure 2x slower and writes 3x slower than even the slowest DRAM. As a result, NVM cannot be used directly as a drop-in replacement for DRAM in existing in-memory applications without incurring a substantial performance overhead. Second, correct recovery after failures is challenging since program failures may occur at any point, potentially leaving the application in an inconsistent state. To ensure correctness, the system must be made crash-consistent, i.e., guaranteed to recover into a consistent state after a failure. Here, a range of correctness conditions have been defined such as persistent atomicity, (buffered) durable linearizability and recoverable linearizability. These define correctness in terms of both failure atomicity (in the event of a crash, an operation either occurs in its entirety or not at all), and consistency (concurrent operations can be understood in terms of sequential executions of the operations). Achieving both correctness and good performance with NVM is notoriously difficult. Existing NVM interfaces require programmers to explicitly use costly persistence instructions (such as fences) to achieve durability. As a result, there is a thin line to thread between avoiding unnecessary persistence instructions to achieve good performance and ensuring the desired correctness properties. To address this issue, recent research proposed higher level abstractions, such as persistent transactional memory (PTM), persistent object libraries, and failure-atomic sections (FASEs) to facilitate durability. From a programming perspective, these approaches are still non-trivial to use because of source annotation effort, compiler modifications or explicit library calls that need to be added for each durable update. Furthermore, they require programmers to reason which parts of their code need to be made transactionally durable. This is particularly difficult and error-prone for large legacy codebases with complex existing dependencies (e.g. libraries and memory allocators). The disclosure herein presents a new framework for transforming existing in-memory applications to exploit NVM on modern multi-core hardware. Mangosteen provides a high-level (API) consisting of a small set of callback hooks to interpose on an application’s request processing with minimal developer effort. The Mangosteen runtime executes client requests concurrently on a DRAM copy of the application state. It exploits dynamic binary instrumentation (DBI) to transparently persist the effects of update requests in a persistent redo log, which are then applied asynchronously to a persistent copy of the application state on NVM. In comparison to existing high-level persistence frameworks, Mangosteen improves state-of-the-art performance on a range of real-world workloads. Mangosteen’s design is carefully optimized for performance. Mangosteen’s concurrency control allows read-only requests to execute in parallel and employs flat combining to batch execution of read-write requests. This facilitates a novel intra-batch deduplication mechanism to further reduce persistence overheads for common OLTP workloads. Overall, Mangosteen is able to persist update effects using just 2 persistent fences per batch on the critical path, and another 2 persistent fences per batch to make them durable asynchronously on the persistent copy of the application state. Unlike prior approaches based on redo logging, Mangosteen’s binary instrumentation only needs to intercept write instructions. Mangosteen also enables applications to use their existing memory allocators unmodified, and employs a novel split allocation scheme to avoid unnecessary overhead for updates to transient memory that do not need to be persisted for correct recovery. As discussed later, herein, Mangosteen’s performance is compared against state-of- the-art persistence frameworks “Persimmon” [Wen Zhang, Scott Shenker, and Irene Zhang. Persistent state machines for recoverable in-memory storage systems with NVRAM. In Proceedings of the 14th USENIX, Conference on Operating Systems Design and Implementation, OSDI’20, USA, 2020. USENIX Association] and “Romulus” [Andreia Correia, Pascal Felber, and Pedro Ramalhete. Romulus: Efficient algorithms for persistent transactional memory. In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures, SPAA ’18, page 271–282, New York, NY, USA, 2018. Association for Computing Machinery]. For that, two prototypes of Mangosteen-enabled applications are implemented: a persistent version of Redis and a persistent key-value store compatible with the LevelDB API. The results demonstrate that both prototypes achieve throughput gains of between two to five times compared to the Persimmon-based version of Redis. Mangosteen supports durable linearizability as its main correctness guarantee. Linearizability is a correctness condition that ensures that each operation on a concurrent or distributed system (such as reads, writes, and updates to a data store) appear to occur instantaneously at some point between their start and end times. As in standard linearizability, a durably linearizable concurrent execution is correct with respect to its sequential specification if it can be linearized in a manner consistent with the order of non-overlapping requests to form a history of its specification. In addition, a durably linearizable object must ensure failure atomicity, i.e., completed operations are never lost even after a crash, while incomplete operations may either be discarded completely or be kept in their entirety. With respect to liveness, Mangosteen provides deadlock freedom (i.e., in the absence of system crashes, some operation that has been previously invoked is guaranteed to eventually return) under the assumption the original application is deadlock-free. FIG.1 shows a conversion of a non-durable computer program to a durable version. A computer program code 100 is shown including at least one function 101 defining an operation in its interface. The computer program code is for a non-durable application and it defines a set of operations (known as application programmer interface, or API) to be performed on the volatile application state. An arrow 102 depicts a conversion of the computer program from a non-durable version to a durable one. On the left of the arrow in FIG.1 the computer program code is non-durable and only executes in a volatile memory 103 and to the right of the arrow in FIG.1 the computer program code is transformed into a durable version which makes use of both volatile and non-volatile memory 104. The durable and non-durable versions of the computer program code are both linearizable, however, the durable version is also durably linearizable. FIG. 2 demonstrates by way of example, a set of callback functions 105 which are used to convert the non-durable version of the application to the durable version. By way of example, six callback functions 105 are listed in FIG.2, however, there may be more or fewer callback functions 105 depending on the requirements of the application on which the computer code is running. FIG.3 shows, by way of example, a flowchart of a method 300 according to example embodiments. The method 300 is a method according to Mangosteen as referred to herein. Each element of the flowchart may comprise one or more operations. The operations may be performed in hardware, software, firmware or a combination thereof. For example, the operations may be performed, individually or collectively, by a means, wherein the means may comprise at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the performance of the operations. The method 300 shown in FIG.3 demonstrates managing computer program code at a non-durable version of an application and a durable version of the application. The non-durable version of the application comprises volatile memory. Volatile memory is characterised by usage of temporary storage of the computer code in memory during processing (for example DRAM or RAM). The durable version of the application comprises both volatile and non-volatile memory. Non-volatile memory characterised by permanent storage of the computer code data in memory that can retain stored information even after power is removed. The method 300 may be performed at an API. The method 300 may comprise a first operation 301 of determining computer program code for the non-durable version of the application. The method 300 may comprise a second operation 302 of converting the computer program code from the non-durable version into the durable version. The conversion may be conducted by any means as described herein. The method 300 may comprise a third operation 303 of determining that the computer program code comprises a request to be performed by the application. The request may be a read-only request or a read-write request. The request indicates an operation to be performed at the application. Both terms ‘read-only’ and ‘read-write’ take on the conventional meanings. A ‘read-only’ operation only accesses data, but does not modify the application state whereas a ‘read-write’ operation performs at least one modification to the application state. The method 300 may comprise a fourth operation 304 of determining that the operation is a read-only operation. The method 300 may comprise a fifth operation 305 of upon determining that the request is read-only, processing the request at the non-durable version of the application. Processing the request at the durable version of the application is over the volatile memory. In other words, the processing of the read-only request at the durable version of the application is conducted via volatile memory such as DRAM or RAM. The method 300 allows concurrent execution of the computer program code. Concurrently executing the computer program code in the volatile state includes executing multiple read-only operations in parallel on the volatile state. This is referred to as ‘concurrency’ and will be further discussed herein. The method 300 may optionally further comprise converting the request to the format that can be interpreted by the application via deserialization, wherein the deserialization comprises converting the binary encoding of the operation into an in- memory representation. As such, the method includes decoding specific operations from binary data so that it can be processed in a suitable way to maintain durability. ‘Deserialization’ will be further discussed herein. The method 300 may optionally further comprise determining that the computer program code comprises a second request to be performed by the application and determining that the request is read-write. Upon determining that the request is read- write, the method 300 may comprise processing the second request over the volatile memory of the durable version of the application. Processing the request comprises modifying data stored in the volatile memory. Then, the modified data stored in the in the volatile memory is copied to the non-volatile memory for durability. As such, read- only request may be performed at the volatile memory, whereas read-write request are performed at the volatile memory, but any changes to the data stored in the volatile memory are also copied to the non-volatile memory. This is beneficial as it is only required to enact the read-write requests at the non-volatile memory for long term data storage to maintain durability of the application. The method 300 may further include grouping together multiple operations into a batch of operations. Each of the multiple operations are read-write operations and each of the multiple operations comprise computer program code. As such, a batch of read- write operations are compiled together for processing sequentially at the durable version of the application. The method may further include processing the batch of operations simultaneously to ensure the effects of the read-write operations on the application state are durably stored. ‘Batching’ of requests will be further discussed herein. Processing the request to convert the non-durable computer program into a durable program may comprise use of dynamic binary instrumentation. Alternatively, this conversion can be achieved via compile-time instrumentation by statically annotating the source code. The purpose of both is to identify modifying operations on the memory such as memory allocation and writes. Fig.4 shows, by way of example, a flowchart of a method 400 according to example embodiments. The method 400 is a method according to Mangosteen as referred to herein. Each element of the flowchart may comprise one or more operations. The operations may be performed in hardware, software, firmware or a combination thereof. For example, the operations may be performed, individually or collectively, by a means, wherein the means may comprise at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the performance of the operations. The method 400 shown in FIG.4 demonstrates managing computer program code at a non-durable version of an application and a durable version of the application. The non-durable version of the application comprises volatile memory. Volatile memory is characterised by usage of temporary storage of the computer code in memory during processing (for example DRAM or RAM). The durable version of the application comprises both volatile and non-volatile memory. Non-volatile memory characterised by permanent storage of the computer code data in memory that can retain stored information even after power is removed. The method may be performed at an API. The method 400 may comprise a first operation 401 of determining computer program code for the non-durable version of the application. The first operation 401 is the same as the first operation 301 of method 300. The method 400 may comprise a second operation 402 of converting the computer program code from the non-durable version into the durable version. The conversion may be conducted by any means as described herein. The second operation 402 is the same as the first operation 401 of method 400. The method 400 may comprise a third operation 403 of determining that the computer program code comprises a request to be performed by the application. The request may be a read-only request or a read-write request. The request indicates an operation to be performed at the application. Both terms ‘read-only’ and ‘read-write’ take on the conventional meanings. A ‘read-only’ operation only accesses data, but does not modify the application state whereas a ‘read-write’ operation performs at least one modification to the application state. The method 400 may comprise a fourth operation 404 of determining that the request is read-write request. In other words, the request is for a read-write operation to be performed at the application. The method 400 may comprise a fifth operation 405 of upon determining that the request is read-write, processing the request at the durable version of the application. Processing the request at the durable version of the application may comprise modifying data stored in the volatile memory of the durable version of the application. Furthermore, upon processing the request at the durable version of the application, the modified data stored in the in the volatile memory is copied to the non-volatile memory. As such, all modifications to the volatile memory are automatically captured at the non-volatile memory. This ensures that the read-write operations are saved at the non-volatile memory. The read-write request is therefore also enacted at the non- volatile memory of the durable version of the application to ensure that any modification contained in the read-write request is included in the durable version of the application such that it can be restored in the instance of a system crash. As such, read-write requests are performed at the volatile memory, but any changes to the data stored in the volatile memory are also copied to the non-volatile memory. The method 400 ensures that it is determined first that a request is a read-write request before the necessary computing process in enacted to store the modifications to data caused by the read-write request (which would otherwise not be needed by a read- only request). This is beneficial as it is only required to enact the read-write requests at the non-volatile memory for long term data storage to maintain durability of the application. For the method 400 of FIG.4 deserialization and batching of read-write requests may also be performed as discussed in relation to the method 300 of FIG.3. Processing the request to convert the non-durable program into a durable version may also comprise using dynamic or static binary instrumentation. Mangosteen functions as a generic wrapper for a linearizable application that resides in volatile memory (e.g. DRAM), allowing the application to handle concurrent requests in a fault tolerant (recoverable) manner. The following describes the Mangosteen framework’s API and how it can be integrated with an existing application. An overview is given of the key architectural features of the Mangosteen runtime. By way of example, the Mangosteen framework is discussed with reference to exemplary listings 1 to 7. These are examples of how the Mangosteen approach can be carried out, however, the skilled person will understand how these are not intended to be limiting on the scope of the disclosure herein and are instead provided for illustrative purposes to aid with understanding the function of Mangosteen. Mangosteen API The Mangosteen framework is implemented as a user-level library. Mangosteen’s application programming interface is given in Listing 1. The API may, by way of example, consist of 6 callback functions through which the Mangosteen framework interacts with the application. A framework initialization function (initMangosteen) takes as its arguments pointers to the implementations of each callback. Two of the callbacks (initAppClient and destroyAppClient) are executed once for every application client, while the remaining callbacks are executed once for each client request. 1 int initMangosteen( / *... Callback fn ptrs int mode); 2 3 void* initAppClient(void *params); 4 void destroyAppClient(void *c); 5 void deserializeRequest(void *c, void *req); 6 bool isReadOnly(void *c); 7 void processRequest(void *c); 8mangoResponse* getResponse(void *c);Listing 1: Mangosteen API The framework is flexible in that it supports both a remote procedure call (RPC) mode and a local mode (as indicated by an additional mode argument to initMangosteen). RPC mode targets settings where clients interact with the application over a network. Local mode can be used for integration of persistent data structures into a standalone application. To use Mangosteen in the RPC mode, the application developer must implement all callbacks in Listing 1 though for most applications this can be achieved using simple wrapper functions around existing application code. For the local mode, the applications are only required to implement isReadOnly and processRequest, and call ClientCmd directly to execute requests. A basic in-memory key-value (KV) store is used illustrate how a developer can integrate with the Mangosteen API. First the core functionality of an initial (non- persistent) version of the KV store (Listing 2) is given. Its state consists of an array store that is initialized in initKVStore (line 16). The store supports two operations, getValueAtIndex and putValueAtIndex (lines 20 and 25), which get and set respectively the value at a given index into the array. Both operations take, as an argument, a client context object c that identifies the client issuing ^ the command (c->id), ^ the command itself (c->currCmd) ^ and a buffer to store the response (c->response). The handleRequest function (line 31) dispatches to the appropriate operation according to the type of the current command (c->currCmd->type). On return from handleRequest, the store’s networking layer (not shown) replies to the client with the response contained in c->resp. 1 char *store; 2 3 typedef struct kvCmd { 4 enum {PUT, GET} type; 5 unsigned int index; 6 char value[VAL_SZ]; 7 } kvCmd; 8 9 typedef struct kvClient { 10 int id; 11 kvCmd* currCmd; 12 char resp[RESP_SZ]; 13 } kvClient; 14 15 16 void initKVStore() { 17 store = malloc(STORE_SZ * VAL_SZ); 18 } 19 20 void getValueAtIndex(kvClient *c) { 21 memcpy 22 &store 23 } 24 25 void putValueAtIndex(kvClient 26 memcpy(&store[c->currCmd- 27 c->currCmd->value, VAL_SZ); 28 memcpy(c->resp, "OK",2); 29 } 30 31 void handleRequest(kvClient *c) { 32 if(c->currCmd->type == PUT) 33 putValueAtIndex(c); 34 else 35 getValueAtIndex(c); 36 } Listing 2: Basic In-Memory KV Store Mangosteen Integration FIG.5 shows, by way of example, Mangosteen operations 500. A first deserialization step 501 deserialize(req) may optionally be implemented. Subsequent to the optional deserialization step 501, a read-write question request 502 isRW(opi) is asked to determine whether an operation is a read-only or read-write operation. A request is then processed according to whether the operation is read-only or read- write. A read-only process request 503 procReq(opi) is processed in the case that the operation is read-only. A read-write process request 504 procReq(opi) is processed in the case that the operation is read-write. Finally, the conversion from the volatile state to the non-volatile state is completed. Listings 3 and 4 illustrate, by way of example, how to use the Mangosteen API to create a durable version of the in-memory KV store. The application first initializes its state as usual using initKVStore (Listing 3). It then invokes initMangosteen to start the Mangosteen framework’s runtime, passing as parameters function pointers to implementations of the Mangosteen API callbacks and configuring Mangosteen to use RPC mode. Mangosteen invokes the first callback, initAppClient (Listing 4, line 1), whenever a new client is initialized, e.g. when a client first connects to the application in RPC mode. The KV store application uses this callback to initialize a new client context object (line 2). The structure of this object is application-specific and remains opaque to Mangosteen, but is passed as a parameter to subsequent application callbacks during request processing. The params argument contains a unique client identifier that the application can optionally record within the client object. When a client terminates, Mangosteen invokes a corresponding destroyAppClient callback to allow the application to clean up (line 7). In some embodiments, a deserializeRequest 501 callback performs application- specific deserialization of an incoming request req (line 11). The deserializeRequest 501 includes determining that the operation is specific to a type of application and performing deserialization, based on the type of application. The deserialization comprises converting the computer program code from binary data to determine the operation. Mangosteen assumes the deserialized request is stored in an application-specific location in opaqueCtx. For a basic KV store, no complex deserialization is required, and the location of the request buffer is recorded directly in currCmd. The isReadOnly 502 callback (line 15) allows the application to indicate to Mangosteen which requests (if any) do not modify the application’s state. As such it may be determined which operations are read-only and which are read-write. This enables Mangosteen to exploit read-read concurrency for improved performance. For the KV store, getValueAtIndex is considered read-only, whereas putValueAtIndex is read-write since it modifies store. The processRequest callback (line 21) interposes on the application’s core request processing logic. For the KV store example it is a simple wrapper around the handleRequest function. As shown in FIG. 5 a different process is followed depending on whether the operations are read-write or read-only. The computer program code may be processed differently to allow concurrency for the read-only requests and batching of the read- write requests to ensure that durability is achieved for both types of requests. Mangosteen assumes that the results of request processing are recorded in opaqueCtx, e.g., in the KV store they are stored in a client c’s response buffer c->resp. The last callback, getResponse, allows Mangosteen to extract the response (line 25). It returns a mangoResponse object consisting of a pointer to the start of the response (i.e. c->resp for the KV store) and the size of the response. The Mangosteen runtime then responds to the client. 1 int main() { 2 initKVStore(); 3 4 initMangosteen(&initAppClient, 5 &deserializeRequest, 6 &isReadOnly, 7 &processRequest, 8 &destroyAppClient, 9 &getResponse, 10 MODE_RPC); 11 return 0; 12 } Listing 3: Mangosteen Initialization 1 void *initAppClient(void *params) { 2 kvClient* c = malloc(sizeof(kvClient)); 3 c->id = *(int*)params; 4 return c; 5 } 6 7 void destroyAppClient(void *opaqueCtx) { 8 free(opaqueCtx); 9 } 10 11 void deserializeRequest(void *opaqueCtx, void *req) { 12 ((kvClient*)opaqueCtx)->currCmd = (kvCmd*)req; 13 } 14 15 bool isReadOnly(void *opaqueCtx) { 16 if(((kvClient*)opaqueCtx)->currCmd->type == PUT) 17 return false; 18 return true; 19 } 20 21 void processRequest(void *opaqueCtx) { 22 handleRequest((kvClient*)opaqueCtx); 23 } 24 25 mangoResponse *getResponse(void *opaqueCtx) { 26 kvClient *c = (kvClient*)opaqueCtx; 27 mangoResponse *mResponse = malloc(sizeof(mangoResponse)); 28 mResponse->response = c->resp; 29 mResponse->size = VAL_SZ; 30 return mResponse; 31 } Listing 4: KV Store Implementation of Mangosteen Callbacks Architecture Overview An overview of the Mangosteen architecture is given in FIG.6. It consists of two main components: (1) the frontend 601, which stores the application state in DRAM 603 and processes client requests concurrently, and (2) the backend 602, which interfaces with the frontend 601 using a persistent redo log 604 and also persists the application state in NVM 605, enabling recovery. When the application starts it calls initMangosteen to initialize the Mangosteen runtime. The frontend 601 executes within the application process. It first allocates the shared redo log in persistent memory, in addition to a persistent region table that the backend 602 will use to record mapped regions in the application’s address space. The frontend 601 then checkpoints itself using, for example checkpoint and restore in userspace (CRIU), a Linux process checkpointing tool. This checkpoint serves as a base image of the application’s address space during recovery. The frontend 601 then forks to create the backend 602 and blocks until the backend initialization completes. To initialize itself, the backend 602 maps relevant parts of its address space to persistent memory. To achieve this, it creates a persistent memory region for all application memory regions listed in proc / self / maps except the stack, read-only regions and the redo log 604. Each persistent memory region is a contiguous chunk of NVM 605 backed by a file. The backend 602 records the set of persistent memory regions in the persistent region table. Once backend 602 initialization completes, its virtual address space memory mappings are identical to that of the frontend 601. The frontend then initializes a pool of worker threads (and in RPC mode an event loop for each thread) and begins processing client requests. When a client first establishes a connection with Mangosteen, it is assigned a thread from a thread pool 606, which allocates and initializes a thread-local Mangosteen client context and triggers the Mangosteen API’s initAppClient callback. Clients then submit requests to be executed, with each submitted request handled by the thread responsible for the client in RPC mode, and the client thread itself in local mode. Mangosteen deserializes requests in an application-specific manner using the deserializeRequest callback (see FIG.6). It then passes the resulting client context object to the frontend’s 601 concurrency control mechanism 607 (see clientCmd(ctx) in FIG. 6). Mangosteen executes read-only operations in parallel and batches execution of read-write operations, for example by using flat-combining techniques. Before responding to the client, the frontend 601 persists updates from read-write operations in a persistent redo log (implemented as a fixed size ring-buffer). To capture updates in a transparent fashion, Mangosteen uses a DBI framework to intercept write instructions. To minimize the cost of persistence, Mangosteen employs a novel intra-batch deduplication mechanism to filter redundant updates. Once the updates for a read-write command are persisted in the redo log, a response is returned to the client. The Mangosteen backend asynchronously copies entries from the persistent redo log 604 and applies the updates to an NVM 605 copy of the DRAM 603 application state. Once the changes to the NVM 605 copy are persisted, the backend 602 may remove the corresponding entries from the redo log 604. During recovery from a crash, Mangosteen first restores the CRIU checkpoint and then forks. Next, the backend 602 iterates over the region table and remaps all persistent memory regions. The backend 602 then processes all committed entries in the redo log and applies their updates. Once the backend 602 is finished, the frontend 601 iterates over the region table and recreates all memory mappings in DRAM 603. Finally, it copies data from the corresponding regions in persistent memory and resumes execution. Mangosteen Concurrency Control The Mangosteen concurrency control ensures durably linearizable execution of commands supported by the underlying data store API. Its implementation aims to maximize concurrency while minimizing the cost of read-write commands persistence. To achieve this, full read-only command parallelism is supported while forgoing read- write vs read-only and read-write vs read-write parallelism for the sake of optimizing the persistent memory performance. Read-write commands are batched, for example via flat-combining and executed under an exclusive lock. However, Mangosteen uses binary instrumentation to convert the batched commands into a stream of elementary store instructions, which are then deduplicated and appended to a redo log. This reduces the overall amount of data that needs to be persisted per batch, and ensures that the persistent memory is always accessed sequentially on a critical path. Furthermore, persisting a batch only requires 1 persistent fence (pfence) and 1 persistent sync (psync) instruction, amortizing the cost of synchronous persistence across multiple read-write commands. Overall, a concurrency control implementation as shown in Listing 5 below follows the writer-preference lock (C-RW-WP) algorithm of Irina Calciu, Dave Dice, Yossi Lev, Victor Luchangco, Virendra J. Marathe, and Nir Shavit. Numa-aware reader-writer locks. SIGPLAN Not., 48(8):157–166, February 2013, which is adapted to a multi-core setting and integrated with flat combining, as explained below. Once a frontend thread i finishes receiving and deserializing a client request, it creates an opaque client request context ctx, and calls the client request handler ClientCmd with ctx as argument (line 1). The thread then calls isReadOnly (line 2) to determine the request type, and based on the outcome proceeds to execute either the read-only (lines 3–11) or the read- write (lines 13–31) request handling branch. Upon completion of either branch, the response is returned to the client in line 32. Below, it is referred to a thread executing in the read-only branch as a reader and a thread executing in the read-write branch as a writer. 1 when notified ClientCmd(ctx) 2 if isReadOnly(ctx) then 3 while true do 4 rFlag[i] ← true; 5 if ¬wrFlag then 6 processRequest(ctx); 7 rFlag[i] ← false; 8 break; 9 else 10 rFlag[i] ← false; 11 while wrFlag do pause; 12 else 13 (ctx, status)[i] ← (ctx, READY); 14 while true do 15 if status[i] = STARTED then 16 while status[i] ≠ DONE do pause; 17 if status[i] = DONE then break; 18 if ¬wrFlag then 19 if CAS(wrFlag, false, true) then 20 readySet ← ∅; 21 repeat 22 forall {j ∣ status[j] = READY} do 23 status[j] ← STARTED; 24 readySet ← readySet∪{j}; 25 until ∀k.¬rFlag[k]; 26 start_tx(); 27 forall j ∈ readySet do 28 processRequest(ctx[ j]); 29 end_tx(); 30 forall j ∈ readySet do status[ j] ← DONE; 31 wrFlag ← false; 32 trigger Response(getResponse(ctx)); Listing 5: Client request handler executed by thread Read- branch. Once in the read-only branch, a reader i enters a while loop where it attempts to acquire a shared lock. To this end, it first sets its entry rFlag[i] in a shared memory array rFlag to true (line 4) to announce its intention to perform a read- only command. To prevent false sharing among the readers, the entries in rFlag are cache-line aligned. It then inspects wrFlag (line 5) to determine whether there is a concurrently executing writer. If not, the reader proceeds to execute the read-only request (line 6) using a main memory copy of the data store state, and sets rFlag[i] to false (line 7). It then breaks from the while loop and returns the response. Note that a writer will only able to acquire an exclusive lock once all concurrently executing readers have set their rFlag entries to false (see below). If wrFlag is set, the reader first assigns false to rFlag[i] (line 10) to allow a writer that set wrFlag to complete the exclusive lock acquisition. It then proceeds to execute a busy- waiting loop in line 11 until wrFlag becomes false. The pause instruction suspends the thread for a short time duration to optimize busy waiting. Once wrFlag indicates a shared lock is available, the reader goes back to the beginning of the while loop to retry the shared lock acquisition. Read-write branch. Upon entering the read-write branch, a writer i first advertises ctx as ready for processing by setting ctx[i] to ctx and status[i] to READY (line 13). Both arrays are cache-line aligned to avoid false sharing among the writers. It then enters a while loop where it attempts to acquire an exclusive lock by setting wrFlag to true via a compare-and- swap (CAS) instruction (line 19). To minimize the cache invalidation traffic caused by CAS, the writer first reads the current value of wrFlag, and only proceeds with CAS, if it finds wrFlag = false (line 18). A writer, which is able to acquire an exclusive lock, becomes a combiner, and is responsible for processing all the read-write commands that have been advertized in ctx. To this end, it first executes the loop in lines 22–24 to ensure all shared locks previously acquired by the readers have been released. While in this loop, the combiner collects in readySet the indices of the status array entries corresponding to the read-write commands that are ready to be processed. It also flags these commands as STARTED thus causing the remaining (non-combiner) writers to abandon any further attempts to acquire the exclusive lock. These writers then proceed directly to line 16 where they await the completion of their requests by the combiner. This early notification mechanism helps to avoid unnecessary querying of wrFlag by non-combiners thus reducing the number of non-local shared memory accesses. Once the combiner validates all shared locks have been relinquished, it calls start_tx to activate the instrumentation state machine (Listing 6). The combiner then proceeds to execute the read-write commands occupying the entries ctx[ j] for each j ∈ readySet under instrumentation, which in particular, causes all subsequent write instructions to be aggregated and deduplicated by the instrumentation code. Once the combiner finishes processing the advertized commands, it issues a DynamoRIO clean call to end_tx, which disables instrumentation, and persists the accumulated write records to the redo log. The combiner then proceeds to flag the processed commands as DONE (line 30), which enables the writers awaiting their completion to respond to their clients (lines 17 and 32). Further details of deduplication and persistence implementations are discussed later herein. Minimizing Combiner Instrumentation Mangosteen relies on program instrumentation at the frontend to capture the effects of commands that modify the application’s persistent state. While this simplifies the task of the persistent memory programmer, it is crucial the instrumentation is lightweight since it is on the critical path. To ensure this, Mangosteen exploits redo logging and the concurrency control mechanism described in the previous section to minimize the number of instrumented instructions. Read-write commands. Write instructions in read-write commands may modify the application’s persistent state, and hence need to be instrumented. Unlike prior work that combines DBI with undo logging, Mangosteen’s redo logging does not need to immediately persist data for intercepted write instructions. Instead, it only needs to add an address to a hashset, which can be implemented efficiently as inline instrumentation without needing an expensive DynamoRIO ‘clean call’. Furthermore, many large updates are implemented as calls to memcpy which can be intercepted by Mangosteen directly. Like prior work, Mangosteen assumes data on the stack is transient and filters any associated writes by checking if the destination address is below the stack pointer rsp. For read instructions within read-write commands, it is observed that due to Mangosteen’s flat-combining concurrency control mechanism, batches of read-write commands are effectively executed sequentially by the same combiner thread. As a result, read instructions within a read-write command cannot observe conflicting updates from another concurrently executing read-write command. Therefore, in contrast to prior approaches based on redo logging, it is unnecessary for Mangosteen to instrument read instructions within read-write commands. commands. In contrast to read-write commands, for read-only commands it is assumed that the application’s persistent state is never modified nor accessed at the same time as read-only commands. This means that no instrumentation is required for read or write instructions in read-only commands. It is noted that for read-only commands front-end threads still execute under the control of a DBI framework, but the overhead is found to be minimal in the absence of any instrumentation. Persistence Apart from synchronization and instrumentation overheads, a key performance concern for the Mangosteen frontend is the cost of storing updates in the persistent redo log. Persisting to NVM efficiently involves minimizing (i) the amount of data to persist and (ii) the number of fence instructions needed for correctness. Mangosteen employs write deduplication and batch redo logging to meet these objectives. Write As discussed previously, Mangosteen’s frontend instrumentation (Listing 6) is responsible for capturing the effects of relevant write instructions and storing them in a persistent redo log. A straightforward implementation of frontend redo logging involves persisting an update for every relevant write instruction. However, this approach wastes persistent memory bandwidth when there are multiple updates to the same memory location, since the effects of earlier updates in the same batch of advertised commands are no longer relevant. To minimize the amount of data that needs to be persisted to the redo log, Mangosteen deduplicates updates before persisting them. A challenge here is that write instructions that modify the same location in memory may have different granularities and alignments. Furthermore, the deduplication mechanism is on the critical path, so it must be carefully designed to minimize performance overheads. To overcome these challenges, the Mangosteen deduplication algorithm (e.g. lines 6-12 in Listing 6) employs a thread-local hash table (H) to record the addresses modified by the write instructions within a batch of commands. The hash table is implemented as an array of 64-bit integers and uses open addressing with linear probing as a collision resolution scheme. The array’s starting address is cache line aligned, and the entries are tightly packed without any internal padding. This allows a set of adjacent entries to be loaded together on each lookup thus optimizing the linear probing. Each entry in the hash table represents the start of an aligned memory block of size BLOCK_SIZE bytes. This approach allows us to coalesce small non-overlapping writes within the same BLOCK_SIZE byte memory range into a single redo log entry, but may lead to write amplification if most writes are smaller than BLOCK_SIZE bytes. Whenever an instruction to write a payload of size size to an address addr is intercepted by the instrumentation (Listing 6, line 4), it computes the addresses x1and x2of the first and the last BLOCK_SIZE-aligned blocks falling within the range [addr, addr + size) (Listing 6, lines 3.6–7). It then inserts the addresses x1+ i ⋅ BLOCK_SIZE for all integers i such that 0 ≤ i ≤ (x2− x1) / BLOCK_SIZE into the hash table (Listing 6, lines 3.9–12) using linear probing to resolve collisions (Listing 6 lines 3.10–11). The hash table is statically allocated to fit into the CPU cache and is not dynamically resized. Instead, whenever its load factor becomes too high for linear probing to work efficiently (60% in the implementation), the newly produced addresses are not stored in the hash table, but instead spilled into a separate overflow buffer. This is not shown in the pseudocode in Listing 6 for clarity. 1 when called start_tx() 2 instr_status ← ACTIVE; 3 (head, tail) ← (rb_head, rb_tail); 4 when called write(addr, size) 5 pre: instr_status = ACTIVE 6 x1← ⌊addr / BLOCK_SIZE⌋; 7 x2← ⌊(addr + size) / BLOCK_SIZE⌋; 8 for a ← x1to x2by BLOCK_SIZE do 9 i ← a % length(H); / / linear probing 10 while H[i] ≠ null∧ H[i] 11 i ← (i + 1) % length(H); 12 H[i] ← a; 13 when called mmap(addr, size) 14 pre: instr_status = ACTIVE 15 rb_enque(^ MMAP, addr, size^); 16 pfence(); 17 p_rb_tail ← tail; 18 pwb(p_rb_tail); 19 pfence(); 20 when called end_tx() 21 instr_status ← IDLE; 22 forall k = 0..length(H) − 1 do 23 if H[k] ≠ null then 24 memcpy(&blk, H[k], BLOCK_SIZE); 25 rb_enque(^REDO, H[k], blk^); 26 H[k] ← null; 27 rb_enque(^COMMIT^); 28 pfence(); 29 p_rb_tail ← tail; 30 pwb(p_rb_tail); 31 psync(); 32 (rb_head, rb_tail) ← (head, tail); 33 function rb_enque(R) 34 if (tail + 1) % ringBufSize head then 35 ringBuf[tail] ← R; 36 pwb(ringBuf[tail]); 37 tail ← (tail+ 1) % ringBufSize; 38 else 39 while true do 40 head ← p_rb_head; 41 if (tail+ 1) % ringBufSize = head then pause else break; Listing 6: Instrumentation State Machine The overflow buffer is implemented as a collection of dynamically allocated integer arrays each of which is cache line aligned and fits into the CPU cache. The addresses are added to the overflow buffer in the order of their generation, and are not deduplicated. Experimental evaluation shows that most batches produced by the workloads studied can be processed in full without using the overflow buffer thus maximizing the deduplication benefits, thereby providing a technical benefit of this approach. Batched Redo Logging. The content of the persistent redo log is stored in a ring buffer ringBuf of size ringBufSize on NVM. New records are added to the redo log by the frontend and are removed (consumed) by the backend. The two end points are synchronized using persistent head (p_rb_head) and tail (p_rb_tail) indices stored on NVM. For efficiency, wherever possible, both frontend and backend use cached copies of these indices and defer their persistent updates until it is necessary for durability. Details of the log handling mechanism at both frontend and backend are discussed below. Frontend redo The logic of the redo log handling at the frontend is shown as a part of the instrumentation state machine pseudocode in Listing 6. The frontend caches the latest known values of p_rb_head and p_rb_tail in the shared variables rb_head and rb_tail, respectively. These variables are copied into thread-local variables head and tail by a combiner thread before it starts processing a new batch of read-write requests (Listing 6, line 3). Once all the read-write requests in the current batch have been executed, the combiner calls end_tx (Listing 6, line 20) where it executes the following steps for each non-null address A stored in the deduplication hash table H (Listing 6, lines 22–26). First, it creates a REDO record R consisting of A and the block of data pointed by A, which it fetches directly from the main memory (Listing 6, line 24). It then executes the following logic to add R to the persistent log. Since the log can be full, the addition requires extra care to synchronize with the backend (see function rb_enque in Listing 6, line 33). To this end, the combiner first attempts to use the cached copies of the head and tail indices to test whether the ring buffer entry at index j = (tail + 1) % ringBufSize is not occupied by head (Listing 6, line 34). If so, it copies R to that entry, persists it with pwb, and sets tail = j (Listing 6, lines 35–37). Note that tail does not need to be persisted at this point as no other thread, apart from the combiner, can use it to add records concurrently. If the entry at j is occupied, the combiner enters a busy-waiting loop (Listing 6, lines 39–41) where it loads p_rb_head into head, and checks whether it has advanced past j. If so, it breaks from the loop, persistently stores R at j, and assigns tail to j as above. Otherwise, it pauses for a short while, and then resumes the loop. Once the combiner has finished processing the addresses in H, it proceeds to process those in the overflow buffer in the same fashion. This is not shown in the pseudocode of Listing 6. Once all addresses have been processed, it calls rb_enque to add and persist a COMMIT record in the log (Listing 6, line 27). It then calls pfence, copies the value of tail to p_rb_tail, and persists it with pwb (Listing 6 lines 28–30). This guarantees integrity of the ring buffer: i.e., 1. a COMMIT record always appears in the log immediately after all previously persisted REDO records, and 2. the index stored in p_rb_tail always points to the entry immediately following the one occupied by a COMMIT record. The combiner then calls psync (Listing 6, line 31) to ensure all previously issued persistent stores have been made durable thus enabling the read-write requests in the current batch to return to their clients. Finally, it copies head and tail back to rb_head and rb_tail (Listing 6, line 32) to make them available for the next combiner. Backend redo log handling. The pseudocode of the redo log handling at the backend is shown in Listing 7 below. It is executed by a single dedicated thread in a separate operating system process. 1 (b_head, b_tail) ← (p_rb_head, p_rb_tail); 2 while b_head ≠ b_tail do / * Startup * / 3 proc_redo_record(ringBuf[b_head]); 4 b_head ← (b_head+1) % ringBufSize; 5 notify_ready(frontend); 6 while true do / * Main loop * / 7 if b_head ≠ b_tail then 8 proc_redo_record(ringBuf[b_head]); 9 b_head ← (b_head+1) % ringBufSize; 10 else 11 while true do 12 b_tail ← p_rb_tail; 13 if b_head = b_tail then pause else break; 14 function proc_redo_record(R) 15 if R = ^REDO, addr, blk^ then 16 memcpy(addr, blk, BLOCK_SIZE); 17 forall w ∈ [addr, addr +BLOCK_SIZE) do pwb(w); 18 else if R = ^MMAP, addr, size^ then / / Map page-aligned pmem region / / [addr,addr +size) 19 p_mmap(addr, size); 20 pfence(); 21 else if R = ^COMMIT^ then 22 pfence(); 23 p_rb_head ← b_head; 24 pwb(p_rb_head); 25 psync() Listing 7: Backend thread As it is the case for the frontend, the backend caches the persistent head and tail indices in thread-local variables b_head and b_tail, respectively (Listing 7, line 1) before it starts processing the log. It then proceeds to execute the following logic in an infinite loop. First, if the ring buffer is not empty (i.e., b_head ≠ b_tail) (Listing 7, line 7), the backend reads the record R stored at head and inspects its type. If R is a REDO record with an address A and payload D, it copies D to the address A on NVM, and persists the words spanned by D with pwb (Listing 7, lines 16–17). If R is either mmap or munmap record (the latter is not shown in the pseudocode for conciseness), it either allocates or deallocates persistent memory. Finally, if R is a COMMIT record, the backend reads the record R stored at head and inspects its type. If R executes pfence, copies head to p_rb_head, and persists it with pwb (Listing 7, lines 22–25). This ensures that p_rb_head is consistent with the log entries that have been processed. The backend thread reads the record R stored at head and inspects its type. If R then calls psync to flush all previously issued persistent stores to NVM. Note that although in principle, this psync can be replaced with pfence, using psync allows the backend to keep up with the frontend in terms of its persistence granularity. This ensures that under optimal load, the ring buffer is emptied at roughly the same rate as new records are produced thus minimizing frontend blocking and the system recovery time. Once a record has been processed the backend reads the record R stored at head and inspects its type. If R increments head modulo ringBufSize, and proceeds to handle the next record (Listing 7, lines 9). If the ring buffer is empty, the backend reads the record R stored at head and inspects its type. If R enters a busy-waiting loop (Listing 7, lines 11) where it copies p_rb_tail into tail, and checks whether b_head is still equal b_tail. If so, it pauses for a short while, and resumes the loop. Otherwise, it breaks from the loop and handles the next record on the log. Persistence cost. Let F and S denote the costs of pfence and psync respectively, and B be the average number of requests occupying a single flat combining batch. The total cost of persistence at the frontend is then F + S per batch or (F + S) / B per request on average. Likewise, the total cost of persistence at the backend is F + S per batch or the average per-request of (F + S) / B. The above indicates that the backend is able to process requests at the same rate as the frontend, and therefore, unlike, is not a bottleneck. Split Allocation Memory allocators are a challenge when porting applications to persistent memory since any modifications to allocator metadata must also be persisted. However, for many applications the memory allocator has been chosen carefully by developers. Forcing them to switch to a custom persistent allocator violates Mangosteen’s transparency goals. Furthermore, persisting every modification to an application’s address space is inefficient, since in many cases the corresponding data is transient and does not need to be persisted to ensure correct recovery (i.e. durable linearizability). For example, in Redis the contents of incoming request buffers do not need to be persisted. To overcome these challenges, we propose a novel split allocation scheme that allows applications to reuse their existing allocator while minimizing unnecessary persistence of transient data. During application initialization, the Mangosteen framework loads an additional instance of the application’s memory allocator. After application initialization completes, we use binary instrumentation to intercept all memory allocator functions (e.g., malloc, free). Memory allocator function calls that occur within processRequest during execution of a read-write command are directed to the persistent allocator. Other memory allocator operations, e.g. those that occur during calls to init / destroyAppClient, deserializeRequest or processRequest for RO commands, are considered transient and directed to the transient allocator. The combiner is permitted to access and modify objects allocated using the transient allocator (e.g., the incoming request buffer). However, any modifications to transient objects that occur within processRequest for read-write commands are intercepted by Mangosteen’s binary instrumentation. This allows them to be filtered based on the virtual memory address ranges assigned to the transient allocator by the operating systems (e.g. in response to calls to mmap / munmap, see Listing 6, line 13). Empirically, due to additional instrumentation overhead on the critical path it is found to be more expensive to filter transient writes from read-write commands at the front-end than to persist them in the redo log and perform filtering at the backend. To enable the backend to filter transient writes, the Mangosteen front-end therefore adds redo log entries for any internal calls to mmap / munmap made by the transient allocator. Such calls occur infrequently in practice. We emphasize that only transient writes that occur within processRequest for read-write commands are added to the redo log, with all other transient writes incurring no persistence overhead. Advantages of Mangosteen Several advantages are provided by the Mangosteen infrastructure disclosed herein. Mangosteen, is an easy-to-use programming framework to enable linearizable in- memory applications to gain durability using NVM. Mangosteen allows read-only requests to execute in parallel while batching and deduplicating read-write requests to minimize the cost of persistence. It enables applications to use their existing memory allocators unmodified, and employs a novel split allocation scheme to avoid unnecessary overhead for updates to transient memory that do not need to be persisted for correct recovery. The performance evaluation demonstrates Mangosteen-enabled applications achieve significant throughput improvements on realistic workloads compared to the state-of-the-art persistence frameworks. As Mangosteen only accesses NVM for read-write operations, its performance depends on the workload’s read-only to read-write ratio. FIG. 7 shows how peak throughput changes as we vary this ratio. Mangosteen outperforms Persimmon for all update ratios by between two to five times. Furthermore, Mangosteen’s throughput degrades gracefully and does not become bottlenecked by its backend, demonstrating the effectiveness of its persistence optimizations (e.g. write deduplication). In contrast, for write-intensive workloads Persimmon’s shadow execution with undo logging becomes the bottleneck at an update ratio of between 70–80%. In comparison to Redis [Wen Zhang, Scott Shenker, and Irene Zhang. Persistent state machines for recoverable in-memory storage systems with NVRAM. In Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation, OSDI’20, USA, 2020. USENIX Association], Mangosteen also achieves higher peak throughput even for an update-only workload because it is still able to parallelize some transient data processing (e.g. request parsing in deserializeRequest). Furthermore, concurrency control is provided that facilitates concurrent execution of read-only requests and read-write request batching to facilitate efficient logging and deduplication of write instructions. This minimizes the amount of data that needs to be written to the slower NVM storage and ensures that it is always written sequentially on a critical path. It also reduces the frequency of inefficient random access to the persistent media when the log is processed by the backend. The sequential writing maximizes the NVM throughput thanks to the efficient use of the internal buffer which is able to coalesce adjacent writes and persist them simultaneously. The serial (i.e., non-concurrent) access to the media further reduces contention on the internal device queues, which is known to increase the device bandwidth. Finally, the batching allows the implementation to reduce the frequency of highly inefficient persistent fence instructions to just 2 per batch under normal operation. If not otherwise stated or otherwise made clear from the context, the statement that two entities are different means that they perform different functions. It does not necessarily mean that they are based on different hardware. That is, each of the entities described in the present description may be based on a different hardware, or some or all of the entities may be based on the same hardware. It does not necessarily mean that they are based on different software. That is, each of the entities described in the present description may be based on different software, or some or all of the entities may be based on the same software. Each of the entities described in the present description may be embodied in the cloud. A further benefit of the disclosure herein is that transparency is provided between the non-durable and durable versions of the application. Transparency means that the computer program code present in the non-durable version of the application has very similar or identical logic to the computer program code present in the durable version of the application. Additionally, the disclosure herein provides for generality of application, since the approach may be applied many any different types of applications. Implementations of any of the above-described blocks, apparatuses, systems, techniques or methods include, as non-limiting examples, implementations as hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof. Some embodiments may be implemented in the cloud. It is to be understood that what is described above is what is presently considered the preferred embodiments. However, it should be noted that the description of the preferred embodiments is given by way of example only and that various modifications may be made without departing from the scope as defined by the appended claims.

Claims

Claims 1. A method for managing computer program code at a non-durable version of an application and a durable version of the application, the method comprising: determining computer program code for the non-durable version of the application; converting the computer program code from the non-durable version into the durable version, wherein the durable version of the application comprises volatile memory and non-volatile memory; determining that the computer program code comprises a request to be performed by the application; determining that the request is read-only; and upon determining that the request is read-only, processing the request over the volatile memory of the durable version of the application.

2. The method of claim 1, further comprising: concurrently executing multiple read-only requests of the computer program in parallel at the durable version of the application over the volatile memory.

3. The method of any preceding claim, wherein the read-only request comprises allowing data to be accessed at the application but not modified.

4. The method of any preceding claim, further comprising: converting the request to a format that can be interpreted by the application; performing deserialization, based on a type of the request, wherein the deserialization comprises converting the request from a binary representation to an in-memory representation.

5. The method of any preceding claim, further comprising: determining that the computer program code comprises a second request to be performed by the application; determining that the second request is read-write; and upon determining that the second request is read-write,processing the second request over the volatile memory of the durable version of the application, wherein processing the request comprises modifying data stored in the volatile memory; and copying the modified data stored in the volatile memory to the non- volatile memory.

6. The method of claim 5, further comprising: grouping together multiple requests into a batch of requests, wherein each of the multiple requests are read-write requests; processing the batch of requests sequentially at the durable version of the application.

7. The method of claims 5 and 6, wherein the read-write operation comprises allowing data to be both accessed and modified.

8. The method of any preceding claim, wherein the volatile memory is characterised by temporary storage of data in memory during processing; and the non-volatile memory is characterised by permanent storage of data in memory that can retain stored information even after power is removed.

9. The method of any preceding claim, wherein converting the computer program code from the non-durable version into the durable version comprises using dynamic code instrumentation.

10. The method of any of claims 1 to 9, wherein converting the computer program code from the non-durable version into the durable version comprises using static code instrumentation.

11. A method for managing computer program code at a non-durable version of an application and a durable version of the application, the method comprising: determining computer program code for the non-durable version of the application;converting the computer program code from the non-durable version into the durable version, wherein the durable version of the application comprises volatile memory and non-volatile memory; determining that the computer program code comprises a request to be performed by the application; determining that the request is read-write; and upon determining that the request is read-write processing the request over the volatile memory of the durable version of the application, wherein processing the request comprises modifying data stored in the volatile memory; and copying the modified data stored in the volatile memory to the non- volatile memory.

12. The method of claim 11, wherein: processing the request at the durable version of the application comprises modifying data of the durable version of the application.

13. The method of claims 11 or 12, further comprising: converting the request to a format that can be interpreted by the application; performing deserialization, based on a type of the request, wherein the deserialization comprises converting the request from a binary representation to an in-memory representation.

14. The method of any of claims 11 to 13, further comprising: grouping together multiple requests into a batch of requests, wherein each of the multiple requests are read-write requests; processing the batch of requests sequentially at the durable version of the application.

15. The method of any of claims 11 to 14, wherein the read-write request comprises allowing data to be accessed and modified.

16. The method of any of claims 11 to 15, wherein:the non-durable version of the application is characterised by temporary storage of data in memory during processing; and the durable version of the application is characterised by permanent storage of data in memory that can retain stored information even after power is removed.

17. The method of any of claims 11 to 16, wherein converting the computer program code from the non-durable version into the durable version comprises using dynamic code instrumentation.

18. The method of any of claims 11 to 16, wherein converting the computer program code from the non-durable version into the durable version comprises using static code instrumentation.

19. The method of any preceding claim, wherein upon converting the computer program code from the non-durable version into the durable version, logic of the computer program code is not modified.

20. An application programming interface, API, comprising means for carrying out the method of any preceding claim.

21. The API of claim 20, further comprising: a front-end comprising a random access memory, RAM, component suitable for storing the computer program code and data of the durable version of an application; a back-end comprising a non-volatile memory, NVM, component suitable for storing the computer program code and data of the durable version of an application.

22. Computer program instructions, configured to perform the method of any of claims 1 to 19.

Images