Systems, methods, and media for client updates to data modification tracking structures

Abstract

The present disclosure relates to client updates to data modification tracking structures. In some examples, in response to a modification request by a client to a first data page at a memory server that is remotely accessed by the client over a network, the client system sends a request to the memory server to update a data modification tracking structure stored by the memory server to indicate that the first data page was modified. The client system initiates an incremental data backup from the memory server to a backup storage system of the data page that is indicated as having been modified by the data modification tracking structure stored at the memory server.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to systems, methods, and media for client updates to data modification tracking structures. BACKGROUND

[0002] Fabric attached memory refers to memory that is accessible by any of a plurality of clients through a fabric. A "fabric" can refer to a network that allows communication between computing nodes connected to the network. Fabric attached memory can be implemented using memory devices such as flash memory devices or other types of persistent memory devices. SUMMARY

[0003] A non-transitory machine-readable storage medium is disclosed that includes instructions that, when executed, cause a client system to: in response to a modification request by the client to a first data page at a memory server that is remotely accessed by the client over a network, send, from the client system to the memory server, a request to update a data modification tracking structure stored by the memory server to indicate that the first data page was modified; and initiate, by the client system, an incremental data backup from the memory server to a backup storage system of data pages indicated by the data modification tracking structure stored at the memory server as having been modified. BRIEF DESCRIPTION OF DRAWINGS

[0004] Some embodiments of the present disclosure are described with respect to the following figures.

[0005] Figure 1 is a block diagram of an arrangement according to some examples, including a fabric attached memory including a memory server coupled to a network, a client system that is able to access the fabric attached memory over the network, and a backup storage system to which an incremental data backup can be performed.

[0006] Figure 2 is a message flow diagram of a process to write data from a client system to a fabric attached memory according to some examples.

[0007] Figure 3 is a message flow diagram of a process to perform an incremental data backup from a fabric attached memory to a backup storage system according to some examples.

[0008] Figure 4 is a block diagram of a storage medium storing machine-readable instructions according to some examples.

[0009] Figure 5 is a block diagram of a fabric attached memory system according to some examples.

[0010] Figure 6 is a flow diagram of a process in accordance with some examples.

[0011] In all of the drawings, like reference numerals refer to similar but not necessarily identical elements. The drawings are not necessarily to scale and the dimensions of some parts can have been exaggerated for the sake of clarity and / or illustration. Additionally, the drawings provide examples and / or embodiments consistent with the description; however, the description is not limited to the examples and / or embodiments provided in the drawings. DETAILED DESCRIPTION

[0012] Clients are able to perform remote access (remote read or remote write) to the structure-attached memory. A“client” can refer to any entity (machine or program) that is able to issue a request to access data in the structure-attached memory. Examples of networks through which a client is able to access the structure-attached memory include any one or some combination of: a COMPUTE EXPRESS LINK (CXL) interconnect, a Slingshot interconnect, an InfiniBand interconnect, and the like.

[0013] The structure-attached memory can include a distributed arrangement of memory servers, where each memory server can include or be coupled to persistent memory. In some examples, a client can access the structure-attached memory over a network using remote direct memory access (RDMA).

[0014] RDMA data transfer from a client to a memory server involves transferring data from the client to the persistent memory of the memory server over a network, where the data transfer does not involve work performed by a host processor of the memory server. A“host processor” of a memory server can refer to a processor that executes an operating system (OS) and other machine-readable instructions of the memory server, including firmware such as a basic input / output system or BIOS and applications.

[0015] In RDMA data transfer, data is transferred into or out of a client from the persistent memory of a memory server over a network through a network interface of the memory server. A“network interface” can refer to a network communication component that transmits and receives signals, messages, packets, and the like (more generally referred to as“information”) over a network. A network interface can include a network transceiver that transmits and receives information.

[0016] A memory server can include a main memory that includes a set of memory devices (a single memory device or multiple memory devices), such as dynamic random access memory (DRAM) devices, static random access memory (SRAM) devices, and the like. A memory server can also include a cache memory that can be part of or associated with a main processor. The cache memory is associated with the main processor if the main processor uses the cache memory to store data used by the main processor.

[0017] Note that data for an RDMA data transfer passes through a network interface of a memory server, so the data can be stored in a memory of the network interface and can be processed by a processor of the network interface. However, the processor and memory of the network interface are separate and distinct from the main processor and a cache memory that is part of or associated with the main processor.

[0018] Data backups can be formed by the structural attached memory. Data backups involve copying data at a set of memory servers (a single memory server or multiple memory servers) to a backup storage system. The backup storage system can include a set of storage devices (a single storage device or multiple storage devices), where the storage devices can be implemented using disk-based storage devices, solid state drives, and the like. If data in the structural attached memory is lost or has an unrecoverable error, a recovery operation can be performed from the backup storage system to recover the lost data or to recover to a previous version of the data that is free of errors.

[0019] In some examples, a full data backup can copy all data of the structural attached memory to the backup storage system. However, if the structural attached memory stores a large amount of data, a full data backup can be resource intensive (e.g., in terms of use of processing resources, use of communication resources, and the like) and can have a relatively long completion time (e.g., hours or days).

[0020] In other examples, an incremental data backup can be performed, where modified data portions are copied to the backup storage system without unmodified data portions being copied to the backup storage system. A data portion is “modified” if the data portion has been changed relative to a copy of the data portion stored at the backup storage system 130.

[0021] Incremental data backups can reduce backup time and resource usage. Moreover, performing incremental data backups can reduce wear and tear on hardware, such as storage media, as the amount of writes to the storage media can be reduced compared to full data backups.

[0022] In an example using RDMA, because the host processor of the memory server is not involved in the RDMA data transfer, the host processor of the memory server does not track modified data portions (also referred to as "dirty" data portions). If the memory server of the structure attached memory does not track modified data portions, then incremental data backup will not be possible because the memory server does not know which data portions stored at the memory server are modified (or dirty) and which data portions are unmodified (or clean).

[0023] According to some embodiments of the present disclosure, in response to an update by a client to a data page (or more simply referred to as a "page") of a memory server over a network, a client triggered update to a data modification tracking structure at the memory server of the structure attached memory as part of a remote access arrangement is performed. A "remote access arrangement" can refer to an arrangement in which a client accesses data stored at a remote memory server coupled to the client over a network. A "page" can refer to any specified amount of data (the term "page" can be used interchangeably with "data portion"). A page is a unit of data for which the data modification tracking structure tracks data modification.

[0024] The data modification tracking structure tracks pages that have been modified. The data modification tracking structure is used to perform an incremental backup operation to a backup storage system in which modified pages of the memory server are copied from the structure attached memory to the backup storage system but unmodified pages are not copied from the structure attached memory to the backup storage system.

[0025] By using the client triggered update to the data modification tracking structure, the memory server will not have to be configured with logic to track modified pages. This can reduce the complexity and cost of the memory server of the structure attached memory.

[0026] Figure 1 is a block diagram of an example arrangement including a structure attached memory 102 that includes a distributed arrangement of memory servers 104 connected to a network 106. Client systems 108-1 to 108-N (N ≥ 1) are also connected to the network 106. Each client system can access the structure attached memory 102.

[0027] Figure 1An arrangement of components of client system 108-1 is shown. The remaining client system(s) (including 108-N) can have the same arrangement of components or a different arrangement of components. Client system 108-1 includes a network interface 110 for communicating over network 106, a processor 112 for executing machine-readable instructions (including firmware, an operating system (OS), application programs, etc.) of client system 108-1. Client system 108-1 also includes a local memory 114 for storing local data at client system 108-1. A “processor” can include a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or another hardware processing circuit.

[0028] Examples of client systems include any one or some combination of a desktop computer, a notebook computer, a tablet computer, a server computer, a smartphone, a gaming appliance, an Internet of Things (IoT) device, etc.

[0029] Client systems are examples of “clients” that can issue requests to perform remote access to structure-attached memory 102. Programs (including machine-readable instructions) executing in client systems 108 can be another example of “clients.”

[0030] According to some embodiments of the present disclosure, client system 108-1 includes an incremental backup management engine 116. As used herein, an “engine” can refer to one or more hardware processing circuits, which can include any one or some combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or another hardware processing circuit. Alternatively, an “engine” can refer to a combination of one or more hardware processing circuits and machine-readable instructions (software and / or firmware) executable on the one or more hardware processing circuits.

[0031] Incremental backup management engine 116 in client system 108-1 can perform client-triggered management of modified page tracking by updating a modified page tracking structure (discussed further below), and incremental backup management engine 116 can use the modified page tracking structure to initiate, over network 106, an incremental data backup from structure-attached memory 102 to a backup storage system 130 connected to network 106.

[0032] Incremental backup operations from the set of memory servers 104 to a backup storage system 130 can be performed over the network 106. The backup storage system 130 can be located at a location remote from the structure attached memory 102. The backup storage system 130 includes a backup storage medium 136 for storing instances of incremental backup data 138. The backup storage medium 136 can include a set of storage devices (a single storage device or multiple storage devices).

[0033] The instances of incremental backup data 138 can represent copies of backup data taken at different points in time. For example, a first instance of incremental backup data 138 can include copies of modified pages at a first point in time, a second instance of incremental backup data 138 can include copies of modified pages at a second point in time, and so on.

[0034] The backup storage system 130 includes a storage controller 140 for controlling access (read access and write access) to the backup storage medium 136. For example, the storage controller 140 can interact with the set of memory servers 104 to perform incremental backup operations, resulting in instances of incremental backup data 138 stored in the backup storage medium 136. The storage controller 140 can also interact with the set of memory servers 104 to perform restore operations, resulting in data being restored from instances of incremental backup data 138 to the set of memory servers 104.

[0035] Each memory server 104 includes a host processor 118 for executing an OS 120 of the memory server 104, and for executing other machine-readable instructions, such as firmware (e.g., BIOS code), applications, and so on. Note that each memory server 104 can include multiple different types of processors (including the host processor 118) to perform respective different functions. A "host processor" can include a single processor or multiple processors.

[0036] Each memory server 104 also includes persistent memory 122 (which is the main memory of the memory server 104) that can store data on behalf of the client systems 108-1 through 108-N. "Persistent" memory refers to memory that is capable of retaining data stored in the memory even if the memory is powered off. For example, the persistent memory can include non-volatile memory, where the stored data is retained (i.e., not lost) even at power off. In another example, the persistent memory can refer to an arrangement where the data of the memory is flushed to a backup storage (e.g., disk-based storage, etc.) at power off, and then the data is restored from the backup storage to the memory after power is restored.

[0037] The memory servers 104 also include network interfaces 124 that enable the memory servers 104 to communicate over the network 106.

[0038] In examples in which a client system such as the client system 108-1 is capable of performing RDMA access to the structure attached memory 102, an RDMA data transfer will cause data to be transmitted from the client system to the selected memory servers 104 of the structure attached memory 102 over the network 106. The data of the RDMA data transfer will pass through the network interfaces 124 of each selected memory server 104 and be written to the persistent memory 122 of each selected memory server 104 without involving the host processor 118 of each selected memory server 104. In the RDMA data transfer, the processing cycles of the host processor 118 of the memory servers 104 are not used to put the data into the persistent memory 122.

[0039] In examples in which the structure attached memory 102 is implemented as a distributed memory system, each persistent memory 122 of a respective memory server 104 stores a modified page tracking bitmap 132 as an example of the modified page tracking structure mentioned further above. The modified page tracking bitmap 132 contains a set of bits (e.g., an array of bits) that represent the modification state of respective pages 134 stored in the persistent memory 122 of the respective memory server 104. Although the modified page tracking bitmap 132 is shown as being stored in the persistent memory 122, in other examples, the modified page tracking bitmap 132 can be stored in a different memory of the memory server 104, such as in a cache memory or another memory. Figure 1 Figure 1 Any given bit of the modified page tracking bitmap 132 can have a first value or a second value that is different from the first value. For example, the first value can be 1 or 0, and the second value can be 0 or 1 (i.e., the value opposite the first value). More specifically, if the value of a bit is 1 (an example of the first value), this indicates that the corresponding page is modified. On the other hand, if the value of a bit is 0 (an example of the second value), this indicates that the corresponding page is not modified. If a given bit of the modified page tracking bitmap 132 has the first value, this indicates that the corresponding page 134 is modified. On the other hand, if a given bit of the modified page tracking bitmap 132 has the second value, this indicates that the corresponding page 134 is not modified. A page is “modified” if it has been changed relative to a copy of the page stored at the backup storage system 130.

[0040] Any given bit of the modified page tracking bitmap 132 can have a first value or a second value that is different from the first value. For example, the first value can be 1 or 0, and the second value can be 0 or 1 (i.e., the value opposite the first value). More specifically, if the value of a bit is 1 (an example of the first value), this indicates that the corresponding page is modified. On the other hand, if the value of a bit is 0 (an example of the second value), this indicates that the corresponding page is not modified. If a given bit of the modified page tracking bitmap 132 has the first value, this indicates that the corresponding page 134 is modified. On the other hand, if a given bit of the modified page tracking bitmap 132 has the second value, this indicates that the corresponding page 134 is not modified. A page is “modified” if it has been changed relative to a copy of the page stored at the backup storage system 130.

[0041] ​In other examples, other forms of modified page tracking structures can be employed in place of the modified page tracking bitmap 132. More generally, the modified page tracking structure includes a set of entries, where each entry can have a first value indicating that a corresponding page is modified and a second value indicating that the corresponding page is not modified. Thus, a first entry of the modified page tracking structure can indicate whether a first page is modified, a second entry of the modified page tracking structure can indicate whether a second page is modified, and so on.

[0042] Figure 2 is a message flow diagram of a process by which the client system 108-1 writes data to the structure attached memory 102. Although Figure 2 shows operations involving one memory server 104 in the structure attached memory 102, in other examples, Figure 2 the process can involve multiple memory servers 104 in the structure attached memory 102. Further, although Figure 2 shows a particular order of tasks, in other examples, the tasks can be performed in a different order, some tasks can be omitted or replaced, and / or additional tasks can be added.

[0043] The client system 108-1 receives (at 202) a write request to write data to the structure attached memory 102. In Figure 2 examples, the write request is a Put(FAM_Location, Write_Data, Data_Size) request. The write request can be sent by a requesting program in the client system 108-1 (e.g., a program in the client system 108-1) or a requesting program external to the client system 108-1. The parameter Fam_Location in the Put() request includes a memory address, such as a memory address within the persistent memory 122 of the memory server 104. In some examples, the memory address can be a start address offset that identifies a memory location in the persistent memory 122 to which the write data represented by Write_Data is to be written. In some examples, the Put() request can also include a Data_Size parameter that specifies a size of the write data Write_Data.

[0044] In response to the Put() request, the client system 108-1, and more specifically the incremental backup management engine 116, determines (at 204) the location of the modified page tracking bitmap 132 in the memory server 104. The location of the modified page tracking bitmap 132 can be indicated by a location in the persistent memory 122 or a memory address of a location in another memory in the memory server 104. In some examples, the incremental backup management engine 116 can determine the location of the modified page tracking bitmap 132 based on configuration information stored in the client system 108-1 or based on information retrieved from a source external to the client system 108-1.

[0045] The client system 108-1 determines (at 206) which page(s) in the persistent memory 122 of the memory server 104 are involved in the write operation specified by the Put() request.

[0046] In some examples, the client system 108-1 can map memory addresses of the persistent memory 122 of the memory server 104 to respective page numbers. For example, since the size of each page is predefined, the page contains data in a respective range of memory addresses. Given a memory address specified by FAM_Location, the client system 108-1 can identify which page includes the memory address specified by FAM_Location. The identified page has a page number that refers to the identified page. In this way, the client system 108-1 can identify a starting page number of a starting page that contains the memory address specified by FAM_Location.

[0047] Furthermore, once the starting page is identified, the client system 108-1 can determine, based on the size of the write data specified by Data_Size, whether the write data is to be written to only the starting page or to a set of the starting page and other page(s). The size of the write data is such that the write data is to be written to the starting page plus other page(s), then the pages involved in the write operation will include the starting page and the contiguous page(s) after the starting page.

[0048] The incremental backup management engine 116 in the client system 108-1 then issues (at 208) a remote memory access (RMA) scatter operation (at 208) to the structure-attached memory 102. The RMA scatter operation performs both the data update of the write request (to update the page(s) determined at 206 to be written to) and the bitmap update of the modified page tracking bitmap 132 in the memory server 104. The RMA scatter operation refers to an operation that initiates updates to different data structures at the structure-attached memory by a remote entity (in this case, the client system 108-1).

[0049] In Figure 2 In the example, in one input / output (I / O) operation, the data structures to be updated by the RMA scatter operation include the modified page tracking bitmap 132 and the page(s) determined at 206. With the RMA scatter operation, one I / O operation can be issued to update the page(s) and the modified page tracking bitmap 132, instead of multiple different I / O operations to do so. The RMA scatter operation is issued by sending an RMA scatter request that specifies the page number(s) (or alternatively, memory addresses) of the page(s) involved in the update, and the memory address of the modified page tracking bitmap 132 to be updated to reflect the modified state of the page(s).

[0050] In some examples, the RMA scatter operation is conducted according to RDMA, in which the update to the data in the persistent memory 122 of the memory server 104, including the page(s) and the modified page tracking bitmap 132, is performed without involving the host processor 118 of the memory server 104.

[0051] In response to the RMA scatter operation, the memory server 104 updates (at 210) the page(s) identified by the page number(s) specified by the RMA scatter request, and the memory server 104 updates (at 212) the corresponding bit(s) of the modified page tracking bitmap 132. The bit(s) of the updated modified page tracking bitmap 132 reflect the modified state of the page(s) identified by the page number(s) specified by the RMA scatter request.

[0052] In the example, if a single page is updated in response to the RMA scatter operation, a single bit in the modified page tracking bitmap 132 corresponding to the single page changes from a second value (e.g., 0, indicating not modified) to a first value (e.g., 1, indicating modified). On the other hand, if multiple pages are modified in response to the RMA scatter operation, multiple bits in the modified page tracking bitmap 132 corresponding to the multiple pages change from the second value to the first value.

[0053] The client system waits (at 214) for completion of the RMA scatter operation. In some examples, when the page(s) and the modified page tracking bitmap 132 have been updated according to the RMA scatter operation, the memory server 104 can return a completion indication to the client system 108-1.

[0054] In response to receiving the completion indication from the memory server 104, the client system 108-1 returns (at 216) the completion indication to the requestor that sent the Put() request. The requestor can be in the client system 108-1 or external to the client system 108-1.

[0055] Figure 3 is a message flow diagram of a client-initiated incremental data backup operation to perform an incremental data backup from the fabric-attached memory 102 to the backup storage system 130. Although Figure 3 the example process is shown as involving only a single memory server 104 in the fabric-attached memory 102, in other examples, Figure 3 the process of can involve multiple memory servers 104. Also, although Figure 3 a particular order of tasks is shown, in other examples, the tasks can be performed in a different order, some tasks can be omitted or replaced, and / or additional tasks can be added.

[0056] The client system 108-1 receives (at 302) a data backup request to perform a data backup from the fabric-attached memory 102 to the storage system 130. In Figure 3 the example, the data backup request is a Backup(FAM_Region) request. The data backup request is received from a requestor, which can be internal to the client system 108-1 or external to the client system 108-1.

[0057] The parameter FAM_Region in the Backup(FAM_Region) request specifies a memory region of the fabric-attached memory 102. A memory region can be a portion of the fabric-attached memory 102 that is allocated to store one data item (or multiple data items). A "data item" can refer to a file, an object, a collection of files or objects, or any other identifiable unit of data. The parameter FAM_Region can have an identification value that identifies the memory region. Different memory regions of the fabric-attached memory 102 are assigned different identification values (e.g., different numbers, different strings, etc.).

[0058] The memory region can be distributed across multiple memory servers 104 (i.e., the data of the memory region is stored in multiple persistent memories 122 of the memory servers 104). Note that the parameter FAM_Location in the Put() request received (at 202 in Figure 2 ) can refer to a memory address in the memory region.

[0059] In response to the Backup(FAM_Region) request, the incremental backup management engine 116 of the client system 108-1 sends a backup request (e.g., a FAM_Backup(FAM_Region) request) at 304.

[0060] In response to the FAM_Backup(FAM_Region) request, storage server 104 locks (at 306) the storage region identified by FAM_Region. Locking storage server 104 means preventing any further data updates in the storage region; that is, new data cannot be written to the storage region, and the data in the storage region cannot be modified.

[0061] In some examples, locking a memory region identified by FAM_Region includes unregistering (at 308) the memory region identified by FAM_Region. Memory server 104 may maintain a registered memory region data structure (e.g., a list or other type of data structure) whose storage has been allocated to represent memory regions used to store data on behalf of client systems. Once a new memory region is allocated and ready to accept data updates, the new memory region is "registered" to memory server 104 by adding an identifier value identifying the new memory region to the registered memory region data structure. Unregistering a given memory region may refer to deleting the identifier value of the given memory region from the registered memory region data structure. Unregistered memory regions attached to memory 102 cannot be updated via network 106.

[0062] When a memory region identified by FAM_Region is locked, any writes to that memory region will be redirected (at 310) to log 142 in memory server 104. In other words, in response to a write request to the locked memory region, the write request is redirected to log 142, and no write is performed on the locked memory region.

[0063] In some examples, such as Figure 1 As shown, log 142 can be stored in persistent storage 122 of storage server 104. In other examples, log 142 can be stored in another storage device of storage server 104.

[0064] A "log" can refer to any data structure that stores write requests for a memory region that cannot be processed because the memory region is locked.

[0065] In other examples, a different locking mechanism can be used instead of logging the attachment of a memory region of the structure-attached memory 102 to lock the memory region. For example, each memory region can be associated with a lock indicator (e.g., a flag) that indicates the memory region is locked if set to a first value and indicates the memory is not locked if set to a different second value.

[0066] After locking (at 306) the memory region, the memory server 104 returns (at 312) an indication of a backup in progress to the client system 108-1 to indicate that the data backup is in progress.

[0067] The memory server 104 performs (at 314) an incremental backup loop in which the memory server 104 iteratively copies the modified pages of the memory server 104 to the backup storage system 130. The memory server 104 identifies (at 316) the pages that have been modified (i.e., the corresponding bit of the page has the first value) based on the modified page tracking bitmap 132. The memory server 104 then copies (at 318) the identified modified pages to the backup storage system 130. After the pages are copied to the backup storage system 130, the memory server 104 resets (at 320) the corresponding bit in the modified page tracking bitmap 132, changing the corresponding bit from the first value to the second value to indicate that the page was not modified.

[0068] The memory server 104 then performs another iteration to identify another modified page to backup to the backup storage system 130 and continues the incremental backup loop 314 until all of the modified pages identified by the modified page tracking bitmap 132 have been backed up.

[0069] In some examples, note that the modified pages are not written to the backup storage system 130 individually in different I / O operations; rather, one I / O operation can write all of the modified pages identified in the incremental backup loop 314 to the backup storage system 130.

[0070] Upon completion of the incremental backup loop 314 (i.e., after copying all of the modified pages identified by the modified page tracking bitmap 132 to the backup storage system 130), the memory server 104 accesses (at 322) the log 142 and replays the write requests in the log 142 to complete the I / O operations specified by the write requests. The replayed write requests in the log 142 cause the corresponding data to be written to the memory region (which remains locked for write requests received from the client system).

[0071] After all of the write requests in the log 142 have been played back, the memory server 104 unlocks (at 324) the locked memory region (identified by FAM_Region) to enable updates to the unlocked memory region in response to write requests from client systems. Unlocking the memory region can include registering the memory region with a registered memory region data structure.

[0072] In some examples, a client system or a request program that is part of or coupled to a client system can use a backup status application programming interface (API) to check the status information of all previously issued backups. For example, the client system or request program can send a status query including an identifying value of a memory region to the backup status API, which returns information about when incremental backups were performed on the memory region, as well as information identifying each incremental backup instance. Such information can be used to perform a recovery operation, for example, in the event of data loss or data corruption at the client system or request program.

[0073] Figure 4 is a block diagram of a non-transitory machine-readable or computer-readable storage medium 400 that stores machine-readable instructions that, when executed, cause a client system to perform various tasks.

[0074] The machine-readable instructions include data modification tracking structure update instructions 402 to, in response to a modification request by a client for a first data page at a memory server that is remotely accessed by the client over a network, send a request from the client system to the memory server to update a data modification tracking structure stored by the memory server to indicate that the first data page was modified. In some examples, the data modification tracking structure includes Figure 1 a modified page tracking bitmap 132 of the memory server.

[0075] In some examples, the machine-readable instructions receive a write request to write data to a structure-attached memory that includes the memory server, and in response to the write request, the machine-readable instructions determine a set of data pages (one data page or multiple data pages) involved in a write operation specified by the write request, and issue an I / O scatter operation (e.g., an RMA scatter operation issued at 208 in Figure 2 the data modification tracking structure to update the set of data pages involved in the write operation.

[0076] In some examples, the issuance of the I / O scatter operation includes sending a scatter request that includes information identifying the set of data pages (e.g., page numbers and / or memory addresses of the pages) and a memory address of the data modification tracking structure.

[0077] The machine-readable instructions include incremental data backup initiation instructions 404 for initiating, by the client system, an incremental data backup of data pages from the memory server to the backup storage system, the data pages indicated as having been modified by the data modification tracking structure stored at the memory server.

[0078] In some examples, the memory server is part of a structure attached memory that includes a plurality of memory servers, where the incremental data backup is a backup of data pages from a plurality of the memory servers of the plurality of memory servers to the backup storage system.

[0079] In some examples, each of the plurality of memory servers includes a respective data modification tracking structure indicating which data pages are modified, where the incremental data backup is a backup of data pages identified as having been modified by the data modification tracking structures of the plurality of memory servers from the plurality of memory servers to the backup storage system.

[0080] In some examples, the modification request is for performing the modification of the first data page according to an RDMA write by the client system to the memory of the memory server over the network.

[0081] In some examples, the updating of the data modification tracking structure is according to an RDMA write by the client system to the memory of the memory server over the network.

[0082] Figure 5 is a block diagram of a structure attached memory system 502 according to some examples. The structure attached memory system 502 includes an arrangement of processing, communication, and storage resources to provide structure attached memory for remote access by a client system over a network.

[0083] The structure attached memory system 502 includes a memory server 504 that includes a memory 506 for storing a data modification tracking structure 508.

[0084] The structure attached memory system 502 includes a network interface 510 for communicating with the client system over the network. The network interface 510 can receive an update request from the client system to update the data modification tracking structure 508. The update request can be an RDMA write to the data modification tracking structure 508 in the memory 506. The update request is in response to a write request to write data from the client system to the memory server 504, where the update request is to cause a modification to the data modification tracking structure 508 to indicate that a set of data pages are modified in response to the write request.

[0085] The memory server 504 includes a processor 512 for performing various tasks (e.g., Figure 1The processor that performs a task can refer to a single processor that performs the task or multiple processors that perform the task.

[0086] The tasks include a backup request receiving task 514 that receives a backup request from a client system. The tasks include an incremental data backup task 516 that identifies a set of data pages that have been modified based on the data modification tracking structure 508 and performs an incremental data backup of the set of data pages from the memory server 504 to a backup storage system in response to the backup request.

[0087] In some examples, the update request includes an RDMA write to the memory 506 of the memory server 504 to update the data modification tracking structure 508.

[0088] In some examples, the update request is part of an I / O scatter operation that updates the data modification tracking structure 508 and the set of data pages.

[0089] In some examples, the processor 512 is to lock a memory region in the memory 506 of the memory server 504 in response to the backup request from the client system to prevent other write requests from the client system from updating the memory region and to redirect the other write requests to a log in the memory server 504.

[0090] In some examples, the processor 512 is to replay the other write requests in the log after the incremental data backup is complete to update the memory region according to the other write requests.

[0091] In some examples, the processor 512 is to unlock the memory region after replaying the other write requests in the log.

[0092] Figure 6 is a flowchart of a process 600 according to some examples. The process 600 includes receiving (at 602) a write request at a client system to write data to a structure-attached memory over a network. The write request can be from a requesting program in the client system or from a requesting program outside the client system.

[0093] The process 600 includes sending (at 604) a request from the client system to a memory server in the structure-attached memory to update a data modification tracking structure stored by the memory server to indicate that a set of data pages was modified by the write request in response to the write request. In some examples, the request to update the data modification tracking structure is part of a scatter request that identifies both the data pages to update and the data modification tracking structure to update.

[0094] Process 600 includes an incremental data backup initiated by the client system (at 606) from the storage server to the backup storage system of the data page set, which is indicated by a data modification tracking structure stored at the storage server as having been modified.

[0095] In this disclosure, unless the context clearly indicates otherwise, the terms “a / an” or “(the)” are intended to include plural forms as well. Furthermore, the terms “includes / including / comprises / comprising” and “have or having” as used in this disclosure specify the presence of an element, but do not preclude the presence or addition of other elements.

[0096] Storage media (e.g., Figure 4 The 400 in the specification can include any one or a combination of the following: semiconductor memory devices such as dynamic or static random access memory (DRAM or SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory; disks such as fixed disks, floppy disks, and removable disks; another magnetic medium, including magnetic tape; optical media such as optical discs (CDs) or digital video discs (DVDs); or other types of storage devices. Note that the instructions discussed above may be provided on a single computer-readable or machine-readable storage medium, or alternatively on multiple computer-readable or machine-readable storage media distributed across a large system having potentially multiple nodes. Such one or more computer-readable or machine-readable storage media are considered part of an article (or article of manufacture). An article or article of manufacture can refer to any single or multiple manufactured components. One or more storage media may be located in a machine that executes the machine-readable instructions, or at a remote site from which machine-readable instructions can be downloaded via a network for execution.

[0097] In the foregoing description, numerous details have been set forth to provide an understanding of the subject matter disclosed herein. However, embodiments may be practiced without some of these details. Other embodiments may include modifications and variations to the details discussed above. The appended claims are intended to cover such modifications and variations.

Claims

1. A non-transitory machine-readable storage medium comprising instructions that, when executed, cause a client system to: in response to a modification request by the client to modify a first data page at a memory server that is remotely accessed by the client over a network, send, from the client system to the memory server, a request to update a data modification tracking structure stored by the memory server to indicate that the first data page was modified; and initiate, by the client system, an incremental data backup from the memory server to a backup storage system of data pages indicated by the data modification tracking structure stored at the memory server as having been modified, wherein the update of the data modification tracking structure is performed in accordance with a remote direct memory access (RDMA) write by the client system to memory of the memory server over the network.

2. The non-transitory machine-readable storage medium of claim 1, wherein, the memory server is part of a structure attached memory comprising a plurality of memory servers, and wherein the incremental data backup is a backup of data pages from more than one of the plurality of memory servers to the backup storage system.

3. The non-transitory machine-readable storage medium of claim 2, wherein, each of the more than one of the memory servers comprises a respective data modification tracking structure indicating which data pages were modified, and wherein the incremental data backup is a backup of the data pages identified by the data modification tracking structures in the more than one of the memory servers as having been modified from the more than one of the memory servers to the backup storage system.

4. The non-transitory machine-readable storage medium of claim 1, wherein, the data modification tracking structure comprises a modified page tracking bitmap comprising a plurality of bits indicating which data pages were modified.

5. The non-transitory machine-readable storage medium of claim 1, wherein, the instructions, when executed, cause the client system to: receive a write request to write data to a structure attached memory comprising the memory server; and in response to the write request: determine a set of data pages involved in a write operation specified by the write request, and issue an input / output (I / O) scatter operation to the structure attached memory to update the set of data pages involved in the write operation and to update the data modification tracking structure.

6. The non-transitory machine-readable storage medium of claim 5, wherein, the issuance of the I / O scatter operation comprises sending a scatter request comprising information identifying the set of data pages and a memory address of the data modification tracking structure.

7. The non-transitory machine-readable storage medium of claim 5, wherein, the I / O scatter operation is to cause the memory server to update the set of data pages and the data modification tracking structure.

8. The non-transitory machine-readable storage medium of claim 1, wherein, the modification request is to perform the modification of the first data page in accordance with a remote direct memory access (RDMA) write by the client system to memory of the memory server over the network.

9. A structure attached memory system comprising: a memory server comprising memory to store a data modification tracking structure; a network interface to communicate with a client system over a network, the network interface to receive an update request from the client system to update the data modification tracking structure, the update request in response to a write request to write data from the client system to the memory server, wherein the update request is to cause modification to the data modification tracking structure to indicate that a set of data pages are modified in response to the write request; and a processor to: receive a backup request from the client system; and in response to the backup request: identify the set of data pages that have been modified based on the data modification tracking structure, and perform an incremental data backup of the set of data pages from the memory server to a backup storage system, wherein the update request comprises a remote direct memory access (RDMA) write to the memory of the memory server to update the data modification tracking structure.

10. The structure-attached memory system of claim 9, wherein, the update request is part of an input / output (I / O) scatter operation to update the data modification tracking structure and the set of data pages.

11. The structure-attached memory system of claim 10, wherein, the I / O scatter operation is in response to a scatter request, the scatter request comprising information identifying the set of data pages and a memory address of the data modification tracking structure.

12. The structure-attached memory system of claim 9, wherein, the processor to: in response to the backup request from the client system: lock a memory region in the memory of the memory server to prevent other write requests from the client system from updating the memory region, and redirect the other write requests to a log in the memory server.

13. The structure-attached memory system of claim 12, wherein, the processor to: after the incremental data backup is complete, replay the other write requests in the log to update the memory region according to the other write requests.

14. The structure-attached memory system of claim 13, wherein, the processor to: after replaying the other write requests in the log, unlock the memory region.

15. The structure-attached memory system of claim 12, wherein, the memory region is identified by the backup request from the client system.

16. A method for structurally attached memory, comprising: receiving, at a client system, a write request to write data to a structurally attached memory over a network; in response to the write request, sending, from the client system, a request to a memory server in the structurally attached memory to update a data modification tracking structure stored by the memory server to indicate that a set of data pages are modified by the write request; and initiating, by the client system, an incremental data backup from the memory server to a backup storage system of the set of data pages indicated as modified by the data modification tracking structure stored at the memory server, wherein the request to update the data modification tracking structure comprises a remote direct memory access (RDMA) write to the structurally attached memory.

17. The method of claim 16, wherein, the request to update the data modification tracking structure is part of a scatter request sent to the memory server to update both the set of data pages and the data modification tracking structure.

Images