JOURNALS FOR DATA CLONING PROCESSES

Abstract

A computer-implemented method comprising: detecting (510) a cloning operation of a manifest area by a memory controller (110) of a deduplication storage system; loading a journal (130, 320) from persistent storage (140) into memory in response to the detected cloning operation, wherein the journal (130, 320) serves to store changes to a container index (220) associated with the manifest area, and wherein the container index (220) is not loaded into memory in response to the detected cloning operation; and updating the journal (220) in memory to include a display of changes to metadata of the container index (220) that is not loaded into memory, wherein the changes to the metadata are associated with the detected cloning operation.

Description

background Data reduction techniques can be used to decrease the amount of data stored in a storage system. One example of a data reduction technique is data deduplication. Data deduplication identifies duplicate data units and attempts to reduce or eliminate the number of duplicate data units stored in the storage system. US 2020 / 0241784A1 discloses a region cloning process that includes retrieving a clone request to clone a logical range of a source backup. The clone request can specify the source backup, a logical start offset, and a logical end offset. A source manifest associated with the logical range of the source backup can be determined. Based on the logical start offset, a start entry of the source manifest can be identified. This start entry can represent a start data block. It can be determined whether the logical start offset represents an intermediate position of the start data block. If so, a start subblock entry representing a subrange of the start data block can be appended to a target manifest. Based on the target manifest with the appended start subblock entry, a backup of the logical range of the source backup can be created. US 2015 / 0039577A1 discloses an atomic storage module that can be configured to perform atomic storage operations on a first set of identifiers with respect to a second, different set of identifiers. Upon completion of the atomic storage operation, the module can move the corresponding data to the first target set of identifiers. The move may involve modifying a logical interface of the data. Furthermore, the move may involve storing persistent metadata configured to bind the data to the first set of identifiers. US 2015 / 0052112A1 discloses a file server connected via a network to a client terminal, comprising a storage unit for storing received files and a control unit for controlling the writing or reading of files to or from the storage unit, wherein the control unit: performs deduplication by designating one of the files stored in the storage unit with identical content as the clone source file and designating another file as the clone file that references data in the clone source file; and appends data to the clone source file according to an update instruction for the clone file from the client terminal. US 2017 / 0123931A1 discloses a distributed object storage system that supports snapshots and clones without requiring any form of distributed locking or centralized processing. A clone tree can be modified in isolation, and the changes can subsequently be either discarded or merged into the main tree of the distributed object storage system. Brief description of the drawings Some embodiments are described with reference to the following figures. Fig. 1 is a schematic diagram of an example system according to some implementations. Fig. 2 shows an example of data structures according to some implementations. Figs. 3A-3C are illustrations of example data structures according to some implementations. Fig. 4 is an illustration of an example process according to some implementations. Fig. 5 is an illustration of an example process according to some implementations. Fig. 6 is a diagram of a machine-readable medium that stores instructions according to some implementations. Fig. 7 is a schematic diagram of an example computer device according to some implementations. In the drawings, identical reference numbers denote similar, but not necessarily identical, elements. The illustrations are not necessarily to scale, and the size of some parts may be exaggerated to make the example shown clearer. Furthermore, the drawings contain examples and / or embodiments that correspond to the description; however, the description is not limited to the examples and / or embodiments shown in the drawings. Detailed description In this disclosure, the use of the terms "a" or "the" includes the plural forms unless the context clearly indicates otherwise. Similarly, the terms "comprises," "including," or "have," when used in this disclosure, specify the presence of the elements indicated but do not exclude the presence or addition of other elements. In some examples, a storage system can deduplicate data to reduce the storage space required. The storage system can perform a deduplication process, dividing a data stream into discrete data units or "chunks." Furthermore, the storage system can identify identifiers or "fingerprints" of incoming data units and determine which incoming data units are duplicates of previously stored data units. In the case of duplicate data units, the storage system can store references to the previous data units instead of storing the duplicate incoming data units themselves. As used here, the term "fingerprint" refers to a value derived by applying a function to the contents of the data unit (where "content" can be all or a subset of the data unit's contents). An example of a function that can be applied is a hash function, which produces a hash value based on the incoming data unit. Examples of hash functions include cryptographic hash functions such as those of Secure Hash Algorithm 2 (SHA-2), e.g., SHA-224, SHA-256, SHA-384, etc. Other examples may use other types of hash functions or other types of fingerprint functions. A "storage system" can comprise a storage device or an array of storage devices. A storage system can also include one or more storage controllers that manage access to the storage device(s). A "data unit" can refer to any portion of data that can be separately identified within the storage system. In some cases, a data unit can refer to a chunk, a collection of chunks, or some other part of data. In some examples, a storage system can store data units in persistent storage. Persistent storage can be implemented using one or more persistent (e.g., non-volatile) storage devices, such as disk-based storage devices (e.g., hard disk drives (HDDs)), solid-state devices (SSDs), such as flash memory devices, or a combination thereof. A "control unit" can refer to a hardware processing circuit that may include any or a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, a digital signal processor, or another hardware processing circuit. Alternatively, a "control unit" can refer to a combination of a hardware processing circuit and machine-readable instructions (software and / or firmware) that can be executed on the hardware processing circuit. In some examples, a deduplication storage system can use stored metadata to process and reconstruct the original data stream from the stored data units. This allows the deduplication process to avoid storing duplicate copies of repeated data units, thereby reducing the storage space required for the data stream. In some examples, the deduplication metadata can contain data recipes (also referred to here as "manifests") that specify the order in which particular data units were received (e.g., in a data stream). To retrieve the stored data (e.g., in response to a read or clone request), the deduplication system can use a manifest to determine the order of the received data units and thus reconstruct the original data stream.The manifest can contain a sequence of records, each representing a specific set of data units. The manifest records can contain one or more fields (referred to here as "pointer information") that identify indexes containing storage information for the data units. For example, the storage information might include one or more index fields specifying location information (such as containers, offsets, etc.) for the stored data units, compression and / or encryption characteristics of the stored data units, and so on. In some examples, the manifests and indexes can each be read in addressable sections of fixed size (e.g., 4KB sections). A "cloning" operation can refer to the process of creating a copy of a particular data stream stored in the deduplication storage system. For example, a cloning operation might involve loading a source manifest for a particular data stream into memory and identifying a set of indexes based on the source manifest. Each identified index is loaded into memory as a whole unit, decompressed, and deserialized. The index can then be updated to specify the reference numbers of the cloned data units (i.e., the number of instances in which the data unit appears in manifests) and written from memory to permanent storage. The new manifest (i.e., the cloned data) can be constructed from the components of the source manifest. However, in some examples, the cloning operation might only involve a relatively small portion of the index (e.g.,(ten out of ten thousand records in the index), while the index was being loaded into memory. Therefore, in such examples, loading the entire index into memory can consume more system processing time and bandwidth than loading only the changed portion into memory. In accordance with some implementations of this disclosure, a deduplication storage system can perform a cloning operation by loading journals into memory without loading the associated indexes. Each journal can store only information indicating changes to data stored in a corresponding index and can therefore be relatively smaller than the corresponding index. Furthermore, the journal can contain a data structure (referred to here as the "clone data structure") intended for recording metadata changes associated with cloning operations. The clone data structure can accumulate these changes until a triggering event (e.g., when the journal is full) and can be used to update the corresponding index during a single load into memory.Since each journal is smaller than its corresponding index, performing a cloning operation using journals can require relatively less processing time and bandwidth than using the corresponding indexes. Accordingly, the described cloning technique can significantly improve the performance of the deduplication storage system. Example 1: Storage system Figure 1 shows an example of a storage system 100 comprising a storage controller 110, a memory 115, and a persistent storage 140 according to some implementations. As shown, the persistent storage 140 can contain any number of manifests 150, indexes 160, and data containers 170. The persistent storage 140 can comprise one or more non-transient storage media such as hard disk drives (HDDs), solid-state drives (SSDs), optical disks, etc., or a combination thereof. The memory 115 can be implemented in the form of semiconductor memory, e.g., random access memory (RAM). In some implementations, the storage system 100 can perform deduplication of the stored data. For example, the storage controller 110 can divide a stream of input data into data units and store at least one copy of each data unit in a data container 170 (e.g., by appending the data units to the end of the container 170). In some examples, each data container 170 can be subdivided into parts (also referred to here as "entities"). In one or more implementations, the storage controller 110 can generate a fingerprint for each data unit. The fingerprint can, for example, contain a full or partial hash value based on the data unit. To determine whether an incoming data unit is a duplicate of a stored data unit, the storage controller 110 can compare the fingerprint generated for the incoming data unit with the fingerprints of the stored data units. If this comparison yields a match, the storage controller 110 can determine that a duplicate of the incoming data unit is already stored in the storage system 100. As shown in Fig. 1, the persistent storage 140 can store manifests 150, indexes 160, data containers 170, and journal groups 120. In some implementations, the storage controller 110 can generate a manifest 150 to record the order in which the data units were received. The manifest 150 can also contain a pointer or other information specifying the index 160 associated with each data unit. In some implementations, the associated index 160 can indicate the location where the data unit is stored. For example, the associated index 160 can contain information indicating that the data unit is stored at a specific offset in an entity, and that the entity is stored at a specific offset in a data container 170. In some implementations, the storage controller 110 can receive a read request to access the stored data and then access the manifest 150 to determine the order of the data units that comprised the original data. The storage controller 110 can then use the pointer data contained in the manifest 150 to identify the indices 160 associated with the data units. Furthermore, the storage controller 110 can use the information contained in the identified indices 160 to determine the storage locations of the data units (e.g., data container 170, entity, offsets, etc.) and can then read the data units from those determined storage locations. In some implementations, each index 160 can be associated with a journal 130. Journal 130 can contain information indicating changes to the data stored in index 160. For example, if a copy of index 160 in memory 115 is modified to reflect a metadata change, this change can also be recorded as an entry in the associated journal 130. In some implementations, multiple journals 130 can be grouped into a journal group 120, which is associated with a single file or object stored in the deduplication system. For example, the multiple journals can correspond to indexes that store metadata for a single file. In some implementations, the storage controller 110 can receive a request to perform a cloning operation of a source manifest 150. The cloning operation can involve creating a new manifest (also called a "clone manifest") that duplicates all or part of the source manifest 150 at a specific point in time. Therefore, the clone manifest can be used to create a copy of the sequence of data units at a particular point in time. For example, the clone manifest can be used to create a backup of the sequence of data units at a specific point in time. In one or more implementations, during a cloning operation of a manifest part, the memory system 100 loads the journals 130 associated with the manifest part into memory 115, but not the indexes 160 associated with the manifest part into memory 115. In some implementations, each journal 130 may contain a clone data structure (not shown in Fig. 1) intended for recording metadata changes related to the cloning operation for the manifest part. The clone data structure can accumulate these metadata changes and subsequently be used to update the corresponding index 160 to reflect the same metadata changes. The use of the journal 130 during the cloning operation is discussed below with reference to Fig. 2-7. 2. Exemplary data structures Figure 2 shows an example of data structures 200 used in deduplication according to some implementations. As shown, the data structures 200 can comprise a manifest record 210, a container index 220, a container 250, and an entity 260. In some examples, the container index 220 and the container 250 can generally correspond to the example implementations of index 160 and data container 170, respectively (shown in Figure 1). In some examples, the data structures 200 can be created and / or managed by the storage controller 110 (shown in Figure 1). Furthermore, the data structures 200 can be used by the storage controller 110 to retrieve stored deduplicated data. As shown in Fig. 2, the manifest record 210 can have multiple fields in some examples, including an offset field, a container index field, a length field, and a unit address field. In some implementations, the manifest record 210 can represent a range of data units in a run-length reference format. For example, to represent a range that starts at a first data unit and continues for N data units, the unit address field can specify an arrival number of a first data unit in the range (i.e., the numerical order in which the first data unit was added to the identified container index), and the length field can specify the number of data units in the range that follow the first data unit within the container index. In some implementations, each container index 220 can contain any number of data unit records 230 and entity records 240. Each record 230 can contain various metadata fields, such as a fingerprint (e.g., a hash of the data unit), a unit address, an entity ID, a unit offset (i.e., an offset of the data unit within the entity), a count value, and a unit length. Furthermore, each entity record 240 can contain various metadata fields, such as an entity ID, an entity offset (i.e., an offset of the entity within the container), a stored length (i.e., a length of the data unit within the entity), a decompressed length, a checksum value, and compression / encryption information (e.g., type of compression, type of encryption, etc.).In some embodiments, each container 250 can contain any number of units 260, and each entity 260 can contain any number of stored data units. In some implementations, each container index 220 can contain a version number 235. The version number 235 can indicate a generation or relative age of the metadata in the container index. For example, the version number 235 can be compared to the version number of an associated journal (not shown in Fig. 2). If the version number 235 is higher than the version number of the associated journal, it can be determined that the container index 220 contains newer metadata than the associated journal. 3A. Example data structures in a non-cloning operation Fig. 3A shows a representation of memory 115 during a non-cloning operation (e.g., a read operation). As shown, during the non-cloning operation, memory 115 can contain multiple journals 320 in a journal group 310 and also multiple indexes 330. For example, a particular journal 320 and its associated index 330 can be loaded into memory together in response to a non-cloning detection. In some examples, the journal group 310, the journals 320, and the indexes 330 can generally correspond to the example implementations of the journal group 120, the journals 130, and the indexes 160, respectively (shown in Fig. 1). In some implementations, each journal 320 can be associated with a corresponding index 330 and record changes to the metadata stored in the corresponding index 330. Furthermore, for each journal group 120, all corresponding indexes 330 can be associated with a single stored object (e.g., a document, a database table, a data file, etc.). For example, all corresponding indexes 330 can contain metadata for data units contained in a single file stored in a deduplication system (e.g., the storage system 100 shown in Fig. 1). In some implementations, each journal 320 can contain or be associated with version number 325. Similarly, each index 330 can contain or be associated with version number 335. In some implementations, during non-cloning, version number 325 can be compared to version number 335 to determine whether journal 320 or its associated index 330 contains the latest version of the metadata. For example, if version number 325 is higher than version number 335, it can be determined that the change data contained in journal 320 reflects a more recent state of the metadata than the metadata stored in index 330. If this is the case, index 330 can be updated to incorporate the changes recorded in journal 320.However, if version number 325 is lower than version number 335, it can be determined that the change data contained in journal 320 reflects a metadata state that is older than the metadata stored in index 330. In this case, journal 320 can be deleted without updating index 330. 3B-3C. Example data structures during a cloning process Figures 3B-3C show examples of data structures used during a cloning operation. As shown in Figure 3A, the memory 115 can contain multiple journals 320 in a journal group 310 during the cloning operation, but not the corresponding indexes 330 (shown in Figure 3A). Furthermore, each journal 320 can contain a clone data structure 327, which is intended for recording metadata changes during the cloning operation. In some implementations, performing a clone operation of a source manifest (or part of it) may involve identifying the container index 330 associated with the manifest. Journal group 310, which contains journal 320 corresponding to the identified container index 330, is loaded from permanent memory into memory 115, but the container index 330 itself is not loaded into memory 115. In some implementations, journal group 310 is loaded into memory as a single unit. Performing the clone operation involves creating a new cloned manifest that references the same data units referenced by the source manifest, and therefore results in increments to the reference numbers of the cloned data units. In some implementations, these increments may be recorded in the clone data structure 327 of journal 320 loaded into memory 115. Figure 3C shows an example implementation of Clone Data Structure 327. As shown, Clone Data Structure 327 can have multiple fields, including a manifest ID field, a unit address field, a length field, and a reference number field. Clone Data Structure 327 can contain multiple records or rows, each record corresponding to a specific range in a cloned manifest. In some examples, the unit address and length fields of each record can identify the corresponding range in a run-length reference format. For example, the unit address field can specify the arrival number of the first data unit in the range, and the length field can specify the number of data units in the range that follow the first data unit. In some implementations, the reference number field can store or otherwise indicate a change in the reference count that occurred during a cloning operation.For example, if the cloning operation duplicates a particular range, the reference number field can indicate that the reference count for that range has been incremented by one. In some implementations, the reference number field can store a numeric value representing the total change in the reference count that has occurred during one or more cloning operations. In some implementations, in response to a triggering event, the data stored in the clone data structure 327 of journal 320 can be used to update the associated container index 330 (shown in Fig. 3A). This update can be described as "folding" the clone data structure 327 into the container index 330. For example, when journal 320 is full (e.g., when the data stored in journal 320 exceeds a maximum threshold), the container index 330 can be loaded into memory 115, and the reference counts in the container index 330 can be incremented to include the values ​​in the reference number field of the corresponding records in the clone data structure 327. Once each record of the clone data structure 327 has been folded into the container index 330, the entire clone data structure 327 is cleared of its stored data (i.e., all rows are deleted). When the clone data structure 327 is folded into the container index 330, in some implementations the container index 330 can also be updated based on changes in journal 320 that are not included in the clone data structure 327 (i.e., metadata changes not associated with cloning operations). In some implementations, the metadata changes not associated with cloning operations (also called "non-clone updates") can be performed in the manner described above with reference to Figure 3A. For example, a non-clone update might involve comparing the version number 325 of journal 320 with the version number 335 of container index 330 and performing the non-clone update if version number 325 is greater than version number 335. In some situations, an error event (e.g., a power outage) can interrupt the folding of container index 330 into clone data structure 327 before completion. After such interruptions, it can be difficult or impossible to determine which records of clone data structure 327 have already been folded into container index 330, and therefore the stored data may be lost or corrupted. In some implementations, the Manifest ID field can be used to recover the stored data after an error event. For example, if each record of clone data structure 327 is folded into container index 330, the Manifest ID field in that record of clone data structure 327 can be populated with an identifier from the clone manifest (i.e., the manifest generated by the cloning operation). Note that the Manifest ID field for a record remains empty until the folding of that record is complete.After an error event, the Manifest ID field of each record can be analyzed. The presence of the clone manifest's identifier in the Manifest ID field may indicate that the record in question has already been folded into the container index and therefore should not be folded again. Furthermore, in some examples, any record that identifies the clone manifest in the Manifest ID field can be deleted from journal 320. 4. Example Process Figure 4 shows an example of Process 400 according to some implementations. In some examples, Process 400 can be performed by the memory controller 110 (shown in Figure 1) during cloning. Process 400 can be implemented in hardware or a combination of hardware and programming (e.g., machine-readable instructions that can be executed by one or more processors). The machine-readable instructions can be stored on a non-transient, computer-readable medium, such as an optical, semiconductor, or magnetic storage device. The machine-readable instructions can be executed by a single processor, multiple processors, a single processing machine, multiple processing machines, etc. For illustration, details of Processes 400-404 are described below with reference to Figures 1-3C, which show examples of some implementations.However, other implementations are also possible. Block 410 can include the detection of a manifest area cloning operation. For example, as shown in Fig. 1-3C, the memory controller 110 can receive a request to clone a source manifest 150 and, in response, initiate a cloning operation to create a clone manifest that duplicates a portion of the source manifest 150 at a given time. Block 420 can involve identifying a container index associated with the manifest portion. Block 430 can involve loading a journal group into memory, where the journal group contains a journal associated with the container index. For example, as shown in Fig. 1-3C, memory controller 110 can identify container index 160, which is associated with source manifest 150, and can therefore identify journal 320, which is associated with the identified container index 160. Memory controller 110 can then load journal group 310, which contains the identified journal 320, from persistent memory 140 into memory 115. However, in some implementations, memory controller 110 cannot load the identified container index 160 into memory 115. Block 440 can include storing a reference to metadata changes in a clone data structure of the journal. For example, with reference to Fig. 1-3C, the memory controller 110 can create a new cloned manifest that references the same data units referenced by the source manifest and determine the changes to the reference counts of the cloned data units. These changes can be recorded in the clone data structure 327 of the journal 320 loaded into memory 115 (e.g., in a reference number field of a record associated with a cloned range). Decision block 450 can include determining whether the journal is full. If it is determined that the journal is not full ("No" in block 450), process 400 can be terminated. However, if it is determined that the journal is full ("Yes" in block 450), process 400 can continue to block 460, which may involve loading the container index into memory. Block 470 may involve updating the container index based on the clone data structure of the journal. Block 480 may involve deleting the clone data structure of the journal. For example, as shown in Fig. 1-3C, the memory controller 110 can continue the cloning operation until it is complete. However, if it is determined that the journal 320 loaded into memory 115 has reached a maximum threshold of stored data, the associated container index 160 can be loaded from persistent memory 140 into memory 115.The data stored in journal 320 can then be copied to container index 160 in memory 115, and subsequently the data stored in journal 320 can be deleted. After block 480, process 400 can be completed. In some examples, blocks 420-440 of process 400 can be repeated multiple times during a single cloning operation. For example, blocks 420-440 of process 400 can be executed for a sequence of container indices associated with the manifest area to be cloned. 5. Example Process Figure 5 shows an example Process 500 in accordance with some implementations. In some examples, Process 500 can be carried out using the memory controller 110 (shown in Figure 1). Process 500 can be implemented in hardware or a combination of hardware and programming (e.g., machine-readable instructions that can be executed by one or more processors). The machine-readable instructions can be stored on a non-transitory, computer-readable medium, such as an optical, semiconductor, or magnetic storage device. The machine-readable instructions can be executed by a single processor, multiple processors, a single processing machine, multiple processing machines, etc. For illustration, details of Process 500 are described below with reference to Figures 1-3C, which show examples of some implementations.However, other implementations are also possible. Block 510 can include the detection of a manifest-area cloning operation by a memory controller of a deduplication storage system. For example, see Fig. 1-3C, the memory controller 110 can receive a request to clone a source manifest 150 and, in response, initiate a cloning operation to create a clone manifest that duplicates a portion of the source manifest 150 at a given time. Block 520 can include the loading of a journal from permanent memory into memory by the memory controller in response to the detected cloning operation, where the journal is intended to store changes to a container index associated with the manifest area, and where the container index is not loaded into memory in response to the detected cloning operation. For example, see Fig. 1-3C, memory controller 110 can identify container index 160, which is associated with source manifest 150, and can identify journal 320, which is associated with this container index 160. Memory controller 110 can then identify journal group 310, which contains the identified journal 320, and can then load the identified journal group 310 as a whole from permanent memory 140 into memory 115.However, it is possible that the memory controller 110 will not load the container index 160 into memory 115 in response to the cloning operation. Block 530 can include updating the journal in memory by the storage controller to include a display of changes to the container index metadata, where the changes are associated with the detected cloning operation. For example, as shown in Fig. 1-3C, the storage controller 110 can, for instance, create a new cloned manifest that references the same data units referenced by the source manifest and determine the changes to the reference numbers of the cloned data units. These changes can be recorded in the clone data structure 327 of the journal 320 loaded into memory 115 (e.g., in a reference number field of a record associated with a cloned area). 6. Example of a machine-readable medium Fig. 6 shows a machine-readable medium 600 on which instructions 610-630 are stored in accordance with some implementations. The instructions 610-630 can be executed by a single processor, multiple processors, a single processing machine, multiple processing machines, etc. The machine-readable medium 600 can be a non-transient storage medium, such as an optical, semiconductor, or magnetic storage medium. Instruction 610 can be executed to detect a cloning operation of a manifest area. Instruction 620 can be executed to load a journal from persistent storage into memory in response to the detected cloning operation. The journal should store changes to a container index associated with the manifest area, and the container index itself should not be loaded into memory in response to the detected cloning operation. Instruction 630 can be executed to update the journal in memory to include a reference to changes in the metadata of the container index, where the changes are associated with the detected cloning operation. 7. Example of a computer device Fig. 7 shows a schematic diagram of an example computer device 700. In some examples, the computer device 700 can generally correspond to part or all of the storage system 100 (shown in Fig. 1). As shown, the computer device 700 can include a hardware processor 702 and a machine-readable memory 705 containing instructions 710-730. The machine-readable memory 705 can be a non-transferable medium. The instructions 710-730 can be executed by the hardware processor 702 or by a processing unit contained within the hardware processor 702. Instruction 710 can be executed to detect a cloning operation of a manifest area. Instruction 720 can be executed to load a journal from persistent storage into memory in response to the detected cloning operation. The journal should store changes to a container index associated with the manifest area, and the container index itself should not be loaded into memory in response to the detected cloning operation. Instruction 730 can be executed to update the journal in memory to include a reference to changes in the container index's metadata, where the changes are associated with the detected cloning operation. In accordance with the implementations described here, a deduplication storage system can perform a cloning operation by loading journals into memory but not the associated indexes. Each journal can contain a clone data structure, which is used to record or otherwise display metadata changes associated with cloning operations. The clone data structure can accumulate these changes until a triggering event and can be used to update the corresponding index during a single load into memory. In some examples, performing the cloning operation using journals can consume relatively less processing time and bandwidth than if the associated indexes were loaded into memory. Accordingly, the disclosed cloning technique can significantly improve the performance of the deduplication storage system. It should be noted that, although Figures 1-7 show various examples, implementations in this respect are not limited. For example, the storage system 100, as shown in Figure 1, may include additional devices and / or components, fewer components, different components, different arrangements, etc. Another example is that the functionality of the storage controller 110 described above may be contained in a different machine or software of the storage system 100. Other combinations and / or variations are also possible. Data and instructions are stored in appropriate storage devices, which are implemented as one or more computer-readable or machine-readable storage media. Storage media include various forms of non-transient memory, including semiconductor memory such as dynamic or static random-access memory (DRAM or SRAM), erasable and programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), and flash memory; magnetic disks such as hard disks, floppy disks, and removable disks; other magnetic media including tapes; optical media such as compact discs (CDs) or digital video discs (DVDs); or other types of storage devices. It should be noted that the instructions discussed above can be provided on a single computer-readable or machine-readable storage medium, or alternatively, on multiple computer-readable or machine-readable storage media distributed throughout a large system, possibly with multiple nodes. Such computer-readable or machine-readable storage medium or media are considered part of an article (or a manufactured item). An article or manufactured item can refer to any single manufactured component or multiple components. The storage medium or media can be located either in the machine on which the machine-readable instructions are executed or at a remote location from which machine-readable instructions can be downloaded for execution over a network. The foregoing description includes numerous details to provide an understanding of the subject matter disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and deviations from the details described above. It is intended that the accompanying claims cover such modifications and variations.

Claims

A computer-implemented method comprising: detecting (510) a cloning operation of a manifest area by a memory controller (110) of a deduplication storage system; loading a journal (130, 320) from persistent storage (140) into memory in response to the detected cloning operation, wherein the journal (130, 320) serves to store changes to a container index (220) associated with the manifest area, and wherein the container index (220) is not loaded into memory in response to the detected cloning operation; and updating the journal (220) in memory to include a display of changes to metadata of the container index (220) that is not loaded into memory, wherein the changes to the metadata are associated with the detected cloning operation. The computer-implemented method according to claim 1, comprising: detecting a non-cloning operation of a second manifest area; loading a second journal (130, 320) and a second container index (220) together into memory in response to a detection of the non-cloning operation; and updating the second journal (130, 320) in memory to accommodate second changes associated with the second container index (220), wherein the second changes are associated with the non-cloning operation. The computer-implemented method according to claim 1, wherein loading the journal (130, 320) into memory comprises: identifying a container index (220) associated with the manifest area; identifying the journal (130, 320) based on the identified container index (220); identifying a journal group (310) containing the identified journal (130, 320); and loading the entire identified journal group (310) into memory, wherein the identified journal group (310) is a data structure for containing a plurality of journals (130, 320), and wherein each of the plurality of journals (130, 320) in the identified journal group (310) is designated to record changes associated with different container indexes (220). The computer-implemented method according to claim 1, wherein updating the journal (130, 320) to include the display of changes to the metadata comprises: adding a record to a clone data structure (327) contained in the journal (130, 320), wherein the clone data structure (327) is exclusively for recording indications of metadata changes during cloning operations. The computer-implemented method according to claim 4, comprising: updating the data set to include a unit address and a length value, wherein the unit address and a length value identify the manifest area in a run-length reference format. The computer-implemented method according to claim 4, comprising: updating the data set to include the specification of a reference number for the manifest area. The computer-implemented method according to claim 4, comprising: in response to the finding that the journal (130, 320) in memory has reached a maximum threshold of stored data: loading (460) the container index (220) from permanent storage into memory; updating (470) the loaded container index (220) in memory based on the records of the clone data structure (327); and after updating the loaded container index (220) in memory based on the records of the clone data structure (327), deleting (480) the records of the clone data structure (327). The computer-implemented method according to claim 7, comprising: for each record of the clone data structure (327): updating the index (220) of the loaded container in memory based on the record; and after updating the loaded container index (220) in memory based on the record, updating the record to include an identifier of a manifest generated by the cloning process. A non-transitory machine-readable medium that stores instructions which, when executed, cause a processor to: detect a cloning operation of a manifest area; load a journal (130, 320) from persistent storage (140) into memory in response to the detected cloning operation, wherein the journal (130, 320) serves to record changes to a container index (220) associated with the manifest area, and wherein the container index (220) is not loaded into memory in response to the detected cloning operation; and update the journal (130, 320) in memory to include a display of changes to metadata of the container index (220) that is not loaded into memory, wherein the changes to the metadata are associated with the detected cloning operation. The non-transitory machine-readable medium according to claim 9, which contains instructions which, when executed, cause the processor to: identify a container index (220) associated with the manifest area; identify the journal (130, 320) based on the identified container index (220); identify a journal group (310) containing the identified journals (130, 320); and load the entire identified journal group (310) into memory, wherein the journal group (310) is a data structure for containing a plurality of journals (130, 320), and wherein each of the plurality of journals (130, 320) in the journal group (310) is designated to record changes associated with different container indexes (220). The non-transitory machine-readable medium according to claim 9, which contains instructions which, when executed, cause the processor to: add a record to a clone data structure (327) contained in the journal (130, 320), wherein the clone data structure (327) is exclusively for recording indications of metadata changes during cloning operations. The non-transitory machine-readable medium according to claim 11, which contains instructions which, when executed, cause the processor to: update the data set to include a unit address and a length value, wherein the unit address and a length value identify the manifest area in a run-length reference format. The non-transitory machine-readable medium according to claim 11, which contains instructions which, when executed, cause the processor to: detect a non-clone operation of a second manifest area; in response to the detected non-clone operation, load a second journal (130, 320) and a second container index (220) together from permanent storage into memory; and update the second journal (130, 320) in memory to store second changes associated with the second container index (220), wherein the second changes are associated with the detected non-clone operation. The non-transitory machine-readable medium according to claim 11, which contains instructions which, when executed, cause the processor to: in response to the detection that the journal (130, 320) in memory has reached a maximum threshold of stored data: load the container index (220) from permanent storage into memory; update the loaded container index (220) in memory based on the records of the clone data structure (327); and, after updating the loaded container index (220) in memory based on the records of the clone data structure (327), delete the records of the clone data structure (327). A memory system comprising: a processor with a plurality of processing machines; and a machine-readable memory that stores instructions, wherein the instructions can be executed by the processor to: detect a cloning operation of a manifest area; load a journal (130, 320) from a persistent memory (140) into memory in response to the detected cloning operation, wherein the journal (130, 320) serves to record changes to a container index (220) associated with the manifest area, and wherein the container index (220) is not loaded into memory in response to the detected cloning operation; and update the journal (130, 320) in memory to include a display of changes to metadata of the container index (220) that is not loaded into memory, wherein the changes to the metadata are associated with the detected cloning operation. The memory system according to claim 15, including instructions that can be executed by the processor to: identify a container index (220) associated with the manifest area; identify the journal (130, 320) based on the identified container index (220); identify a journal group (310) containing the identified journal (130, 320); and load the entire identified journal group (310) into memory, wherein the journal group (310) is a data structure for containing a plurality of journals (130, 320), and wherein each of the plurality of journals (130, 320) in the journal group (310) is designated to record changes associated with different container indexes (220). The storage system according to claim 15, which contains instructions that can be executed by the processor, for: adding a record to a clone data structure (327) contained in the journal (130, 320), wherein the clone data structure (327) is exclusively for recording indications of metadata changes during cloning operations. The storage system according to claim 17, including instructions that can be executed by the processor to: update the record to include a unit address and a length value, wherein the unit address and a length value identify the manifest area in a run-length reference format. The storage system according to claim 17, including instructions that can be executed by the processor to: update the data record to include the specification of a reference number for the manifest area. The storage system according to claim 15, which contains instructions that can be executed by the processor to: in response to the detection that the journal (130, 320) in memory has reached a maximum threshold of stored data: load the container index from permanent storage (140) into memory; update the loaded container index (220) based on the records of the clone data structure (327) in memory; and after updating the loaded container index (220) in memory based on the records of the clone data structure (327), delete the records of the clone data structure (327).

Images