Snapshots for point-in-time copy

Abstract

Systems and methods for providing point-in-time functionality. Point-in-time protection is achieved by combining metadata streams with snapshots of a storage system, such as a VSAN. This allows for the generation of snapshots after the fact, enabling point-in-time functionality to be provided.

Description

TECHNICAL FIELD

[0001] Embodiments of the present invention relate to systems and methods for performing computing operations that include data protection operations. More specifically, embodiments of the present invention relate to systems and methods for any point in time replication using snapshots. More specifically, embodiments of the present invention relate to providing any point in time replication in a VSAN (virtual storage area network) system using UPiT (universal point in time) snapshots. BACKGROUND

[0002] It is common to create backups of data so that in the event of a failure of the primary or production data, the production data can be restored from the backup. Today, backups provide the ability to perform restore operations so that data can be restored at different points in time.

[0003] Unfortunately, this ability to provide any point in time functionality is not always available for all systems. This ability depends on how the data is stored. For example, a system can store data in blobs or in a manner where the data only represents the most recent point in time. One consequence of storing data in this manner is that snapshots can only be created for specific points in time. In other words, snapshots in these systems cannot be created after the fact. Thus, any point in time capability is not available. BRIEF DESCRIPTION OF DRAWINGS

[0004] In order to describe the manner in which at least one embodiment of the present invention can be obtained, a more particular description will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope, the embodiments of the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[0005] Figure 1 An example of a system that generates a metadata stream that allows for any point in time functionality for production data is shown;

[0006] Figure 2 A metadata stream and the relationship of the metadata stream to data used for any point in time functionality is shown;

[0007] Figure 3 An example of the relationship between an identifier and a data offset in a production volume is shown;

[0008] Figure 4 An example of rehydrating data from a data bucket to a replica production volume is shown.

[0009] Figure 5An example of a node in a storage environment, such as a VSAN environment, is shown, and the generation of a metadata stream at the node is shown;

[0010] Figure 6 An example of generating and storing a metadata stream in a storage environment is shown; and

[0011] Figure 7 An example of a method for providing arbitrary point-in-time functionality to a storage environment that uses snapshots is shown. DETAILED DESCRIPTION

[0012] Embodiments of the present invention relate generally to data protection and to systems, methods, and apparatus for performing data protection operations. Examples of data protection operations include, but are not limited to, backup operations, restore operations, replication operations, snapshot operations, arbitrary point-in-time related operations (including generating arbitrary point-in-time snapshots), and the like. More specifically, embodiments of the present invention relate to systems, hardware, software, computer readable media, and methods for arbitrary point-in-time restore operations and the ability to generate snapshots post-hoc. More specifically still, embodiments of the present invention relate to providing arbitrary point-in-time functionality to systems that do not have arbitrary point-in-time functionality, even if specific point-in-time functionality is available.

[0013] Embodiments of the present invention relate to systems and methods for providing efficient storage and data protection schemes for systems, including VSAN systems. Embodiments of the present invention disclose an efficient arbitrary point-in-time data structure that enables arbitrary point-in-time functionality for systems in which the data stored represents only the most recent point-in-time. More specifically, some of these systems only support specific point-in-time functionality because the data is stored in a blob that represents the most recent point-in-time. In these systems, it is not possible to create a snapshot post-hoc, and embodiments of the present invention enable these systems to generate snapshots post-hoc and to implement arbitrary point-in-time functionality.

[0014] By way of example only, some implementations of VSAN are systems in which the data stored represents only the most recent point-in-time. In these systems, snapshots such as UPiT (Universal Point-in-Time) snapshots will be taken. UPiT snapshots generally represent a specific point-in-time, but do not necessarily provide arbitrary point-in-time functionality.

[0015] Embodiments of the present invention relate to systems and methods that enable arbitrary point-in-time functionality for these types of systems. More specifically, embodiments of the present invention enable metadata volumes or representations of metadata volumes to be associated with or incorporated into UPiT snapshots. Metadata volumes allow UPiT to be generated for arbitrary points-in-time, thereby enabling arbitrary point-in-time functionality for these types of systems. More specifically, embodiments of the present invention store metadata streams as objects in a storage system. Metadata streams contain a record of all inputs and outputs to the storage system. By incorporating metadata streams (or relevant portions of metadata streams) into existing snapshots, arbitrary point-in-time functionality can be achieved from a particular point-in-time snapshot.

[0016] Discussed below is arbitrary point-in-time functionality using metadata volumes adapted in accordance with embodiments of the present invention. Subsequently, a discussion of related VSAN functionality is presented.

[0017] Arbitrary point-in-time functionality using metadata streams

[0018] Figure 1 An example of a computing environment in which data protection operations are performed is shown. Figure 1 A client 102 associated with production data 106 (e.g., production volumes or storage, such as a VSAN) is shown. The client 102 can be a virtual machine, a computing device such as a computer, a laptop, a tablet, a smartphone, a server computer, and the like. The production data 106 can reside on a storage device (storage array or other storage arrangement). The production data 106 can be local to the client 102 (e.g., in the same network) or remote from the client 102. The production data 106 can also be cloud-based storage.

[0019] The client 102 interacts with the production data 106 and can write data or read data or perform other actions. In the context of protecting data or when data is replicated, an input / output (IO) can refer to an action or command that causes the production data 106 to change. Whenever data is written (deleted, moved, modified, copied, etc.) to the production data 106, the replication engine 104 replicates the data or the change. More specifically, the replication engine 104 can replicate the action. The replication engine 104 can be incorporated into the production data 106, can be a server or other appliance, or a software layer configured to detect certain commands including writes and replicate data accordingly.

[0020] In this example, replication engine 104 therefore replicates or writes data to storage device 108, which can be a data center, cloud storage, etc. The replication engine can also generate a metadata stream and write the metadata stream as a metadata stream object 110 to the metadata stream storage device. Stream object 110 can be in the same location (e.g., the same storage system, the same data center) or bucket as storage device 108. Stream storage device 110 and storage device 108 can also be separate storage devices.

[0021] As discussed in more detail in this article Figure 1 The system shown can be adapted to different storage environments. In one example, storage system 112 can be part of a storage system (such as VSAN). In this case, all input / output goes through replication engine 104 (which can be considered the owner). Replication engine 104 does not partition the data, but can simply manipulate metadata so that metadata streams are stored as stream objects 110 and data objects are stored as data objects 108.

[0022] Figure 2 Further shown Figure 1 The replication shown here, or more precisely, the generation of metadata streams, is as follows. In one example, replication engine 202 (an example of replication engine 104) can detect IO 210 being written to production volume 208, which can store production data 106. Replication engine 202 can determine that IO 210 includes data A and is being written to a location in data bucket 204 identified by offset X, which is an example of a blob or data object stored in storage system 214. Offset X represents the location of data A in data bucket 204. Offset X can be specified using other terms, which may depend on the configuration of storage system 214 and / or the configuration of the data in data bucket 204.

[0023] The replication engine then copies IO 210 to data bucket 204. In one example, data bucket 204 is a key-value store. Therefore, replication engine 202 can provide data A and a key (such as identifier 10) to data bucket 204, and IO 210 will be stored in data bucket 204. To retrieve data A, identifier 10 is provided to the data bucket. Using identifier 10, data bucket 204 can retrieve and return data A.

[0024] As the IO 210 or data A is copied to the data bucket 204, the copy engine 202 can add metadata associated with the IO 210 to a metadata stream 206 stored in the storage system 214 (an entry to the metadata stream can be added for each IO). In this example, the metadata stream 206 is stored as an object in the storage system 214. The metadata stream 206 generally includes metadata associated with each IO, such as the IO 210. In this example, the entry 212 for the IO 210 in the metadata stream 206 includes an offset X and an identifier 10. The offset X included in the metadata stream 206 refers to an offset or other positioning in the data bucket 204.

[0025] In one example, point-in-time recovery is generally tied to a specific IO, in part because the metadata stream 206 is related to the IOs that occur at the data bucket 204. By identifying a time or an IO, embodiments of the invention allow the data bucket 204, or more precisely, the blob or data object, to be restored to this point-in-time or this specific IO. This enables an arbitrary point-in-time recovery function.

[0026] In one example, the data associated with each IO is stored as an object in the data bucket 204. The data can be stored in chunks. For example, the IO 210 can be divided into 8 KB chunks. Embodiments of the invention contemplate variable size blocks. However, fixed size objects require less processing overhead, including inherent processing for IO overlap. The actual size of the chunk or object is generally a small number (e.g., 4 KB, 8 KB). While larger object sizes can require fewer uploads, larger size objects can also require completion of the object or chunk if only a portion of the object or chunk is written. Additionally, tracking the area written in a larger size chunk or object also increases complexity.

[0027] Figure 2 It is shown that during storage, data A can be stored as object A. More precisely, data A can actually be chunked into chunks Al, A2, etc. Thus, objects Al, A2, etc. will be stored in the data bucket 204 and each chunk will have its own identifier. The metadata stream 206 stores the relationship between the identifiers (allowing identification and retrieval of the corresponding objects in the data bucket 204) and the offsets in the data bucket 204, which is an example of a production volume. During restoration, the offsets and identifiers in the metadata stream allow the objects to be retrieved and written at the appropriate offsets in the restored production volume.

[0028] Because the data can be chunked before being stored in the data bucket, the ability to restore objects based on offsets effectively restores the data after all chunks are restored.

[0029] Figure 3 An example showing the relationship between the metadata stream and the production volume. Figure 3 A metadata stream 304 associated with the production volume 302 is shown. In one example, the metadata stream 304 is a sequence of IO metadata or a sequence of entries, where each entry corresponds to an IO. The entries in the metadata stream 304 are related to the data stored in the production volume 302. Each entry stores an offset in the production volume and an identifier. The entries in the metadata stream are related to the data after the data is chunked. Thus, offset 0 and identifier 10 correspond to data A (or object A) in the production volume 302. Thus, data A is located at an offset of 0 in the production volume 302. Similarly, objects B, C, D, E, and F are located at offsets 1, 2, 3, 4, and 5, respectively, and have identifiers 20, 30, 40, 50, and 60. These identifiers can be any data that represents the corresponding data.

[0030] Production data rehydration

[0031] The production volume (or production data) can be rehydrated using the objects stored in the storage (or data buckets) and the metadata stream stored in the stream object. Typically, the production volume is rehydrated or restored by creating a blank metadata volume. Once the metadata volume is created, the metadata volume is rolled to the required point in time using the metadata stream by writing the identifiers to the metadata volume at the offsets in the metadata volume. This is accomplished by moving forward from the tail of the metadata stream and updating the metadata volume according to each write encountered.

[0032] Each entry in the metadata volume points to an IO in the stored object. Thus, in one example, the metadata volume includes entries that each point to an object in the object storage (e.g., data buckets). In one example, only the first encountered instance of each IO is updated in the metadata volume. More specifically, as previously described, the metadata volume is updated with each encountered IO. This allows for avoiding copying the actual IO data into the restored volume in the case where this particular offset will be overwritten before reaching the required point in time. In one example, the result is that only one IO is copied for each offset, looking from the required point in time to the tail of the metadata stream.

[0033] Once the appropriate point in time is reached in the metadata stream, and all necessary entries in the metadata stream are written to the metadata volume, the metadata volume is organized. At this point, the actual data has not been moved or copied. However, embodiments of the present invention contemplate copying the data while the metadata volume is being organized.

[0034] Once the metadata volume is prepared, the metadata volume and data buckets can be used to rehydrate production volumes. The rehydrated production volumes are copies of the original production volumes (e.g., virtual machines or disks of virtual machines) at the requested point in time. In one example, actual production volumes are only generated if the resulting replica volumes are accessed as normal volumes.

[0035] Figure 4 An example of rehydrating a production volume or a portion of a VSAN is shown. Figure 4 A process of rehydrating a production volume 406 from a metadata stream 402 and data buckets 410 is shown, which are examples of the metadata stream and data buckets described earlier.

[0036] In Figure 4 In one example, starting with a blank metadata volume, the metadata volume 404 is first generated. In this example, the tail end 412 of the metadata stream 402 is read. The tail end 412 corresponds to an entry in the metadata stream 402. In one example, the entry of the tail end is the oldest entry in the metadata stream 402. In this example, new entries are pushed to the head of the metadata stream. However, the metadata stream can be arranged in another way such that the oldest entry is at the head of the metadata stream. By reading the metadata stream 402 starting from the oldest entry and then proceeding backwards in time, the metadata volume 404 can be populated with object or data identifiers at offsets corresponding to offsets in the production volume.

[0037] In one example, an initialized production volume can be available. This production volume can correspond to a point in time just prior to the oldest entry in the metadata stream. Using the metadata volume that has been loaded with information from the metadata stream, the initial production volume can be rolled forward to the selected point in time. In one example, the production volume can be generated directly from the metadata stream.

[0038] In this example, identifier 10 is copied to a location corresponding to offset 0. When it comes time to prepare the production volume 404, the object is retrieved from the data bucket 410 using the identifier stored at offset 0 in the metadata volume 404, and then the object retrieved from the data bucket 410 is written into the production volume 406 (which can be a replica volume) at a location corresponding to offset 0.

[0039] More specifically, the metadata stream 402 is read until the point-in-time 414 is reached, and if necessary, each entry is written to the metadata volume. The point-in-time 414 is the point-in-time at which restoration is desired. In this example, there are six entries to be written to the metadata volume 404. Thus, the identifiers 10, 20, 30, 40, 50, and 60 are written to the metadata volume 404 at respective locations or offsets stored therein. This results in the metadata volume 404 including the identifiers 10, 20, 30, 40, 50, and 60 written at offsets 0, 1, 2, 3, 4, and 5, respectively.

[0040] At this stage, no data is copied from the data bucket 410 to the production volume 406. However, it is possible to restore or rehydrate the production volume while the metadata volume 406 is being generated.

[0041] Once the metadata volume 404 for the point-in-time 414 is generated, the production volume 406 can be populated with data from the data bucket 410. Because the metadata volume 404 stores the identifiers or keys, these can be used to access the data bucket 410 to retrieve the corresponding objects, which are written at the same or corresponding offsets in the production volume 406. Thus, the identifier 10 in the metadata volume 404 is used to retrieve object A. Object A is then written to offset 0 in the production volume, which corresponds to the identifier 10, which was written to offset 0 in the metadata volume. Once this process is complete, the production volume 406 is ready for use and can be installed, if necessary. The metadata volume 404 can be retained for a period of time or deleted.

[0042] VSAN Design

[0043] In one example, a VSAN is a software-defined storage product that pools storage devices to create a data store. A VSAN system has a concept called a DOM (Distributed Object Manager). A DOM can operate on a specific ESX node and manages a VSAN object. There is only one DOM per object. All IO (input / output operations) to and from the object go through the DOM. Thus, the DOM can maintain consistency of the object because IO through the DOM can be ordered in a consistent manner - any ordering of IO decided by the DOM is a "correct" IO order.

[0044] Typically, the ESX (example of a hypervisor) running a VM (virtual machine) is the DOM for that VM, but this can change when vMotion is complete. In this case, the VM moves to another ESX, but the DOM does not move. After a period of time, the VSAN will move the DOM to the running ESX because this is the most efficient in terms of IO patterns.

[0045] UPiT snapshots are an example of a snapshot that can be used in a VSAN. UPiT snapshots can be the basis for many data protection operations, including local snapshot protection, backup, replication, and archive. UPiT snapshots are general snapshots and can be integrated with many storage systems or file systems that have one place for IO to pass through (e.g., a DOM).

[0046] In one example, UPiT snapshots are implemented as a B+ tree. In a B+ tree, data is stored only in leaf nodes. Because data is stored only in leaf nodes (or pointed to by leaf nodes), the internal nodes of a B+ tree store only pointers. Each page or node can contain multiple pointers. Thus, the internal representation of a B+ tree is compact and efficient. However, in a b tree, items that are accessed once they are encountered in the tree, need to have dummy nodes in a b+ tree in order to have their data placed only in the leaves. Thus, a b+ tree can have extra nodes and all access operations are log(n) instead of just worst case as with a traditional b tree.

[0047] UPiT can efficiently expose user volumes in a system such as a VSAN. However, UPiT can expose a specific point in time.

[0048] Embodiments of the invention combine volume representation or metadata streams with UPiT snapshots (or other snapshot types) to enable arbitrary point in time functionality. Typically, as discussed previously, IOs into the DOM are stored in the metadata stream as separate objects. Thus, efficient arbitrary point in time representation on a VSAN is thereby obtained and allows for the generation of snapshots such as UPiT snapshots after the fact.

[0049] UPiT snapshots can be used when exposing PiT to users, for example, in a VSAN. Metadata streams allow for the creation of any PiT after the fact because the metadata stream stores the IO history of objects of or at least associated with the DOM. By combining metadata streams with UPiT snapshots, arbitrary point in time representation of objects or volumes managed by the DOM or an owner can be exposed through UPiT snapshots. This allows for the generation of snapshots for arbitrary points in time and allows for the generation of the same type of snapshot (e.g., UPiT). This effectively extends arbitrary point in time functionality to UPiT snapshots and related systems.

[0050] Figure 5 An example of a portion of a VSAN is shown that is configured to provide at least arbitrary point in time functionality for that portion of the VSAN. Figure 5 A node 504 of a VSAN is shown. The node 504 is associated with storage and stores one or more objects 508. An owner 506 of the node 504 owns the storage and thus the objects 508.

[0051] All IO 502 to the node 504 goes through the owner 506. Because all IO 502 goes through the owner 502, the metadata stream can be efficiently maintained. The metadata of the IO is stored on the VSAN as a separate object (metadata object 510). In effect, the owner 506 is able to split the IO transaction. However, in this case, the data is not necessarily replicated as discussed previously. Rather, the metadata is manipulated or extracted from the IO and stored in the metadata object 510.

[0052] By adding the metadata stream to the node 504, an efficient arbitrary PiT representation of the VSAN is obtained or can be generated. Moreover, this has little impact on the performance and implementation of the VSAN.

[0053] Figure 6 An example of a method for storing a metadata stream is shown. IO is received 602 by an owner, such as a DOM. The IO is then processed to obtain 604 metadata associated with the IO. The metadata is then stored 606 as an object. More specifically, in one example, the metadata is stored as a separate object. Thus, this allows the metadata volume to be efficiently stored in the VSAN as a metadata object and associated with the owner 506 and / or associated with the portion of the VSAN associated with the node 504 and managed by the owner 506.

[0054] Thus, the metadata object 510 stores pointers to the data in the object 508. The metadata object 510 can be configured as described previously, such that each entry includes an offset and an identifier. An arbitrary point-in-time object can be recovered by generating the metadata volume and then generating a production volume or production object corresponding to a selected point-in-time.

[0055] More specifically, the metadata object 510, which is a representation of the metadata volume, effectively contains the history of IO to the node 504 or object 508. Thus, the metadata object 510 can be used to create a point-in-time representation as described previously that cannot be achieved using UPiT snapshots alone.

[0056] Because the structure of a b+ tree is well known, a detailed discussion is not provided here. However, a b+ (or b plus) tree generally includes a certain number of levels. Typically, all the leaves are on the same level. Also, data is only stored in the leaf nodes of the b+ tree. Other nodes typically contain keys and other metadata. Data is accessed by traversing the b+ tree.

[0057] By storing the metadata stream in a separate object, and because UPiT is implemented as a tree, such as a b+ tree, the metadata object or stream can be converted to any selected PiT representation.

[0058] More specifically, the metadata stream or object stored by the owner contains pointers to data that can be used to create a PiT. As previously mentioned, the UPiT is a b+ tree, and the internal nodes of the b+ tree contain only metadata pointers in a tree format. This allows a UPiT snapshot of a selected PiT to be created by manipulating only the metadata from the metadata volume (metadata object 510). In one example, no data is moved or copied when creating a UPiT snapshot.

[0059] More specifically, to access a time point in this structure, a rehydration process is performed using the metadata volume represented by metadata object 510. This is performed by scanning the relevant portion of the metadata stream to create the metadata volume. The data can then be accessed by providing access to the metadata volume that has been created from the relevant portion of the metadata stream. Alternatively, the metadata volume can be converted to a UPiT snapshot.

[0060] In one example, and because storage is typically limited, a moving protection window can be implemented, and this can cause some limitations to the arbitrary point in time functionality. Old data that has been overwritten can be deleted. In one example, one version of the data should be maintained at each disk location.

[0061] The metadata object or stream can be converted to a UPiT snapshot. As previously mentioned, the metadata object contains pointers to data and can be used to identify the data and an arbitrary point in time. Because the UPiT is a b+ tree implementation, and the internal nodes of the b+ tree contain only metadata pointers, the conversion or generation of a UPiT snapshot corresponding to an arbitrary point in time can be generated by adding the metadata from the metadata object to an existing UPiT.

[0062] Thus, the rehydration method can first obtain a UPiT 702 (or a copy of a UPiT). Next, metadata manipulation is performed 704. This can include appending 706 each pointer or metadata in the metadata object (e.g., associated with a selected point in time) to the internal nodes of the UPiT. The metadata stored in the metadata object can include metadata necessary to comply with the requirements of a b+ tree. This allows the metadata to be appended and effectively generate a new UPIT for the selected point in time. During this appending of metadata, no rebalancing 708 of the b+ tree is performed. After all of the relevant metadata has been appended to the b+ tree, the tree can be rebalanced 710.

[0063] This effectively creates a UPiT for the selected PiT. In addition, this is accomplished without moving or copying data when creating a new UPiT for the selected PiT. The new UPiT allows the volume or objects managed by the owner or DOM to be exposed 712 to the user if necessary. This can also be used for recovery purposes or other operations. In fact, the metadata object can be combined with an existing UPiT in order to generate another UPiT for the selected PiT.

[0064] Because a metadata object can be created for any PiT, Figure 7 This operation, shown in FIG. 6, can be used to recreate a snapshot history. The snapshot history is generated by creating a metadata volume for the requested PiT, converting the metadata volume to a UPiT, and inserting the new UPiT into the managed snapshot tree. This allows the UPiT history to be created after the fact.

[0065] Embodiments of the present invention provide an efficient way to obtain an arbitrary point-in-time representation of a VSAN environment. In addition, the UPiT representation is used to enable conversion and exposure in an efficient manner, moving only metadata. Thus, embodiments of the present invention enhance a VSAN implementation with any PiT data protection capabilities and an efficient way to expose volumes.

[0066] Although embodiments of the present invention have been discussed with respect to UPiT and VSAN environments, embodiments of the present invention are not so limited. Embodiments of the present invention can be applied to snapshots, including snapshots that are implemented using pointers or structures. Embodiments of the present invention can also be applied to existing snapshot technology, such that arbitrary point-in-time functionality can be implemented using a metadata stream. In another example, a metadata stream can be used to implement arbitrary point-in-time functionality in a system that does not use snapshots.

[0067] The following is a discussion of aspects of an example operating environment for various embodiments of the present invention. This discussion is not intended to limit the scope or applicability of embodiments of the present invention in any way.

[0068] Generally, embodiments of the application can be implemented in conjunction with systems, software, and components that individually and / or collectively facilitate and / or cause data management operations to be implemented. Such data management operations can include, but are not limited to, data read / write / delete operations, data backup operations, data restore operations, data clone operations, data archive operations, and disaster recovery operations. Thus, while the discussion herein can in some respects be directed to a discussion of data protection environments and operations, the scope of the application is not so limited. More generally, the scope of the application encompasses any operating environment in which the disclosed concepts can be useful. By way of illustration and not limitation, embodiments of the application can be employed in conjunction with data backup and restore platforms such as Dell-EMC NetWorker and Avamar and Data Domain platforms.

[0069] Data protection environments can take the form of public or private cloud storage environments, local storage environments, and hybrid storage environments that include public elements and private elements, although the scope of the application extends to any other type of data protection environment. Any of these example storage environments can be partially or entirely virtualized. The storage environment can include or consist of a data center that is operable to service read and write operations initiated by one or more clients.

[0070] In addition to the storage environment, the operating environment can include one or more host devices such as, for example, clients, each of which hosts one or more applications. Thus, a particular client can employ or otherwise be associated with one or more instances of each of one or more applications. Generally, the applications employed by a client are not limited to any particular function or type of function. For example, some example applications and data include email applications such as MS Exchange, file systems, and databases such as Oracle databases and SQL Server databases. Applications on a client can generate new and / or modified data that needs to be protected.

[0071] Any device or entity disclosed herein can be protected by one or more data protection policies in accordance with various embodiments of the application. Other examples of devices that can be protected by data protection policies in accordance with embodiments of the application include, but are not limited to, containers and VMs.

[0072] Any device in the operating environment, including clients, servers, and hosts, can take the form of software, a physical machine, or a virtual machine (VM), or any combination of these, although no particular device implementation or configuration is required for any embodiment. Similarly, data protection system components, such as, for example, databases, storage servers, storage volumes (LUNs), storage disks, replication services, backup servers, restore servers, backup clients, and restore clients, can also take the form of software, a physical machine, or a virtual machine (VM), although no particular component implementation is required for any embodiment. Where VMs are employed, a hypervisor or other virtual machine monitor (VMM) can be employed to create and control the VMs.

[0073] As used herein, the term "data" is intended to be very broad in scope. Thus, this term encompasses, by way of example but not limitation, data segments, data blocks, data chunks, atomic data, e-mails, objects of any type, files, directories, subdirectories, volumes, and any combination of one or more of the above, such as can result from a data stream segmentation process.

[0074] Example embodiments of the application are applicable to any system capable of storing and processing various types of objects in analog, digital, or other form. Although the terms such as document, file, chunk, or object can be used by way of example, the principles of the present disclosure are not limited to any particular form of representing and storing data or other information. Rather, such principles are equally applicable to any object capable of representing information.

[0075] Embodiments disclosed herein can include the use of specialized or general purpose computers including various computer hardware or software modules, as discussed in greater detail below. A computer can include a processor and a computer storage medium bearing instructions which, when executed by the processor and / or cause the processor to execute, perform any one or more of the methods disclosed herein.

[0076] As noted above, embodiments within the scope of the present application also include computer storage media, which is physical media that bears computer-executable instructions or data structures that implement the computer-readable instructions. Such computer storage media can be any available physical media that is accessible by a general purpose or special purpose computer.

[0077] By way of example, and not limitation, such computer storage media can include hardware storage devices such as solid state disk / drive (SSD), RAM, ROM, EEPROM, CD-ROM, flash memory, phase-change (‘PCM’) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other hardware storage devices which can be used to store program code in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer system, processor, or other device. Combinations of the above should also be included within the scope of computer storage media. Such media is also referred to as non-transitory storage media, and non-transitory storage media also encompasses cloud-based storage systems and structures, although the scope of the present invention is not limited to these examples of non-transitory storage media.

[0078] Computer-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing devices to perform a certain function or group of functions. Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0079] As used herein, the terms “module” or “component” can refer to software objects or routines that execute on the computing system. The different components, modules, engines, and services described herein can each be implemented as objects or processes that execute on the computing system (e.g., as separate threads). While the systems and methods described herein can be implemented in software, these can also be implemented in hardware or a combination of software and hardware, as can be desired. In the present disclosure, a “computing entity” can be any computing system as previously defined herein, or any combination of modules or components running on a computing system.

[0080] In at least some cases, a hardware processor is provided that is operable to execute executable instructions for performing a method or process, such as the methods and processes disclosed herein. The hardware processor can or can not include other elements of hardware, such as the computing devices and systems disclosed herein.

[0081] With respect to computing environments, embodiments of the present invention can be executed in a client-server environment, whether networked or local, or in any other suitable environment. Operating environments suitable for at least some embodiments of the present invention include cloud computing environments, where one or more of the clients, servers, or other machines can reside in and operate in a cloud environment.

[0082] The application can be implemented in other specific forms without departing from the spirit or essential characteristics thereof. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the application is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within the scope of the claims.

Claims

1. A method for implementing point-in-time functionality in a storage system, the method comprising: receiving an input / output at an owner of an object in the storage system, the storage system storing an object, the input / output associated with an object; extracting metadata associated with the input / output; manipulating the extracted metadata and storing the extracted metadata in a metadata object owned by the owner and storing the input / output in a data bucket, the data bucket storing an object copied from the storage system; performing the input / output on the object at the storage system; generating a new snapshot for a selected point-in-time by manipulating the metadata object without copying or moving data included in the object stored in the data bucket; restoring a production volume by: scanning the metadata object to determine relevant metadata included in the metadata object; creating a metadata volume from a blank metadata volume by populating the blank metadata volume with relevant metadata from the metadata object corresponding to the new snapshot, the metadata including identifiers, wherein each identifier points to data stored in an object in the data bucket, and hydrating the production volume using the metadata volume and the object while creating the metadata volume, wherein the production volume corresponds to the selected point-in-time.

2. The method of claim 1, further comprising: converting the metadata object to a snapshot of the object and appending relevant portions of the metadata object to the snapshot of the object.

3. The method of claim 2, wherein, the snapshot of the object includes a b+ tree, and wherein the new snapshot includes a b+ tree.

4. The method of claim 3, wherein, the storage system includes a VSAN.

5. The method of claim 4, further comprising: relevant portions of the metadata object are appended to internal nodes of the b+ tree.

6. The method of claim 1, wherein, all IO for the object is through the owner.

7. The method of claim 1, further comprising: exposing the object based on the new snapshot.

8. The method of claim 1, further comprising: rebalancing the new snapshot after all relevant portions of the metadata object are appended to the snapshot of the object.

9. The method of claim 1, further comprising: generating the new snapshot a posteriori.

10. A system for implementing point-in-time functionality in a storage system, the system comprising: a processor and a storage device, wherein the processor is configured to: implement an owner of an object in the storage system, wherein all input / output for the object in the storage system is managed by the owner, the input / output associated with an object; receive an input / output at the owner of an object in the storage system; extract metadata associated with the input / output; manipulate the extracted metadata and store the extracted metadata in a metadata object owned by the owner and store the input / output in a data bucket, the data bucket storing an object copied from the storage system; perform the input / output on the object at the storage system; and generate a new snapshot for a selected point-in-time by manipulating the metadata object without copying or moving data included in the object stored in the data bucket; restore a production volume by: scanning the metadata object to determine relevant metadata included in the metadata object; creating a metadata volume from a blank metadata volume by populating the blank metadata volume with relevant metadata from the metadata object corresponding to the new snapshot, the metadata including identifiers, wherein each identifier points to data stored in an object in the data bucket, and hydrating the production volume using the metadata volume and the object while creating the metadata volume, wherein the production volume corresponds to the selected point-in-time. creating a metadata volume from the blank metadata volume by writing relevant metadata in the metadata object corresponding to the new snapshot to fill the blank metadata volume, the metadata including identifiers, wherein each identifier points to data stored in an object in the data bucket, and simultaneously with creating the metadata volume, hydrating the production volume using the metadata volume and the object, wherein the production volume corresponds to the selected time point.

11. The system of claim 10, wherein, The owner is configured to convert the metadata object to a snapshot of the object and append relevant portions of the metadata object to the snapshot of the object.

12. The system of claim 11, wherein, The snapshot of the object includes a b+ tree, and wherein the new snapshot includes a b+ tree.

13. The system of claim 12, wherein, The storage system includes a VSAN.

14. The system of claim 13, wherein, The owner is configured to append relevant portions of the metadata object to internal nodes of the b+ tree.

15. The system of claim 10, wherein, The processor is configured to expose the object based on the new snapshot.

16. The system of claim 10, wherein, The owner is configured to balance the new snapshot after all relevant portions of the metadata object are appended to the snapshot of the object.

17. The system of claim 10, wherein, The processor is configured to generate the new snapshot a posteriori.

18. The system of claim 10, further comprising: generating a history of snapshots for the storage device.

19. The system of claim 10, wherein, The snapshot includes a UPiT snapshot, further comprising converting the UPiT snapshot to a selected time point snapshot having the same format as the UPiT snapshot by incorporating at least a portion of the metadata object into the UPiT snapshot.

Images