Using replicas to perform backup operations
By generating backup images on the virtual machine and leveraging metadata in the replica, the problem of network bandwidth wasting in data backup and replication is solved, and efficient backup and replication operations are achieved, reducing costs and improving data recovery capabilities.
Patent Information
- Application Number
- CN201880076393.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2017-11-30
- Filing Date
- 2018-11-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2039-05-13
AI Technical Summary
When modern enterprises implement data backup and replication solutions, the existing technology is difficult to effectively solve the network bandwidth waste and cost increase caused by repeated data transmission.
By creating a copy of the virtual machine, using metadata to generate backup images, you can generate a full or incremental backup image with just a single data transfer, avoiding redundant transmissions.
Significantly save network bandwidth and storage resources, reduce network costs, and provide granular data restoration capabilities.
Smart Images

Figure CN111417931B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to performing backup operations. In particular, the present disclosure relates to performing backup operations using replicas. Background Art
[0002] To prevent data loss, data can be backed up. Data backup operations produce backup images. Backup images include application data or a backup copy of the contents of a given storage entity (such as a file system or disk drive). Such backup images may be stored and / or maintained in a cloud-based remote storage device in the form of cloud backups for later retrieval and restoration, such as in the event of hardware failure.
[0003] Data replication involves making duplicate copies of data and improving reliability, fault tolerance, and / or accessibility. The purpose of data replication is to prevent data loss if a failure or disaster occurs at one location. If such a failure or disaster does occur that destroys or damages the data replica in one location, a data replica at a secondary site can be accessed. For these and other purposes, the replicated data may also be stored and / or maintained, for example, in a copy in a cloud-based remote storage device.
[0004] Backup solutions and replication solutions each have their advantages and disadvantages, and are therefore often implemented (and packaged) together, for example, in cloud backup and cloud-based disaster recovery (DR) services (e.g., as backup targets and DRaaS (Disaster Recovery as a Service)). For example, backup involves making a secondary replica of the data that can be restored for use if the primary replica is lost or unavailable. A backup image includes a point-in-time replica of the original data that is typically acquired on a repeating period (e.g., daily, monthly, yearly, etc.). Replication, on the other hand, is either synchronous or asynchronous and implements the transfer of data to a remote replica immediately or within a short time delay, respectively. In modern computing environments, both backup and replication require the transmission of large amounts of data over a network (e.g., to a cloud backup and DR service provider).
[0005] Furthermore, because data corruption or user file deletions are immediately or very quickly replicated to the secondary replica, replication is ineffective as a backup method. Additionally, because replication maintains only a unique copy of the data at the cloud location, the replicated copy does not include historical versions of the data (e.g., from previous weeks, months, etc.). In contrast, because the backup copy is a point-in-time copy of the data, full and incremental backup images can be created, providing significant storage benefits.
[0006] It is therefore not surprising that modern businesses and organizations implement both backup solutions and replication solutions to, for example, protect and secure the same data. Unfortunately, such modern businesses and organizations incur significant separate costs associated with network bandwidth—for cloud backup and separately for cloud-based DR services. For example, the same data that is transmitted from a local storage device to a cloud-based storage device as part of a backup operation must be retransmitted from the local storage device to the cloud-based storage device as part of a replication operation. This retransmission of data is not only redundant and prevents cloud adoption, but also results in expensive and wasteful network bandwidth utilization. Summary of the invention
[0007] Disclosed herein are methods, systems, and processes for performing backup operations using replicas. One such method involves creating a replica of a virtual machine as part of a replication operation. In this example, the replica includes data associated with the virtual machine and metadata associated with an application executed on the virtual machine. The method then performs the backup operation by generating a backup image from the replica using the metadata.
[0008] In one embodiment, the metadata includes information for granular restore of the virtual machine, is generated by quiescing the application, and is stored on the virtual disk. In another embodiment, generating the backup image includes generating a snapshot identifying a reference point in time to create a backup image of the data before performing the backup operation, retrieving the metadata from the copy, and generating a full backup image or an incremental backup image based on the reference point in time.
[0009] In some embodiments, the metadata includes data locations of the data blocks, and the data locations are used by the backup server to restore the virtual machine from the backup image. In other embodiments, the backup operation is performed by a backup agent instantiated on the cloud computing device as part of performing the backup operation, the backup agent is another virtual machine, container, or serverless module, the quiescing of the application is performed by a replication agent executing on the virtual machine, the metadata is generated by the backup agent executing on the virtual machine, and the replication operation is performed by the replication agent.
[0010] In some embodiments, generating the backup image includes generating a snapshot of the data at a reference point in time, generating the backup image in a backup format, mounting the snapshot, retrieving granular metadata for one or more applications from metadata in the replica, and sending the granular metadata to a backup server.
[0011] The above content is an overview, and therefore necessarily contains simplifications, generalizations, and omissions of details; therefore, those skilled in the art will understand that the overview is illustrative only and is not intended to be limiting. Other aspects, features, and advantages of the present disclosure, as defined solely by the claims, will become apparent in the non-limiting detailed description set forth below. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The present disclosure may be better understood, and its objects, features and advantages will become apparent to those skilled in the art by referring to the accompanying drawings.
[0013] Figure 1 is a block diagram 100 of a backup and replication computing system according to one embodiment of the present disclosure.
[0014] Figure 2 is a block diagram 200 of a backup and replication computing environment according to one embodiment of the present disclosure.
[0015] Figure 3 is a block diagram 300 of a backup and replication computing environment according to one embodiment of the present disclosure.
[0016] Figure 4 is a flow chart 400 of a process for generating a backup image from a copy according to one embodiment of the present disclosure.
[0017] Figure 5A is a flow diagram 500A of a process for storing metadata for replication according to one embodiment of the present disclosure.
[0018] Figure 5B is a flow chart 500B of a process for replicating metadata according to one embodiment of the present disclosure.
[0019] Figure 6 is a flow chart 600 of a process for storing metadata for a restore operation according to one embodiment of the present disclosure.
[0020] Fig. 7A is a flow diagram 700A of a process for tracking changed data as part of replication according to one embodiment of the present disclosure.
[0021] Figure 7B is a flowchart 700B of a process for generating a full or incremental backup from a copy according to one embodiment of the present disclosure.
[0022] Figure 8 is a block diagram of a computing system 800 according to one embodiment of the present disclosure.
[0023] Fig. 9is a block diagram of a networking system 900 according to one embodiment of the present disclosure.
[0024] Although the present disclosure allows various modifications and alternative forms, specific embodiments of the present disclosure are provided as examples in the drawings and the detailed description. It should be understood that the drawings and the detailed description are not intended to limit the present disclosure to the specific forms disclosed. On the contrary, it is intended to cover all modifications, equivalents and substitutes that fall within the essence and scope of the present disclosure defined by the appended claims. DETAILED DESCRIPTION
[0025] introduce
[0026] Data backup and data replication are not mutually exclusive, and therefore are often implemented together by modern enterprises and organizations to protect and provide the same data. For example, cloud-based service providers often provide cloud-based backup (e.g., for data restoration, data security, etc.) and cloud-based replication services (e.g., Disaster Recovery as a Service (DRaaS), etc.).
[0027] A backup operation produces a point-in-time copy of data, which includes a copy of the operating system (OS) and data associated with the computing device and / or virtual machine, including system state and application configuration. The backup is typically saved as a single file called a backup image (e.g., on a cloud storage device), and the backup process uses snapshots to ensure that even deleted files and empty disk blocks are backed up. Therefore, the backup operation produces backup metadata, which is collected during the backup process and stored as part of the backup image to identify the state of the system and corresponding applications at a certain point in time (e.g., for granular restore purposes). This backup metadata can be used to create future incremental backup images, and thereby provide significant network and storage savings, as well as the ability to restore a computing device from a backup image by reassembling the computing device's data from one or more previous backup images.
[0028] On the other hand, the replication operation produces an identical copy of the data associated with the computing device and / or virtual machine. Such an identical copy is stored as a replica (e.g., on a cloud storage device). Because the replication process only produces a single identical copy of the data (e.g., synchronously or asynchronously, which can be copied back in its entirety in the event of a disaster), the replication process does not (and indeed does not need to) replicate metadata associated with the replicated data (e.g., information about historical versions of the data, block and / or file location information, block-level information for granular restore, one or more data locations, etc.). Therefore, the replica does not contain information that enables the creation of a backup image and / or allows granular restore of data from such a backup image.
[0029] Unfortunately, and as previously mentioned, in modern computing environments, both backup operations and replication operations involve the transmission of large amounts of data over a network. Because enterprises implement both backup solutions and replication solutions to protect and secure the same data, these enterprises incur significant and different costs associated with network bandwidth—for cloud backup and separately for cloud-based DR services. For example, the same data that is transmitted from a local storage device to a cloud-based storage device as part of a backup operation must be retransmitted from the local storage device to the cloud-based storage device as part of a replication operation. This retransmission of data is not only redundant and prevents cloud adoption, but also results in expensive and wasteful network bandwidth utilization.
[0030] Disclosed herein are methods, systems, and processes for using replicas (eg, replicated data) to generate full or incremental backup images while requiring only a single transfer of the data as part of the replication operation.
[0031] Exemplary Backup and Replication Computing Environment
[0032] Figure 1 1 is a block diagram 100 of a backup and replication computing system according to one embodiment. Figure 1 As shown, the exemplary integrated backup and replication computing environment includes at least virtual machines 105(1)-(N) communicatively coupled to (corresponding) virtual disks 120(1)-(N). It should be noted that in addition to virtual machines and virtual disks, the exemplary integrated backup and replication computing environment may also include physical computing devices and physical storage disks.
[0033] Virtual machine 105(1) implements at least backup agent 110(1) and replication agent 115(1). Backup agent 110(1) generates metadata associated with a backup operation, and replication agent 110(1) quiesces one or more applications executing on virtual machine 105(1) (e.g., instructs the application to flush data in its memory to virtual disk 120(1)). Virtual disk 120(1) stores data 125(1) and metadata 130(1) (e.g., backup metadata generated by backup agent 110(1)). Virtual machines 105(1)-(N) (including virtual disks 120(1)-(N)) are part of computing systems 190(1)-(N) and are communicatively coupled to backup server 135.
[0034] Backup server 135 performs backup operations, replication operations, and / or restore operations and implements at least backup manager 140, replication manager 145, and catalog 150. Backup manager 140 manages backup operations to cloud computing devices 155 (e.g., in conjunction with backup agent 165), replication manager 145 manages replication operations performed on cloud computing devices 155 (e.g., via replication agents 115(1)-(N)), and catalog 150 is stored and used by backup server 135 to perform restore operations (e.g., granular restore).
[0035] like Figure 1 As shown, cloud computing device 155 is communicatively coupled to backup server 135 via network 185 and implements at least a disaster recovery (DR) coordinator 160 (which provides one or more DRaaS services) and a backup agent 165 (instantiated to perform backup operations). Cloud computing device 155 is communicatively coupled to cloud storage device 170 and, together with cloud storage device 170, is part of cloud computing system 195 (1)-(N). Cloud storage device 170 stores replicas 175 (1)-(N) and backup images 180 (1)-(N).
[0036] Backup server 135, physical computing devices as part of computing systems 190(1)-(N), and / or cloud computing device 155 may be any of a variety of different computing devices, including servers, personal computing devices, laptop computers, cellular phones, etc. Network 185 may be any type of network and / or interconnect (e.g., the Internet, a wide area network (WAN), etc.). Cloud storage device 170 and / or physical storage devices associated with computing systems 190(1)-(N) may include one or more of a variety of different storage devices, including hard disks, optical disks, digital versatile disks, solid state drive (SSD) memory such as flash memory, etc., or one or more logical storage devices such as volumes implemented on one or more such physical storage devices.
[0037] In one embodiment, replication agent 115(1) implemented by virtual machine 105(1) creates a replica of virtual machine 105(1) as part of a replication operation. In this example, the replica (e.g., replica 175(1)) includes data associated with virtual machine 105(1) (e.g., data 125(1)) and metadata associated with applications executing on virtual machine 105(1) (e.g., metadata 130(1)). Backup agent 165 then uses the metadata to generate a backup image (e.g., backup image 180(1)) based on the replica.
[0038] In some embodiments, metadata 130(1) includes information (e.g., one or more data locations, etc.) for granular restoration of virtual machine 105(1). Replica agent 115(1) generates and / or collects metadata 130(1) by quiescing one or more applications executing on virtual machine 105(1) (e.g., by instructing one or more applications to flush data in their memory to virtual disk 120(1) - thus, storing metadata 130(1) on virtual disk 120(1)). In this example, metadata 130(1) includes the locations of data blocks, and backup server 135 uses the data locations to restore virtual machine 105(1) from backup image 180(1) if necessary.
[0039] In other embodiments, generating a backup image includes generating a snapshot identifying a reference point in time to create a backup image of data 125(1) before performing a backup operation, retrieving metadata 130(1) from replica 175(1), and generating a full backup image or an incremental backup image based on the reference point in time. In this example, the backup operation (e.g., generating a backup image) is performed by backup agent 165, which is instantiated by backup manager 140 as another virtual machine, container, or serverless module on cloud computing device 155.
[0040] In some embodiments, generating backup image 180(1)-(N) includes: generating a snapshot of data (e.g., data 125(1)-(N) of virtual machines 105(1)-(N)) at a reference point in time, generating backup image 180(1) in a backup format, mounting the snapshot, retrieving application metadata 130(1) from replica 175(1) (e.g., granular metadata captured by replication agent 115(1) and replicated to cloud storage device 170 by replication manager 145 and DR coordinator 160), and sending the granular metadata to backup server 135 (e.g., for use as catalog 150 for granular restore).
[0041] As previously described, replica 175(1) contains the copied data of virtual machine 105(1) and is stored in cloud storage device 170 in the same format as data 125(1) stored on virtual disk 120(1). Therefore, replica 175(1) is a mirror copy of data 125(1) (e.g., there is a 1:1 correspondence between the data blocks of replica 175(1) and data 125(1)), and is thus available for failover operations to be performed on cloud computing device 155. For example, a cloud virtual machine may be created and / or instantiated based on replica 175(1).
[0042] On the other hand, and as also discussed above, a backup may be generated in a different format than the original data (e.g., as a result of incremental backups, deduplication, etc.). Thus, at least per se, the backup image does not maintain a 1:1 correspondence between the data blocks of backup image 180(1) and data 125(1). Furthermore, the backup image may be generated in a different format than the original format of data 125(1). For example, data 125(1) may be a single virtual machine disk image, whereas backup image 180(1) may include several one-megabyte blocks.
[0043] Example of generating backups on a schedule via a DRaaS system
[0044] Figure 2 2 is a block diagram 200 of a backup and replication computing environment according to one embodiment. Figure 2 As shown, cloud 205 (e.g., cloud computing system 195(1)-(N)) includes at least backup agent 165, replicas 175(1) and 175(2), and DR coordinator 160. DR coordinator 160 continuously replicates the virtual machine disk images of virtual machines 105(1) and 105(2) to cloud 205 (e.g., creates corresponding virtual machine disk images in cloud 205 and continuously replicates them to keep the virtual machine disk images synchronized with virtual disk 120(1)).
[0045] The backup server 135 monitors and manages quiescing and application granular restore (GRT) metadata aggregation (e.g., metadata of an email database and associated information, such as emails, locations, files, etc.) performed by local backup agents and replication agents. When local replication agents (e.g., replication agents 115(1)-(N)) perform replication, replication consistency checkpoints are performed as part of the GRT metadata capture process. For example, when the backup agent 110(1) communicates the need for a backup operation to the replication agent 115(1), the replication agent 115(1) quiesces the application 210(1) and sets a marker, flag, or similar type of indicator that indicates a consistent point in time for the backup operation to be continued if and when the backup agent 165 performs the backup operation in the cloud 205. It should be noted that the replication agents 115(1)-(N) and the backup agents 110(1)-(N) can be implemented on guest virtual machines as well as by the host hypervisor.
[0046] In one embodiment, to create a backup image on a schedule by a DRaaS system (e.g., DR coordinator 160), a local backup agent (e.g., backup agent 110(1)) first interacts with a DRaaS replication agent (e.g., replication agent 115(1)) to indicate that a backup is needed. Next, replication agent 115(1) performs an application quiesce and invokes a custom action of backup agent 110(1). For example, replication agent 115(1) requests and / or instructs application 210(1) to push in-memory data to virtual disk 120(1), and backup agent 110(1) registers a callback function with replication agent 115(1).
[0047] As part of the custom action, backup agent 110(1) collects high-level metadata (e.g., granular metadata, such as metadata for emails in a database, data location of application data, block-level information, etc.) and stores it on virtual disk 120(1) (e.g., as metadata 130(1)). Backup agent 110(1) also communicates backup method information (e.g., whether the backup is a full backup or an incremental backup) to replication manager 145. Next, replication manager 145, in conjunction with replication agent 115(1), synchronizes the captured metadata (e.g., granular metadata, GRT metadata, etc.) to cloud 205. For example, after replication agent 115(1) completes application quiescing, backup agent 110(1) captures the metadata information and stores the metadata information (e.g., in a file on virtual disk 120(1) and / or as metadata 130(1)). As part of the replication operation, this captured and stored metadata is replicated to cloud 205 (e.g., stored as part of replica 175(1)).
[0048] Figure 3 3 is a block diagram of a backup and replication computing environment according to one embodiment. After granular metadata is synchronized with cloud 205, DR coordinator 160 invokes backup agent 165. Backup agent 165 (which, as previously described, may be instantiated as a virtual machine, container, or serverless module in cloud 205 by backup manager 140) generates a snapshot of the data at this point in time (e.g., data 125(1) is replicated as replica 175(1) via a replication consistency checkpoint) and generates a backup image in a backup-friendly storage device provided by cloud computing device 155 (e.g., if backup image 180 is generated as an incremental backup image, backup image 180 is generated in a backup-friendly format from a replication consistency checkpoint) (e.g., writing a block or list of blocks to a block blob, appending a block to an append blob, writing a page to a page blob, etc.).
[0049] Backup agent 165 (which can be instantiated on demand in cloud 205) then mounts the snapshot and, using the information gathered from the application quiesce (e.g., application information), gathers, extracts, and / or collects full item-level granular metadata from the files and applications. For example, backup agent 165 gathers granular metadata for emails in an email database on a machine image (e.g., replica 175(1)). This granular metadata is read by backup server 135 and stored in catalog 150 (e.g., as a rich granular catalog of backup image 180).
[0050] As previously described, backup agent 110(1) sends information to replication agent 115(1) to notify replication agent 115(1) that backup manager 140 has requested a full backup. Replication agent 115(1) uses this point in time as a basis to maintain and / or track changed data (e.g., incremental or log) as part of the replication process. In addition, depending on whether backup agent 165 is creating a full backup image or an incremental backup image, backup agent 165 reads all data or only changed data from a reference / base point in time to create the corresponding backup image.
[0051] Example process for generating a backup image from a copy
[0052] Figure 4 4 is a flowchart of a process for generating a backup image and performing a granular restore from a replica according to one embodiment. The process begins at 405 by receiving a replica of a virtual machine (e.g., replication agent 115(1) implemented on virtual machine 105(1) sends data 125(1) and metadata 130(1) to cloud computing device 155). At 410, the process invokes a backup agent (e.g., backup agent 165, which may be instantiated as a virtual machine, container, serverless module, etc., on demand in cloud 205 by backup manager 140). It should be noted that the backup agent is invoked and is executed according to a backup schedule. Figure 4 At 415, the process takes a snapshot of the data (e.g., data 125(1) in replica 175(1)) and creates a backup image in a backup format at 420 (e.g., backup agent 165 creates backup image 180 on backup storage device 305, such as Figure 3 At 425, the process mounts the snapshot, and at 430, extracts granular metadata (e.g., from metadata 130(1)) in the replica (e.g., replica 175(1)). At 435, the process sends the backup image and granular metadata to the backup server (e.g., backup image 180 and GRT metadata to backup server 135, as shown). Figure 3), and at 440, it is determined whether there is more replicated data. If there is more replicated data (and a backup operation has been requested, followed by a restore operation applicable to the new replicated data), the process loops to 405. Otherwise, the process ends.
[0053] Figure 5A 500A is a flow diagram of a process for storing metadata for replication according to one embodiment. The process begins at 505 by instructing (or requesting) a replication agent (e.g., replication agent 115(1)) to begin a backup operation configuration (e.g., a full backup has been requested and a replication consistency checkpoint has been set). At 510, the process receives a call for a custom action from the replication agent after an application quiesce has been performed (e.g., by backup agent 110(1)). The process ends at 515 by storing application information associated with one or more applications (e.g., GRT metadata with one or more data locations, etc.).
[0054] Figure 5B 500B is a flow diagram of a process for replicating metadata according to one embodiment. The process begins at 520 by receiving a backup operation instruction from a backup agent (e.g., backup method information from backup agent 110(1)). At 525, the process performs an application quiesce (e.g., a flush of data in memory), and at 530, invokes a custom action of the backup agent (e.g., backup agent 110(1) registers a callback function with replication agent 115(1)). At 535, the process receives metadata (e.g., high-level metadata about the application, its data locations, etc., gathered by backup agent 110(1) by executing one or more queries). The process ends at 540 by replicating the metadata along with the virtual machine data to a cloud storage device (e.g., to DR coordinator 160, cloud computing device 155, and / or cloud 205 as part of the replica).
[0055] Figure 6600 is a flow diagram of a process for storing metadata for restore operations according to one embodiment. The process begins at 605 by determining whether replication synchronization is complete. If replication synchronization (e.g., metadata synchronization with a computing-friendly cloud storage device) is not complete, the process waits at 610. However, if replication synchronization is complete, at 615, the process takes a snapshot of the replicated data (e.g., the data in the replica), and at 620, generates a backup image in a backup format. At 625, the process mounts the snapshot, and at 630, aggregates item-level granular metadata for files and applications from the replica. At 635, the process stores the granular metadata for future restore operations from the backup image. At 640, the process determines whether there is more replicated data (e.g., whether a new backup operation has been requested). If there is more replicated data, the process loops to 605. Otherwise, the process ends.
[0056] Fig. 7A 700A is a flow diagram of a process for tracking changed data as part of replication according to one embodiment. The process begins at 705 with receiving information from a backup agent indicating that a full backup has been requested. The process ends at 710 with tracking changed data or log information from that (reference) point in time as part of a replication operation (e.g., a replication consistency checkpoint).
[0057] Figure 7B 700B is a flow diagram of a process for generating a full or incremental backup from a replica according to one embodiment. The process begins at 715 by determining whether the backup agent is creating a full backup or an incremental backup. If the backup agent is creating a full backup, then at 720, the process reads all data from a reference point in time and ends at 725 with generating a full backup image. However, if the backup agent is creating an incremental backup, then at 730, the process reads only changed data from a reference point in time and ends at 735 with generating an incremental backup image.
[0058] In this way, the methods, systems, and processes described herein enable the creation of cloud backups from DRaaS replicas and provide support for rich granular metadata for the backups (e.g., for granular restores). These methods, systems, and processes eliminate the need to perform a separate backup from a local data center to cloud storage when deploying DRaaS replication, and thereby provide significant benefits with respect to bandwidth usage and network costs by building full or incremental backup images and full granular metadata directly from replicas stored in the cloud.
[0059] Other Implementations
[0060] In one embodiment, application quiescing is performed even when it is not required to do so and / or even when there is applicability for granular restore. For example, application quiescing is performed to make the backup consistent with the application. In another embodiment, application metadata, etc., used for granular restore includes information such as: drives (e.g., C:, D:, etc.), volumes installed on virtual machines and / or physical machines, applications installed on virtual machines and / or physical machines, and associated information (e.g., database version information, database name, etc.), and the location of files and folders containing application data.
[0061] In some embodiments, the metadata required to generate incremental backups from replicas may be of at least two types. First, in the case where the virtual machine has built-in data block change tracking capabilities, the metadata may include an identifier that captures the state of the virtual machine disk at the time of the snapshot taken for backup. Therefore, when generating incremental backups from replicas, the potential change tracking mechanism may be queried to obtain a list of changed blocks between the previous full backup or incremental backup (change tracking identifier) and the current change tracking identifier. Once the list of changed data blocks is obtained, an incremental backup including those changed data blocks may be created. Second, in the case where the potential change tracking mechanism does not exist or cannot be used, the replication agent and / or replication system may track the changed blocks between the previous snapshot and the current snapshot (because the replication system detects and only replicates the changed blocks). When creating a backup from a replica, the backup agent may then use this information to obtain a list of changed data blocks since the previous full backup or incremental backup, and create an incremental backup including only those changed data blocks.
[0062] In other embodiments, a single backup image may not necessarily be a single file, and may include multiple objects created in a cloud storage device. In certain embodiments, the use of snapshots in the systems, methods, and processes described herein may be at least twofold. First, snapshots may be used to obtain a frozen view (e.g., in the case where data changes while a backup is in progress, resulting in an inconsistent backup). In this case, the snapshot provides a view of the data that has not changed. Second, the application is first paused and then the snapshot is generated. This process results in application consistency and application consistent backups because the pause causes the application to refresh the data in its memory, etc. to the disk so that the disk state becomes consistent with the application. In this example, a pause of the file system and / or application is performed to enable a consistent backup (e.g., a file system consistent backup and an application consistent backup, respectively).
[0063] In one embodiment, the replication operation may involve replication metadata, which may not be the same information required for backup operations and restore operations (e.g., granular metadata). In another embodiment, and in certain configurations and deployments, the backup agent and the replication agent may not be implemented in a virtual machine, but may be implemented in a hypervisor host or even in a different virtual machine (e.g., a virtual machine appliance) on the hypervisor host. Likewise, the replication manager may be implemented on a different computing device other than the backup server.
[0064] In some embodiments, the backup agent quiesces one or more applications, triggers snapshot creation, and generates metadata. The backup agent then communicates with the replication agent to cause the replication agent to mark this state with a consistency marker in the replication stream so that the replica will enable the state marked for the backup agent (the marked state) to be used to create a backup from the replica (e.g., as opposed to using the latest state of the replica, which may change frequently). In this example, both virtual machines and physical machines can be data sources.
[0065] In one embodiment, a replication consistency checkpoint involves capturing and / or remembering a time at which a consistent snapshot was taken for use in a backup operation. This enables a backup agent in the cloud to return the replica to that point in time when creating a backup image from the replica. Thus, a replication consistency checkpoint is a point in time that a replication agent notes, remembers, and / or tracks in order to be able to return to (e.g., to provide a backup agent with a view of the data at that point in time).
[0066] Exemplary Computing Environment
[0067] Figure 8 800 is a block diagram of a computing system 800 according to one embodiment of the present disclosure. The computing system 800 may include a cloud computing device 155 and broadly represents any single-processor or multi-processor computing device or system capable of executing computer-readable instructions. Examples of the computing system 800 include, but are not limited to, any one or more of the following various devices: a workstation, a personal computer, a laptop, a client-side terminal, a server, a distributed computing system, a handheld device (e.g., a personal digital assistant and a mobile phone), a network device, a storage controller (e.g., an array controller, a tape drive controller, or a hard drive controller), etc. In its most basic configuration, the computing system 800 may include at least one processor 855 and a memory 860. By executing the software executing the backup agent 165, the computing system 800 becomes a dedicated computing device configured to perform backup operations using a replica (e.g., a replica stored in a public cloud, etc.).
[0068] Processor 855 generally represents any type or form of processing unit capable of processing data or interpreting and executing instructions. In certain embodiments, processor 855 may receive instructions from software applications or modules. These instructions may enable processor 855 to perform the functions of one or more embodiments in the embodiments described and / or shown herein. For example, processor 855 may perform all or some of the operations described herein and / or may be a device for performing all or some of the operations described herein. Processor 855 may also perform any other operation, method or process described and / or shown herein and / or may be a device for performing any other operation, method or process described and / or shown herein. Memory 860 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or other computer-readable instructions. Examples include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory or any other suitable memory device. Although not required, in certain embodiments, computing system 800 may include both volatile memory unit and non-volatile storage device. In one example, program instructions implementing backup agent 165 and DR coordinator 160 (or alternatively, backup agent 110 and replication agent 115 ) may be loaded into memory 860 .
[0069] In some embodiments, computing system 800 may include one or more components or elements in addition to processor 855 and / or memory 860. For example, Figure 8 As shown, computing system 800 may include memory controller 820, input / output (I / O) controller 835, and communication interface 845, each of which may be interconnected via communication infrastructure 805. Communication infrastructure 805 generally represents any type or form of infrastructure capable of facilitating communication between one or more components of a computing device. Examples of communication infrastructure 805 include, but are not limited to, communication buses (such as Industry Standard Architecture (ISA), Peripheral Component Interconnect (PCI), PCI express (PCIe), or similar buses) and networks.
[0070] The memory controller 820 generally represents any type / form of device capable of handling memory or data or controlling communications between one or more components of the computing system 800. In certain embodiments, the memory controller 820 may control communications between the processor 855, the memory 860, and the input / output controller 835 via the communication infrastructure 805. In certain embodiments, the memory controller 820 may perform, alone or in combination with other elements, one or more of the operations or features described and / or illustrated herein and / or may be a means for performing, alone or in combination with other elements, one or more of the operations or features described and / or illustrated herein.
[0071] The input / output controller 835 generally represents any type or form of module capable of coordinating and / or controlling the input and output functions of a virtual machine and / or a physical computing device. For example, in certain embodiments, the input / output controller 835 may control or facilitate data transfer between one or more elements of the computing system 800, such as the processor 855, the memory 860, the communication interface 845, the display adapter 815, the input interface 825, and the storage interface 840.
[0072] The communication interface 845 broadly represents any type or form of communication device or adapter capable of facilitating communication between the computing system 800 and one or more other devices. The communication interface 845 can facilitate communication between the computing system 800 and a private or public network including additional computing systems. Examples of the communication interface 845 include, but are not limited to, a wired network interface (such as a network interface card), a wireless network interface (such as a wireless network interface card), a modem, and any other suitable interface. The communication interface 845 can provide a direct connection to a remote server via a direct link to a network such as the Internet, and can also provide such a connection indirectly through, for example, a local area network such as an Ethernet network, a personal area network, a telephone or cable network, a cellular phone connection, a satellite data connection, or any other suitable connection.
[0073] The communication interface 845 may also represent a host adapter that is configured to facilitate communication between the computing system 800 and one or more additional network or storage devices via an external bus or communication channel. Examples of host adapters include: Small Computer System Interface (SCSI) host adapter, Universal Serial Bus (USB) host adapter, Institute of Electrical and Electronics Engineers (IEEE) 1394 host adapter, Serial Advanced Technology Attachment (SATA), Serial SCSI (SAS) and External SATA (eSATA) host adapters, Advanced Technology Attachment (ATA) and Parallel ATA (PATA) host adapters, Fibre Channel interface adapters, Ethernet adapters, etc. The communication interface 845 may also allow the computing system 800 to perform distributed or remote computing (e.g., by receiving instructions from a remote device / sending instructions to a remote device for execution).
[0074] like Figure 8As shown, the computing system 800 may also include at least one display device 810, which is coupled to the communication infrastructure 805 via a display adapter 815. The display device 810 generally represents any type or form of device capable of visually displaying information forwarded by the display adapter 815 (e.g., in a GUI). Similarly, the display adapter 815 generally represents any type or form of device configured to forward graphics, text, and other data from the communication infrastructure 805 (or from a frame buffer, as known in the art) for display on the display device 810. The computing system 800 may also include at least one input device 830 coupled to the communication infrastructure 805 via an input interface 825. The input device 830 generally represents any type or form of input device capable of providing input generated by a computer or a person to the computing system 800. Examples of the input device 830 include a keyboard, a pointing device, a voice recognition device, or any other input device.
[0075] The computing system 800 may also include a storage device 850, which is coupled to the communication infrastructure 805 via a storage interface 840. The storage device 850 generally represents any type or form of storage device or medium capable of storing data and / or other computer-readable instructions. For example, the storage device 850 may include a disk drive (e.g., a so-called hard disk drive), a floppy disk drive, a tape drive, an optical disk drive, a flash drive, etc. The storage interface 840 generally represents any type or form of interface or device for transmitting and / or transferring data between the storage device 850 and other components of the computing system 800. The storage device 850 may be configured to read and / or write to a removable storage unit configured to store computer software, data or other computer-readable information. Examples of suitable removable storage units include floppy disks, tapes, optical disks, flash memory devices, etc. The storage device 850 may also include other similar structures or devices for allowing computer software, data or other computer-readable instructions to be loaded into the computing system 800. For example, the storage device 850 may be configured to read and write software, data or other computer-readable information. The storage device 850 may also be part of the computing system 800 or may be a separate device accessed through other interface systems.
[0076] Many other devices or subsystems may be connected to computing system 800. Conversely, Figure 8 The presence of the components and devices shown is not necessary to practice the embodiments described and / or shown herein. The devices and subsystems mentioned above can also be used in accordance with Figure 8The computing system 800 may also be interconnected in different ways as shown. The computing system 800 may also adopt any number of software configurations, firmware configurations, and / or hardware configurations. For example, one or more of the embodiments disclosed herein may be encoded as a computer program (also referred to as computer software, software applications, computer-readable instructions, or computer control logic) on a computer-readable storage medium. Examples of computer-readable storage media include magnetic storage media (e.g., hard disk drives and floppy disks), optical storage media (e.g., CD-ROMs or DVD-ROMs), electronic storage media (e.g., solid-state drives and flash memory media), etc. Such computer programs may also be transferred to the computing system 800 to be stored in a memory or on a carrier medium via a network such as the Internet.
[0077] A computer-readable medium containing a computer program may be loaded into a computing system 800 (e.g., a virtual machine 105(1)-(N), a backup server 135, or a cloud computing device 155). All or a portion of the computer program stored on the computer-readable medium may then be stored in a virtual disk 120(1)-(N) and / or a cloud storage device 170. When executed by a processor 855, the computer program loaded into the computing system 800 may cause the processor 855 to perform the functions of one or more of the embodiments described herein and / or may cause the processor to become a device for performing the functions of one or more of the embodiments described herein. Alternatively, one or more of the embodiments described and / or illustrated herein may be implemented in firmware and / or hardware. For example, the computing system 800 may be configured as an application-specific integrated circuit (ASIC) suitable for implementing one or more of the embodiments disclosed herein.
[0078] Exemplary Networking Environment
[0079] Fig. 9 1 is a block diagram of a networked system according to one embodiment, illustrating how various computing devices may communicate via a network. In certain embodiments, a network attached storage (NAS) device may be configured to communicate with virtual machines 105(1)-(N), backup server 135, and / or cloud computing device 155 using a network file system (NFS), server message block (SMB), or common internet file system (CIFS). Network 185 generally represents any type or form of computer network or architecture capable of facilitating communications between virtual machines 105(1)-(N), backup server 135, and / or cloud computing device 155.
[0080] In some embodiments, a communication interface such as Figure 8The communication interface 845 in can be used to provide a connection between the virtual machines 105 (1)-(N), the backup server 135 and / or the cloud computing device 155 and the network 185. It should be noted that the embodiments described and / or illustrated herein are not limited to the Internet or any particular network-based environment. In some embodiments, the network 185 can be a storage area network (SAN). In other embodiments, the backup agent 110 and / or the replication agent 115 can be part of the virtual machine, or can be separate. If separate, the backup agent 110 and / or the replication agent 115 implemented in (e.g., the cloud backup system 905) and the computing device backup agent 110 and / or the replication agent 115 can be communicatively coupled via the network 185.
[0081] In one embodiment, all or part of one or more of the disclosed embodiments may be encoded as a computer program and loaded onto and executed by virtual machines 105(1)-(N), one or more physical machines, backup server 135, and / or cloud computing device 155. All or part of one or more of the embodiments disclosed herein may also be encoded as a computer program, stored on storage system 910, and distributed over network 185.
[0082] In some examples, all or part of the cloud backup system 905, cloud computing device 155, and / or cloud storage device 170 may represent part of a cloud computing or network-based environment. A cloud computing environment may provide various services and applications via the Internet. These cloud-based services (e.g., software as a service, platform as a service, infrastructure as a service, etc.) may be accessed via a web browser or other remote interface. The various functions described herein may be provided via a remote desktop environment or any other cloud-based computing environment.
[0083] In addition, one or more of the components described herein may convert data, physical devices, and / or representations of physical devices from one form to another. For example, backup agent 110 and replication agent 115, or alternatively, DR coordinator 160 and backup agent 165, respectively, may convert the behavior of virtual machines 105(1)-(N) and / or backup server 135 or cloud computing device 155 to perform backup operations using replicas stored in a public or private cloud.
[0084] Although the present disclosure has been described in conjunction with several embodiments, the present disclosure is not intended to be limited to the specific forms set forth herein. On the contrary, the present disclosure is intended to cover such alternatives, modifications and equivalents that can be reasonably included within the scope of the present disclosure defined by the appended claims.
Claims
1. A computer-implemented method comprising: Create a copy of the virtual machine where The virtual machine is located at the primary site. creating the replica at the secondary site as part of the replication operation, and The copy includes Copied data, where The replicated data includes a copy of data associated with the virtual machine, Metadata has been copied, where The copied metadata includes A copy of the metadata, The metadata includes one or more data locations of one or more blocks of the data, and The one or more data locations are configured for use in restoring one or more applications executing on the virtual machine; as well as A backup operation is performed at the secondary site, wherein The performing of the backup operation includes generating a backup image from the copy, and storing the backup image and the replicated metadata at the secondary site, and The backup operation is performed by a backup agent at the secondary site; retrieving granular metadata for the one or more applications, wherein retrieving the granular metadata from the replicated metadata; as well as The granular metadata is sent from the backup agent to a backup server at the primary site.
2. The computer-implemented method of claim 1 , wherein The metadata includes block-level information, The block level information is configured to facilitate granular restoration of the virtual machine, generating the metadata as part of quiescing the one or more applications, and The metadata is stored on one or more virtual disks at the primary site.
3. The computer-implemented method of claim 1 , wherein said generating said backup image further comprises: Generate a snapshot where The snapshot identifies a reference point in time at which the backup image of the replicated data is created, and The reference time point is before the time point of executing the backup operation. retrieving the replicated metadata from the replica, and A full backup image or an incremental backup image is generated based on the reference time point.
4. The computer-implemented method of claim 2, wherein The replicated metadata includes one or more additional data locations of the one or more blocks of the replicated data, and The additional one or more data locations are configured to be used by the backup server to restore the virtual machine from the backup image.
5. The computer-implemented method of claim 1 , wherein instantiating the backup agent on the cloud computing device as part of performing the backup operation, and The backup agent is another virtual machine, a container, or a serverless module.
6. The computer-implemented method of claim 1 , further comprising: quiescing the one or more applications, wherein Quiescing the one or more applications is performed by a replication agent executing on the virtual machine, The metadata is generated by a backup agent executing on the virtual machine, and The replication operation is performed by the replication agent.
7. The computer-implemented method of claim 3, wherein said generating said backup image further comprises: Mount the snapshot, where generating the snapshot of the replicated data at the reference point in time, and The backup image is generated in a backup format.
8. A computer system comprising: one or more processors; a non-transitory computer-readable storage medium communicatively coupled to the one or more processors; Backup agencies, including A creation module is used to create a copy of a virtual machine, wherein The virtual machine is located at the primary site, creating the replica at the secondary site as part of the replication operation, and The copy includes Copied data, where The replicated data includes a copy of data associated with the virtual machine, and Metadata has been copied, where The copied metadata includes A copy of the metadata, The metadata includes one or more data locations of one or more blocks of the data, and The one or more data locations are configured for use in restoring one or more applications executing on the virtual machine; as well as An execution module is used to execute a backup operation, wherein The execution module includes a generation module, which generates a backup image based on the copy, and a storage module for storing the backup image and the replicated metadata at the secondary site, and The backup operation is performed by the backup agent at the secondary site; a retrieval module for retrieving granular metadata for the one or more applications, wherein retrieving the granular metadata from the replicated metadata; as well as A sending module is used to send the granular metadata from the backup agent to the backup server at the primary site.
9. The computer system of claim 8, wherein The metadata includes block-level information, The block level information is configured to facilitate granular restoration of the virtual machine, generating the metadata as part of quiescing the one or more applications, and The metadata is stored on the one or more virtual disks at the primary site.
10. The computer system according to claim 8, wherein the generating module further comprises: A snapshot generation module, which is used to generate a snapshot, wherein The snapshot identifies a reference point in time at which the backup image of the replicated data is created, and The reference time point is before the time point of executing the backup operation. a retrieval module for retrieving the replicated metadata from the replica, and The backup generation module is used to generate a complete backup image or an incremental backup image based on the reference time point.
11. The computer system of claim 9, wherein The replicated metadata includes one or more additional data locations of the one or more blocks of the replicated data, and The additional one or more data locations are configured to be used by the backup server to restore the virtual machine from the backup image.
12. The computer system of claim 8, wherein instantiating the backup agent on the cloud computing device as part of performing the backup operation, and The backup agent is another virtual machine, a container, or a serverless module.
13. The computer system according to claim 8, wherein the computer system further comprises: a quiescing module configured to quiesce the one or more applications, wherein the quiescing of the one or more applications is performed by a replication agent executed on the virtual machine, The replication agent includes the quiesce module, The metadata is generated by a backup agent executing on the virtual machine, and The replication operation is performed by the replication agent.
14. The computer system of claim 10, wherein said generating said backup image further comprises: A snapshot mounting module, which is used to mount the snapshot, wherein The snapshot of the replicated data is generated at the reference point in time, and The backup image is generated in a backup format.
15. A system comprising: one or more processors; and A memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to perform a method comprising: Create a copy of the virtual machine where The virtual machine is located at the primary site, creating the replica at the secondary site as part of the replication operation, and The copy includes Copied data, where The replicated data includes a copy of data associated with the virtual machine, Metadata has been copied, where The copied metadata includes A copy of the metadata, The metadata includes one or more data locations of one or more blocks of the data, and The one or more data locations are configured for use in restoring one or more applications executing on the virtual machine, and Perform a backup operation where The performing of the backup operation includes generating a backup image from the copy, and storing the backup image and the replicated metadata at the secondary site, and The backup operation is performed by a backup agent at the secondary site. retrieving granular metadata for the one or more applications, wherein retrieving the granular metadata from the replicated metadata; and The granular metadata is sent from the backup agent to a backup server at the primary site.
16. The system of claim 15, wherein The metadata includes block-level information, The block level information is configured to facilitate granular restoration of the virtual machine, generating the metadata as part of quiescing the one or more applications, and The metadata is stored on one or more virtual disks at the primary site.
17. The system of claim 15, wherein said generating said backup image further comprises: generating a snapshot, wherein the snapshot identifies a reference point in time at which to create the backup image of the replicated data, and The reference time point is before the time point of executing the backup operation. retrieving the replicated metadata from the replica, and A full backup image or an incremental backup image is generated based on the reference time point.
18. The system of claim 16, wherein The replicated metadata includes one or more additional data locations of the one or more blocks of the replicated data, and The additional one or more data locations are configured to be used by the backup server to restore the virtual machine from the backup image.
19. The system of claim 15, wherein the method further comprises: quiescing the one or more applications, wherein The backup operation is performed by a backup agent. Instantiating the backup agent on the cloud computing device as part of performing the backup operation, The backup agent is another virtual machine, container or serverless module, The quiescing of the one or more applications is performed by a replication agent executing on the virtual machine, The metadata is generated by a backup agent executing on the virtual machine, and The replication operation is performed by the replication agent.
20. The system of claim 17, wherein said generating said backup image further comprises: Mount the snapshot, where generating the snapshot of the replicated data at the reference point in time, and The backup image is generated in a backup format.
