System and method for backing up an eventually consistent database in a production cluster

Abstract

The disclosed computer-implemented method for backing up an eventually consistent database in a production cluster may include: (1) forming a stable copy of production data on a production node; (2) configuring storage on a backup node based on the amount of data in the stable copy and the replication factor; (3) transferring information from the stable copy to a backup copy on the backup node; (4) performing record synthesis on the backup copy to merge record updates into a complete backup record; (5) identifying and discarding any obsolete records and any redundant records in the complete backup record; and (6) transferring the complete backup record from the backup node to a cloud storage device. Various other methods, systems, and computer-readable media are also disclosed.

Description

Technical Field

[0001] The present invention relates to a system and method for backing up an eventually consistent database in a production cluster. Background Art

[0002] In recent years, an increasing number of horizontally scalable high-performance databases running on commodity hardware have been produced, such as Apache CASSANDRA. Typical production deployments of these databases may involve a large number of nodes, such as 64 nodes or more. One of the largest production deployments of a high-performance database is the Apple database, which has more than 75,000 nodes storing more than 10PB of data. Other large CASSANDRA installations include Netflix (2,500 nodes, 420TB), the Chinese search engine Easou (270 nodes, 300TB), and eBay (more than 100 nodes, 250TB).

[0003] There are many challenges in backing up a high-performance database. Some considerations include:

[0004] Data size: High-performance databases often have huge data sets (often on the order of multiple terabytes). These large data sets must be protected in a short time span.

[0005] Replica coordination: Systems that achieve eventual consistency may have problems determining points in time due to the lack of an authoritative replica.

[0006] Deduplication Challenges: The storage formats of these systems range from file-level replicas to record-level replicas. Significant resources are required to replicate all the replicas resulting from the replication strategy. Conventional deduplication methods may not work.

[0007] Topological Differences: Next-generation hardware architectures offer better compute and storage stacks. Effectively leveraging these architectures requires topological changes in scale-out high-performance database systems.

[0008] Reconfigure fault-tolerance protection: For horizontally scaled high-performance database systems, nodes joining and leaving the cluster are common scenarios.

[0009] Garbage Collection: To achieve maximum performance, modern databases use log-structured storage formats that are immutable and / or append-only. These formats are never overwritten in place, so the data files continue to grow in size, but there is no way to access previous versions of the data.

[0010] Accordingly, the present disclosure identifies and addresses the need for systems and methods for backing up an eventually consistent database in a production cluster. Summary of the invention

[0011] As will be described in greater detail below, this disclosure describes various systems and methods for backing up an eventually consistent database in a production cluster.

[0012] In one embodiment, a method for backing up an eventually consistent database in a production cluster may include: (1) forming a stable copy of production data on a production node; (2) configuring storage on a backup node based on the amount of data in the stable copy and the replication factor; (3) transferring information from the stable copy to a backup copy on the backup node; (4) performing record synthesis on the backup copy to merge record updates into a full backup record; (5) identifying and discarding any obsolete records and any redundant records in the full backup record; and (6) transferring the full backup record from the backup node to a cloud storage device.

[0013] In an embodiment, the method may include identifying a topology of a production cluster of which the production node is a component to identify the production node as requiring backup. In one example, the method may include determining the amount of data in the stable replica. In an example, the method may include configuring a backup node in the backup cluster based on the amount of data in the stable replica.

[0014] In an embodiment, the method may include restoring the backup node to a pre-transfer state. In some examples, the number of production nodes in a production cluster of which the production node is a component is not equal to the number of backup nodes in a backup cluster of which the backup node is a component. In an example, the method may include restoring the backup copy from the cloud storage device to the production node.

[0015] In one example, a system for backing up an eventually consistent database in a production cluster may include several modules stored in a memory, including (1) a formation module stored in the memory, the formation module forming a stable copy of production data on a production node; (2) a configuration module stored in the memory, the configuration module configuring storage on a backup node based on the amount of data in the stable copy and the replication factor; (3) a first transmission module stored in the memory, the first transmission module transmitting information from the stable copy to the backup copy on the backup node; (4) an execution module stored in the memory, the execution module performing record synthesis on the backup copy to merge record updates into a complete backup record; (5) an identification and discard module stored in the memory, the identification and discard module identifying and discarding any stale records and any redundant records in the complete backup record; and (6) a second transmission module stored in the memory, the second transmission module transmitting the complete backup record from the backup node to a cloud storage device. The system may also include at least one physical processor, the at least one physical processor executing the formation module, the configuration module, the first transmission module, the execution module, the identification and discard module, and the second transmission module.

[0016] In some examples, the above method may be encoded as computer-readable instructions on a non-transitory computer-readable medium. For example, the computer-readable medium may include one or more computer-executable instructions that, when executed by at least one processor of the computing device, may cause the computing device to (1) form a stable copy of the production data on the production node, (2) configure storage on the backup node based on the amount of data in the stable copy and the replication factor, (3) transfer information from the stable copy to the backup copy on the backup node, (4) perform record synthesis on the backup copy to merge record updates into the full backup record, (5) identify and discard any obsolete records and any redundant records in the full backup record, and (6) transfer the full backup record from the backup node to the cloud storage device.

[0017] Features from any of the above embodiments may be used in combination with each other according to the general principles described herein. These and other embodiments, features and advantages will be more fully understood after reading the following detailed description in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings illustrate several exemplary embodiments and are a part of the specification. Together with the following description, these drawings demonstrate and explain various principles of the present disclosure.

[0019] Figure 1 is a block diagram of an exemplary system for backing up an eventually consistent database in a production cluster.

[0020] Figure 2 is a block diagram of an additional exemplary system for backing up an eventually consistent database in a production cluster.

[0021] Figure 3 is a flow chart of an exemplary method for backing up an eventually consistent database in a production cluster.

[0022] Figure 4 is a diagram showing an example of a rebase operation.

[0023] Throughout the drawings, the same reference characters and descriptions indicate similar but not necessarily identical elements. Although the exemplary embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments have been shown in the drawings by way of example and will be described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the specific forms disclosed. On the contrary, the present disclosure encompasses all modifications, equivalents, and alternative forms that fall within the scope of the appended claims. DETAILED DESCRIPTION

[0024] The present disclosure as a whole relates to systems and methods for backing up an eventually consistent database in a production cluster. The provided systems and methods can perform backup and / or recovery of an eventually consistent scale-out database (e.g., such as CASSANDRA) in a staged manner, thereby enabling recovery without repair, replica removal, and record synthesis. In some examples, the systems and methods described herein can form a stable copy of production data on a production node; transfer information from the stable copy to a backup copy on a backup node; merge record updates into a complete backup record while discarding stale and redundant records; and transfer the complete backup record from the backup node to a cloud storage device. In an embodiment, the disclosed technology can be used in conjunction with a cloud-based storage device.

[0025] Thus, in an example, the systems and methods described herein can improve the operation of computing devices by automatically protecting large data sets in a short time span, providing authoritative copies, addressing deduplication challenges, backing up records stored in changing architectures, backing up records of reconfigured nodes, and / or backing up data files that increase in size over time, thereby achieving cost-effective storage management. Additionally, in an example, the systems and methods described herein can also save power and / or better manage network bandwidth utilization.

[0026] The following will refer to Figure 1 to Figure 2 Provides a detailed description of an exemplary system for backing up an eventually consistent database in a production cluster. Figure 3 to Figure 4 A detailed description of the corresponding computer-implemented method is provided.

[0027] Figure 1 1 is a block diagram of an exemplary system 100 for backing up an eventually consistent database in a production cluster. As shown in the figure, the exemplary system 100 may include one or more modules 102 for performing one or more tasks. As will be explained in more detail below, the modules 102 may include a formation module 104, a configuration module 106, a first transmission module 108, an execution module 110, an identification and discard module 112, and / or a second transmission module 114. Although shown as separate elements, Figure 1 One or more of the modules 102 may represent portions of a single module or an application.

[0028] In certain embodiments, Figure 1 One or more of the modules 102 in the example embodiment may represent one or more software applications or programs that, when executed by a computing device, may cause the computing device to perform one or more tasks. For example, and as will be described in more detail below, one or more of the modules 102 may represent modules stored and configured to run on one or more computing devices, such as a Figure 2206, production node 208, backup node 212, and / or cloud storage device 224). Figure 1 One or more of the modules 102 in may also represent all or part of one or more special-purpose computers configured to perform one or more tasks.

[0029] like Figure 1 As shown, the exemplary system 100 may also include one or more storage devices, such as storage device 120. Storage device 120 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, storage device 120 may store, load and / or maintain information indicating one or more of the amount of data 122 and / or replication factor 124 in a stable copy. In one example, storage device 120 may store, load and / or maintain information indicating one or more of production data, a stable copy 210 of production data, a backup copy 214, a record update 216, a complete backup record 218, a redundant record 220 and / or a stale record 222. Examples of storage device 120 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, a hard disk drive (HDD), a solid-state drive (SSD), an optical drive, a cache, an AMAZON ELASTIC BLOCK STORE service, a variant or combination of one or more of the above, and / or any other suitable memory.

[0030] As used herein, the term "replication factor" generally refers to an integer number of copies of data stored across a cluster of nodes. For example, but not limited to, when the replication factor of a cluster is equal to 2 and data is written to the cluster, two copies of the data are stored within the cluster (e.g., in different nodes and / or racks). A replication factor greater than 1 can provide fault tolerance.

[0031] like Figure 1As shown, the exemplary system 100 may also include one or more physical processors, such as physical processor 130. Physical processor 130 generally represents a hardware-implemented processing unit of any type or form capable of interpreting and / or executing computer-readable instructions. In one example, physical processor 130 may access and / or modify one or more of modules 102 stored in memory 140. Additionally or alternatively, physical processor 130 may execute one or more of modules 102 to facilitate backing up an eventually consistent database in a production cluster. Examples of physical processor 130 include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a portion of one or more of the foregoing, a variant or combination of one or more of the foregoing, or any other suitable physical processor.

[0032] like Figure 1 As shown, the exemplary system 100 may also include one or more memory devices, such as memory 140. Memory 140 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, memory 140 may store, load, and / or maintain one or more of modules 102. Examples of memory 140 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical drive, cache, variations or combinations of one or more of the foregoing, or any other suitable memory.

[0033] Figure 1 The exemplary system 100 in FIG. 1 may be implemented in various ways. For example, all or part of the exemplary system 100 may represent Figure 2 As shown in FIG. Figure 2 As shown, system 200 may include computing device 202 in communication with server 206, production node 208, backup node 212, and / or cloud storage device 224 (e.g., via network 204). In one example, all or part of the functionality of module 102 may be performed by computing device 202, server 206, production node 208, backup node 212, cloud storage device 224, and / or any other suitable computing system. In the example, all or part of exemplary system 100 may be a component of computing device 202, server 206, production node 208, production cluster, backup node 212, backup cluster, cloud storage device 224, and / or any other suitable computing system. As will be described in more detail below, the components from Figure 1One or more of modules 102 of may enable computing device 202, server 206, production node 208, backup node 212, and / or cloud storage device 224 to back up an eventually consistent database in a production cluster when executed by at least one processor of computing device 202, server 206, production node 208, backup node 212, and / or cloud storage device 224. In an example, production node 208 may be a component of a production cluster. In an embodiment, backup node 212 may be a component of a backup cluster such as a CASSANDRA cluster.

[0034] Computing device 202 generally represents any type or form of computing device capable of reading computer-executable instructions. In some examples, computing device 202 may represent a computer running storage management software. Additional examples of computing device 202 include, but are not limited to, laptop computers, tablet computers, desktop computers, servers, cellular phones, personal digital assistants (PDAs), multimedia players, embedded systems, wearable devices (e.g., smart watches, smart glasses, etc.), smart vehicles, so-called Internet of Things devices (e.g., smart appliances, etc.), game consoles, variations and combinations of one or more of the above, or any other suitable computing system.

[0035] The network 204 generally represents any medium or architecture capable of facilitating communication or data transfer. In one example, the network 204 can facilitate communication between the computing device 202 and the server 206. In this example, the network 204 can facilitate communication or data transfer using wireless and / or wired connections. Examples of the network 204 include, but are not limited to, an intranet, a wide area network (WAN), a local area network (LAN), a personal area network (PAN), the Internet, a power line communication (PLC), a cellular network (e.g., a global system for mobile communications (GSM) network), portions of one or more of the foregoing, variations or combinations of one or more of the foregoing, or any other suitable network.

[0036] Server 206 generally represents any type or form of computing device capable of reading computer-executable instructions. In some examples, computing device 206 may represent a computer running storage management software. Additional examples of server 206 include, but are not limited to, storage servers, database servers, application servers, and / or web servers, which are configured to run certain software applications and / or provide various storage, database, and / or web services. Figure 2 Although shown as a single entity, server 206 may include and / or represent multiple servers that work and / or operate in conjunction with each other.

[0037] Production nodes 208 generally represent any type or form of computing device capable of reading computer-executable instructions and / or storing information. In some examples, production nodes 208 may represent computer-operated storage software. Additional examples of production nodes 208 include, but are not limited to, storage servers, database servers, application servers, and / or web servers that are configured to run certain software applications and / or provide various storage, database, and / or web services. Figure 2 Although shown as a single entity, production node 208 may include and / or represent multiple servers that work and / or operate in coordination with each other. In an example, production node 208 may be a component of a production cluster. In an embodiment, production node 208 may be part of a node cluster that stores information for an APACHE CASSANDRA database.

[0038] In an example, production node 208 may store stable copy 210 of production information and / or production data.

[0039] Backup node 212 generally represents any type or form of computing device capable of reading computer-executable instructions and / or storing information. In some examples, backup node 212 may represent storage software running on a computer. Additional examples of backup node 212 include, but are not limited to, storage servers, database servers, application servers, and / or web servers that are configured to run certain software applications and / or provide various storage, database, and / or web services. Figure 2 Although shown as a single entity in FIG. 1 , backup node 212 may include and / or represent multiple servers that work and / or operate in coordination with each other. In an embodiment, backup node 212 may be a component of a backup cluster.

[0040] In an example, backup node 212 may store backup copies 214 , record updates 216 , full backup records 218 , redundant records 220 , and / or stale records 222 .

[0041] Cloud storage device 224 generally represents any type or form of computing device capable of reading computer-executable instructions and / or storing information. In some examples, cloud storage device 224 may represent storage management software running on a computer. Additional examples of cloud storage device 224 include, but are not limited to, storage servers, database servers, application servers, and / or web servers that are configured to run certain software applications and / or provide various storage, database, and / or web services. In an example, cloud storage device 224 may be provided by Amazon Simple Storage Service (AMAZON SIMPLE STORAGE SERVICE) (S3). Although in Figure 2Although shown as a single entity, cloud storage device 224 may include and / or represent multiple servers working and / or operating in conjunction with each other.

[0042] In an example, cloud storage 224 may store backup copy 214 and / or full backup record 218 .

[0043] Many other devices or subsystems can be connected to Figure 1 The system 100 and / or Figure 2 In contrast, no Figure 1 and Figure 2 All components and devices shown in the present embodiment are present, and the embodiments described and / or shown in this article can also be practiced. The above-mentioned devices and subsystems can also be used in accordance with Figure 2 The systems 100 and 200 may also be interconnected in different ways as shown. Any number of software configurations, firmware configurations, and / or hardware configurations may also be employed. For example, one or more of the exemplary embodiments disclosed herein may be encoded as a computer program (also referred to as computer software, software application, computer-readable instructions, and / or computer control logic) on a computer-readable medium.

[0044] As used herein, the term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media, such as carrier waves, and non-transitory media, such as magnetic storage media (e.g., hard drives, tape drives, and floppy disks), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), BLU-RAY disks), electronic storage media (e.g., solid-state drives and flash memory media), and other distribution systems.

[0045] Figure 3 is a flow chart of an exemplary computer-implemented method 300 for backing up an eventually consistent database in a production cluster. Figure 3 The steps shown in may be performed by any suitable computer executable code and / or computing system, including Figure 1 System 100 in Figure 2 In one example, the system 200 and / or any variation or combination of any one or more of the foregoing. Figure 3 Each step shown in may represent an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which are provided in more detail below.

[0046] like Figure 3As shown, at step 302, one or more of the systems described herein may form a stable copy of the production data on the production node. Forming a stable production copy may provide an independent instance of the production data, thereby performing a rebase operation and / or expiring a backup made before the rebase time point to reclaim storage space. The systems described herein may perform step 302 in a variety of ways. For example, as Figure 2 As part of the computing device 202 in the production node 208, the module 104 forms a stable copy 210 of the production data on the production node 208. The production stable copy establishes a copy of the production data that changes rapidly at the corresponding time instance.

[0047] In some embodiments, method 300 may include identifying a topology of a production cluster of which the production node is a component to identify the production node as requiring backup. For example, system 100 and / or computing device 202 may identify

[0048] The topology of the production cluster of which the production node 208 is a component. The system 100 and / or computing device 202 may also identify at least one production cluster and / or production node (eg, the production node 208) as requiring backup.

[0049] In some embodiments, method 300 may include determining the amount of data in the stable copy, such as for use in step 304. For example, system 100 and / or computing device 202 may determine the amount of data in stable copy 210 of production data.

[0050] like Figure 3 As shown, at step 304, one or more of the systems described herein may configure storage on the backup node based on the amount of data in the stable replica and / or the replication factor. The replication factor may be the replication factor of the key space participating in the backup. The systems described herein may perform step 304 in a variety of ways. For example, as Figure 2 As part of computing device 202 in , configuration module 106 can configure storage on backup node 212 based on amount of data in stable copy 122 and / or replication factor 124. Amount of data in stable copy 122 can be an amount of data in stable copy 210 of production data.

[0051] In one example, method 300 may include configuring backup nodes in a backup cluster based on the amount of data in a stable copy (i.e., the amount of data to be backed up) and / or the processing power of the backup nodes. For example, system 100 and / or computing device 202 may configure backup node 212 based on the amount of data 122 in a stable copy and / or the processing power of backup node 212.

[0052] In an example, the number of production nodes in a production cluster of which the production node is a component may not be equal to the number of backup nodes in a backup cluster of which the backup node is a component. In some embodiments, the number of production nodes in a production cluster of which the production node is a component may be equal to the number of backup nodes in a backup cluster of which the backup node is a component.

[0053] like Figure 3 As shown, at step 306, one or more of the systems described herein may transfer information from the stable copy to the backup copy on the backup node. The systems described herein may perform step 306 in a variety of ways. For example, as Figure 2 As part of the computing device 202 in the production cluster, the first transmission module 108 can transmit information from the stable copy 210 of the production data to the backup copy 214 on the backup node 212. In an example, data from different stable backup copies (i.e., replicas) can be transmitted in parallel from different nodes in the production cluster to the same node in the backup cluster.

[0054] like Figure 3 As shown, at step 308, one or more of the systems described herein may optimize the record by performing record synthesis on the backup copy to merge the record updates into the complete backup record. The record synthesis may merge the updates to different columns at different times to recreate the complete record. Step 308 may be performed locally at the backup node to process the backup copies in parallel and / or without substantially inter-node communication. The systems described herein may perform step 308 in a variety of ways. For example, as Figure 2 As part of computing device 202 in , execution module 110 may perform record synthesis on backup copy 214 to merge record updates 216 into full backup record 218 .

[0055] like Figure 3 As shown, at step 310, one or more of the systems described herein may optimize the records by identifying and / or discarding at least one obsolete record and / or at least one redundant record in the full backup record and / or the backup copy. The systems described herein may perform step 310 in a variety of ways. For example, as Figure 2 As part of the computing device 202 in the backup copy 214, the identification and discarding module 112 can identify and / or discard at least one stale record 222 and / or at least one redundant record 220 in the full backup record 218 and / or the backup copy 214. Deduplication can provide consistent data to reduce the probability that CASSANDRA will automatically perform repairs to repair inconsistencies in the restored backup.

[0056] In an example, method 300 may include optimizing the record to reclaim storage space. A common problem with forever incremental backup strategies is reclaiming space due to backup expiration. In an embodiment, the provided system and method may rebase a backup by creating a synthetic full backup from an independent instance of a production cluster. This may effectively create an independent chain, expire backups prior to the rebase point, and reclaim storage space previously used by expired backups. For example, method 300 may rebase at least a portion of backup copy 214 to form a complete backup record 218.

[0057] In an example, method 300 may perform rebasing on backup copies in backup nodes instead of on data on production nodes. In one example, backup copies may be distributed between different backup nodes, so each backup node may perform rebasing operations independently. Rebasing may be performed during backup expiration to delete backups and restore associated storage space and / or to achieve faster recovery. Rebasing may be performed periodically (e.g., to consistently reduce the number of incremental backups). Overwritten data may be removed to produce a smaller resulting data set. After the rebasing operation, subsequent backups may be reset to parent to create a new backup chain. In an example, method 300 may delete backup copies before the rebasing point and reclaim storage space previously used by the deleted backup.

[0058] Figure 4 An example of a rebase operation 400 is depicted. Figure 4 In the diagram, “F” represents a full backup, “I” represents an incremental backup, “R” represents a rebase backup, and arrows represent the dependency of a backup on a previous backup (e.g., an incremental backup depends on a full backup).

[0059] At time 402, full backup F1 has six incremental backups (I11, I12, I13, I21, I22, I23) dependent upon it.

[0060] At time 404, a rebase operation creates a rebased backup R2 from the full backup F1 and the incremental backups I11, I12, and I13.

[0061] At time 406, incremental backup I21 is reparented to I13 through R2.

[0062] At time 408, full backup F1 and incremental backups I11, I12, and I13 may expire. Figure 4 In the example, any one of the full backup F1 and the incremental backups I11, I12, and I13 may be deleted to save storage space.

[0063] At time 410, a rebase operation creates a rebased backup R3 from the rebased backup R2 and incremental backups I21, I22, and I23.

[0064] At time 412, the rebased backup R2 and the incremental backups I21, I22, and I23 may expire, such as Figure 4 In the example, any one of the rebased backup R2 and the incremental backups I21, I22, and I23 may be deleted to save storage space.

[0065] At time 414, incremental backup I31 is created and set as the parent of rebased backup R3.

[0066] Return to Figure 3 At step 312, one or more of the systems described herein may transfer (i.e., upload) the complete backup record from the backup node to the cloud storage device. The backup data may be transferred in parallel from the backup nodes in the backup cluster to the backup medium. The systems described herein may perform step 312 in a variety of ways. For example, as Figure 2 As part of the computing device 202 in the second transfer module 114, the second transfer module 114 can transfer the complete backup record 218 from the backup node 212 to the cloud storage device 224.

[0067] In one example, method 300 can include reverting the backup node to a pre-transfer state. For example, at least a portion of backup copy 214 can be deleted from backup node 212.

[0068] In one embodiment, method 300 may include restoring a backup copy from a cloud storage device to a production node. For example, at least a portion of backup copy 214 and / or at least a portion of full backup record 218 may be restored from cloud storage device 224 to computing device 202, server 206, production node 208, backup node 212, a different production node, and / or a device coupled to network 204.

[0069] In one embodiment, recovery may include determining the amount of data to be processed from the backup copy. The keyspaces that need to be recovered may be identified and a chain of incremental backups may be identified until a full backup is identified for the keyspace. Recovery may include calculating the amount of data that needs to be transferred for each keyspace to be recovered. In an embodiment, recovery may include determining the amount of data to be processed from the backup copy on the cloud storage device 224. Recovery may include calculating the amount of data that needs to be transferred from the cloud storage device 224 for each keyspace to be recovered.

[0070] In one example, recovery may include preparing a backup cluster. After the amount of data to be recovered has been identified, the nodes in the backup cluster may be configured and the storage on the backup nodes in the backup cluster may be configured. The number of nodes that need to be configured may depend on the amount of data that needs to be processed and / or the processing power of the backup nodes. Recovery may include assigning the responsibility for processing separate column families to the nodes in the backup cluster so that no two nodes process the same column family and / or the data is evenly distributed among each node. For example, recovery may include preparing a backup cluster (including backup node 212) and / or configuring storage on backup node 212.

[0071] In an example, recovery may include downloading data from a backup medium (eg, a cloud storage device) to each node in the backup cluster in parallel. For example, recovery may include downloading backup copy 214 from cloud storage device 224 to backup node 212.

[0072] In one embodiment, recovery may optionally include performing record synthesis that merges updates to different columns at different times to recreate the complete record. Storage space savings may be obtained by performing record synthesis, such as when restoring from an incremental backup. For example, recovery may include performing record synthesis that merges record updates 216 to different columns at different times to recreate the complete backup record 218. Optimizing backup data in this manner may reduce the amount of data to be restored.

[0073] In one embodiment, recovery may include preparing the production cluster. Recovery may include creating a schema for the keyspace that needs to be recovered and / or preparing the production cluster to receive data from the backup cluster. For example, recovery may include preparing the production cluster (including production node 208) to receive data from backup node 212.

[0074] In an example, recovery may include transferring data. Data may be transferred (e.g., dispersed) in parallel from backup nodes in the backup cluster to production nodes in the production cluster. Recovery may include multiplying write records by production nodes that vary by replication factor. In an example, recovery may include transferring data from backup node 212 to production node 208.

[0075] In an embodiment, recovery may include removing temporary data structures on backup nodes in a backup cluster. In an example, recovery may include restoring the backup node to a pre-transmission state. For example, recovery may include restoring the backup node 212 to a pre-transmission state.

[0076] As detailed above, Figure 3The steps outlined in method 300 in can provide a method for backing up an eventually consistent database in a production cluster and / or restoring an eventually consistent database from a cloud storage device. In an example, the provided system and method can be used with a scale-out high-performance database. Thus, in an example, the systems and methods described herein can improve the operation of a computing device by automatically protecting large data sets in a short time span, providing authoritative copies, addressing deduplication challenges, backing up records stored in a changing architecture, backing up records of reconfigured nodes, and / or backing up data files that increase in size over time, thereby achieving cost-effective storage management. In addition, in an example, the systems and methods described herein can also save power by reducing the amount of data to be transmitted.

[0077] Although the above disclosure uses specific block diagrams, flow charts, and examples to illustrate various embodiments, each block diagram component, flow chart step, operation, and / or component described and / or illustrated herein may be implemented individually and / or collectively using a variety of hardware, software, or firmware (or any combination thereof) configurations. In addition, any disclosure of components contained within other components should be considered exemplary in nature, as many other architectures may be implemented to achieve the same functionality.

[0078] In some examples, Figure 1 All or a portion of the exemplary system 100 in the example embodiment may represent a portion of a cloud computing or network-based environment. A cloud computing environment may provide various services and applications via the Internet. These cloud-based services (e.g., software as a service, platform as a service, infrastructure as a service, etc.) may be accessed through a web browser or other remote interface. Various functions described herein may be provided through a remote desktop environment or any other cloud-based computing environment.

[0079] In various embodiments, Figure 1 All or a portion of the exemplary system 100 in can facilitate multi-tenancy within a cloud-based computing environment. In other words, the modules described herein can configure a computing system (e.g., a server) to facilitate multi-tenancy of one or more of the functions described herein. For example, one or more of the modules described herein can program a server to allow two or more clients (e.g., customers) to share an application running on the server. A server programmed in this manner can share applications, operating systems, processing systems, and / or storage systems between multiple customers (i.e., tenants). One or more of the modules described herein can also partition the data and / or configuration information of a multi-tenant application for each customer so that one customer cannot access the data and / or configuration information of another customer.

[0080] According to various embodiments, Figure 1All or a portion of the exemplary system 100 in the embodiment of the present invention may be implemented in a virtual environment. For example, the modules and / or data described herein may reside and / or execute in a virtual machine. As used herein, the term "virtual machine" generally refers to any operating system environment abstracted from computing hardware by a virtual machine manager (e.g., a hypervisor).

[0081] In some examples, Figure 1 All or a portion of the exemplary system 100 in the example embodiment may represent a portion of a mobile computing environment. The mobile computing environment may be implemented by a variety of mobile computing devices, including mobile phones, tablet computers, e-book readers, personal digital assistants, wearable computing devices (e.g., computing devices with head-mounted displays, smart watches, etc.), variations or combinations of one or more of the foregoing, or any other suitable mobile computing devices. In some examples, the mobile computing environment may have one or more notable features, including, for example, reliance on battery power, presenting only one foreground application at any given time, remote management features, touch screen features, location and movement data (e.g., provided by a global positioning system, a gyroscope, an accelerometer, etc.), a restricted platform for limiting modifications to system-level configurations and / or limiting the ability of third-party software to inspect the behavior of other applications, controls for limiting the installation of applications (e.g., only applications from approved application stores), and the like. The various functions described herein may be provided for and / or may interact with the mobile computing environment.

[0082] The process parameters and step sequences described and / or illustrated herein are given by way of example only and may be changed as desired. For example, although the steps illustrated and / or described herein may be shown or discussed in a particular order, the steps need not be performed in the order illustrated or discussed. The various example methods described and / or illustrated herein may also omit one or more of the steps described or illustrated herein, or include additional steps in addition to those disclosed.

[0083] Although various embodiments have been described and / or shown herein in the context of a fully functional computing system, one or more of these exemplary embodiments may be distributed as various forms of program products, regardless of the specific type of computer-readable medium used to actually distribute. Embodiments disclosed herein may also be implemented using modules that perform certain tasks. These modules may include script files, batch files, or other executable files that may be stored on a computer-readable storage medium or in a computing system. In some embodiments, these modules may configure a computing system to perform one or more of the exemplary embodiments disclosed herein.

[0084] The foregoing description is intended to enable others skilled in the art to best utilize the various aspects of the example embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limited to any exact form disclosed. Many modifications and variations may be made without departing from the spirit and scope of the present disclosure. The embodiments disclosed herein should be considered exemplary and non-restrictive in all respects. The scope of the present disclosure should be determined with reference to the appended claims and their equivalents.

[0085] Unless otherwise indicated, the terms "connected to" and "coupled to" (and their derivative forms) used in this specification and claims should be understood to allow both direct and indirect (i.e., via other elements or components) connections. The singular implies the plural where feasible. In addition, the terms "a" or "an" used in this specification and claims should be understood to mean "at least one ...". Finally, for ease of use, the terms "including" and "having" (and their derivative forms) used in this specification and claims are interchangeable with the word "comprising" and have the same meaning as the word "comprising".

Claims

1. A computer-implemented method for backing up an eventually consistent database in a production cluster, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising: Form a stable copy of production data on production nodes; configuring storage on a backup node based on the amount of data in the stable replica and the replication factor; transferring information from the stable copy to a backup copy on the backup node; performing record synthesis on the backup copy to merge record updates into a full backup record, the performing record synthesis comprising performing a rebase operation to create a synthetic full backup from a previous full backup and one or more incremental backups of the backup copy; Reparent incremental backups to rebase backups from incremental backups; Identifying and discarding any obsolete records and any redundant records in the full backup records; and The complete backup record is transmitted from the backup node to a cloud storage device.

2. The computer-implemented method of claim 1 , further comprising: A topology of a production cluster of which the production node is a component is identified to identify the production node as requiring backup.

3. The computer-implemented method of claim 1 or claim 2, further comprising: The amount of data in the stable copy is determined.

4. The computer-implemented method of claim 1 or claim 2, further comprising: The backup nodes in a backup cluster are configured based on the amount of data in the stable replica.

5. The computer-implemented method of claim 1 or claim 2, further comprising: The backup node is restored to a pre-transmission state.

6. The computer-implemented method of claim 1 or claim 2, wherein the number of production nodes in a production cluster of which the production node is a component is not equal to the number of backup nodes in a backup cluster of which the backup node is a component.

7. The computer-implemented method of claim 1 or claim 2, further comprising: The backup copy is restored from the cloud storage device to the production node.

8. A system for backing up an eventually consistent database in a production cluster, the system comprising: means for forming stored in the memory, the means for forming forming a stable copy of the production data on the production node; first means for configuring stored in the memory, the first means for configuring configuring storage on a backup node based on the amount of data in the stable replica and the replication factor; a first means for transmitting stored in the memory, the first means for transmitting transmitting information from the stable copy to the backup copy on the backup node; means for performing, stored in the memory, the means for performing performing record synthesis on the backup copy to merge record updates into a full backup record, and the means for performing further reparenting incremental backups from incremental backups to rebased backups, wherein the performing record synthesis includes performing a rebase operation to create a synthetic full backup from a previous full backup and one or more incremental backups of the backup copy; means for identifying and discarding stored in the memory, the means for identifying and discarding identifying and discarding any obsolete records and any redundant records in the complete backup records; At least one physical processor that executes the means for forming, the means for configuring, the first means for transmitting, the means for executing, and the means for identifying and discarding.

9. The system according to claim 8, further comprising: Means for identifying stored in the memory, the means for identifying identifying a topology of a production cluster of which the production node is a component to identify the production node as requiring backup.

10. The system according to claim 8 or claim 9, further comprising: Means for determining, stored in the memory, the means for determining determining the amount of data in the stable copy.

11. The system according to claim 8 or claim 9, further comprising: Second means for configuring stored in the memory, the second means for configuring configuring the backup node in a backup cluster based on the amount of data in the stable copy.

12. The system according to claim 8 or claim 9, further comprising: Means for restoring stored in the memory, the means for restoring causing the backup node to restore to a pre-transmission state.

13. The system according to claim 8 or claim 9, wherein the number of production nodes in a production cluster of which the production node is a component is not equal to the number of backup nodes in a backup cluster of which the backup node is a component.

14. The system according to claim 8 or claim 9, further comprising: Means for restoring stored in the memory, the means for restoring causing the backup copy to be restored from the cloud storage device to the production node.

Images