Non-blocking Database Backup via Parallel Write and Snapshot Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data backup methods require additional hardware and resources, leading to increased costs and reduced flexibility, as they necessitate allocating extra machines and disk space outside the established replica set, and often result in higher backup latency due to sequential processing of writes and backups.
Innovation Solution
A database system that performs write commands and backup commands in parallel, utilizing snapshots of data from secondary storage nodes without requiring additional hardware, thereby reducing resource allocation and improving backup efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If additional hardware is allocated for data backups outside the replica set, then backup capability is improved, but resource costs and device complexity increase
Solution Approach 1:
Existing secondary storage nodes in the replica set are made to serve dual purposes: both data replication and backup storage. The backup system utilizes the same hardware infrastructure already allocated for replication, making the existing nodes universal resources that fulfill multiple functions without requiring additional dedicated backup hardware.
Solution Approach 2:
The backup system is designed to automatically utilize available secondary nodes within the replica set without requiring external hardware allocation. The system self-manages backup operations by directing backup commands to appropriate secondary nodes that are already part of the replication infrastructure, eliminating the need for separate backup hardware provisioning.
2Device complexity
If sequential processing of writes and backups is performed, then resource allocation is simplified, but backup latency increases
Solution Approach 1:
The system captures snapshots of data at secondary nodes before backup operations commence. This preliminary snapshot creation establishes a consistent backup point in time without blocking subsequent write operations, allowing the backup process to work from a fixed snapshot while the database continues to accept and process writes independently.
Solution Approach 2:
The backup system dynamically identifies and utilizes available secondary nodes for backup operations while maintaining ongoing replication activities. The system adapts to the current state of the replica set, selecting appropriate secondary nodes based on their availability and capacity, thereby enabling parallel processing of writes and backups without rigid sequential constraints.
3Quantity of substance
If additional machines and disk space are allocated for backups, then backup capacity is improved, but resource costs increase
Solution Approach 1:
The backup storage capacity is merged with the existing replication infrastructure. Secondary nodes that already possess disk space allocated for maintaining replica copies are simultaneously utilized for backup storage. This consolidation allows the system to achieve enhanced backup capacity without proportionally increasing hardware resources, as the same disk space serves both replication and backup functions.
Solution Approach 2:
The system creates data copies for backup purposes using the existing secondary nodes within the replica set. Rather than allocating separate dedicated backup hardware, the system leverages the copying capability of secondary nodes that already maintain replicated copies of data, thereby achieving backup capacity using existing computational resources.
Data Source
AI summary
A database system comprising a database comprising a plurality of data storage nodes, the plurality of data storage nodes including a primary storage node and a plurality of secondary storage nodes. The database system may also include at least one processor configured to: perform at least one write command to a secondary storage node of the plurality of secondary storage nodes; and perform at least one backup command, at least partially in parallel with the at least one write command being performed to the secondary storage node, of at least some data from a snapshot of data stored in the secondary storage node. A method for performing at least one backup command in a database at least partially in parallel with the at least one write command being performed.