Online Backup Restore Using Virtual Data Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for restoring backup data to a distributed system result in significant data unavailability and downtime due to the time required for copying large amounts of backup data from an external storage system to the cluster, as the backup data is not accessible during the restoration process.
Innovation Solution
A system and method for online restore that generates virtual data files on the cluster nodes, redirecting requests to backup data stored externally, allowing immediate access while the backup data is copied, and subsequently switching to locally stored data once the copy is complete.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If backup data is copied from external storage system to cluster storage devices before restoration, then data can be restored to the cluster, but data unavailability increases and system downtime occurs
Solution Approach 1:
The patent introduces virtual data files as an intermediary between the backup data stored externally and the cluster storage devices. These virtual data files act as a mediator that allows the cluster to access and serve data requests using externally stored backup data without requiring the data to be physically copied to local storage devices first. This resolves the contradiction by enabling data availability during restoration while avoiding the time loss associated with conventional copying operations.
Solution Approach 2:
The patent performs preliminary actions by pre-generating virtual data files that point to externally stored backup data before the actual restoration process begins. This allows the system to prepare the necessary metadata and indexing structures in advance, enabling immediate data availability once restoration is initiated, thereby reducing the overall restoration time and avoiding data unavailability periods.
2Reliability
If backup data is copied to cluster storage devices before restoration, then data can be restored, but system availability decreases during the copying process
Solution Approach 1:
The virtual data files serve as an intermediary layer that enables the cluster to access backup data from external storage systems without blocking system operations during the restoration process. This allows read requests to be served using externally stored data while background copying operations proceed independently, maintaining system availability and improving restoration speed simultaneously.
Solution Approach 2:
The patent enables continuous useful action by allowing the cluster to continue serving data requests using virtual data files that point to externally stored backup data, while the physical copying of data to local storage devices proceeds concurrently in the background. This eliminates the stop-start nature of conventional restoration operations, maintaining system productivity throughout the restoration process.
3Loss of time
If backup data is not copied to cluster storage devices, then data availability is maintained during restoration, but data cannot be restored to the cluster
Solution Approach 1:
The virtual data files act as a temporary intermediary that bridges the gap between external backup storage and the cluster. They allow the cluster to access and restore data without requiring the data to be physically present on local storage devices, thereby reducing data unavailability duration while ensuring complete data restoration through the copying process.
Solution Approach 2:
The system performs preliminary action by creating virtual data files that reference externally stored backup data before the actual restoration begins. This preliminary setup allows immediate data availability and enables the restoration process to proceed without waiting for physical data copying, reducing unavailability duration while maintaining restoration completeness.
Data Source
AI summary
Systems and methods for enabling availability of backup data during restoration of a distributed database system are provided. An external storage system stores a plurality of backup files each comprising backup data and one or more backup metadata files. A plurality of ranges are generated and stored among a plurality of computing devices based on the backup files and metadata files. A plurality of virtual data files are generated and stored in the ranges, where each range of the ranges comprises one or more of the virtual data files, where the virtual data files for a respective range identify physical data configured to be stored by the respective range comprising at least a portion of the backup data of one or more of the backup files. The ranges are configured to serve, based on the virtual data files, a request directed to reading or writing the physical data.