Online Database Restore Across Topologies with Custom Data Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face challenges in efficiently resizing resources to match changing workloads without incurring significant downtime, as existing restore techniques often require databases to be unavailable during topology changes, which can be detrimental to performance and efficiency.
Innovation Solution
Implementing an online restore method that allows databases to transition to new topologies with custom data distribution schemes while remaining available, using a two-phase approach where an initial general distribution is followed by a background redistribution to achieve custom distribution without blocking access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional restore techniques are used to change database topology, then the database can be restored to a new topology, but the database must be unavailable during the restore process causing downtime
Solution Approach 1:
The restore process is segmented into two distinct phases: an initial phase that establishes basic functionality with general data distribution, and a subsequent background phase that achieves optimal custom data distribution. This segmentation allows the database to become available immediately for basic operations while the more time-consuming custom distribution occurs in the background without blocking access.
Solution Approach 2:
The system performs preliminary actions by establishing the database topology and initial data distribution before completing the full custom distribution. The initial phase sets up the database structure and enables basic operations in advance, allowing the database to be available sooner while the remaining customization proceeds in the background.
2Adaptability or versatility
If database topology is changed to match changing workloads, then resource allocation efficiency improves, but the restore process becomes more complex
Solution Approach 1:
The system implements dynamic resource allocation by allowing the database topology to be changed and scaled according to varying workload requirements. The two-phase restore process enables flexible adaptation to different topology configurations while managing complexity through automated background processing of custom data distribution.
Solution Approach 2:
The system performs self-service by automatically handling the custom data distribution process in the background without requiring manual intervention. The background process autonomously redistributes data according to the custom distribution scheme, reducing the complexity burden on users while maintaining adaptability to different workload scenarios.
3Productivity
If custom data distribution is implemented during restore, then data processing efficiency improves, but the restore process takes longer and blocks database access
Solution Approach 1:
The system performs preliminary data distribution using a general scheme that enables basic database operations before the custom distribution is complete. This preliminary action allows the database to be available for use sooner, while the more time-consuming custom distribution process occurs in the background to optimize data processing efficiency without blocking access.
Solution Approach 2:
The system maintains continuity of useful action by allowing database operations to continue during the background custom distribution process. The initial phase establishes sufficient data distribution to support basic operations, and the background phase continuously improves distribution efficiency without interrupting database availability or blocking client access.
Data Source
AI summary
Online restore may be performed between databases with different topologies while applying a custom data distribution. A request to restore a database into a different topology of nodes may be received. A plan to move different portions of the database from a current topology to the new topology made using a general distribution scheme. The plan may be performed to move the different portions of the database into the new topology and the database made available for access using the new topology. A background process may be applied to modify the distribution of the database at the new topology to match a custom distribution scheme that was implemented at the current topology.