Database Shard Replication for Fast Multi-Tenant Node Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face issues with high costs, unsuitable elasticity for small tenants, large 'blast radius' failures, and inefficient redundancy recovery due to overprovisioned resources and full database replication.
Innovation Solution
Sharding of tenant data across multiple storage nodes with fine-grained elastic space allocation, parallel query processing, and distributed replication of shards to minimize failure impact and accelerate redundancy restoration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional full database replication is used for high availability, then reliability is improved, but device complexity and cost increase due to requiring complete failover instances
Solution Approach 1:
The patent segments the database into shards distributed across multiple storage nodes. Instead of replicating entire database instances for failover, only individual shards are replicated across different storage nodes. This segmentation allows partial redundancy without requiring complete failover instances, reducing infrastructure complexity while maintaining reliability through distributed shard replication.
2Reliability
If traditional full database replication is used for recovery, then reliability is improved, but loss of time increases due to considerable recovery time
Solution Approach 1:
By segmenting the database into shards and replicating only individual shards across storage nodes, the system enables parallel recovery operations. When a storage node fails, only the specific shards on that node need to be recovered from replicas, rather than recovering an entire database instance. This dramatically reduces recovery time while maintaining reliability.
Solution Approach 2:
The system performs preliminary replication of shards across multiple storage nodes before failures occur. This preliminary distribution of data ensures that replicas are already in place and can be immediately used for recovery, eliminating the need for time-consuming full database replication after failures.
3Productivity
If networking resources are overprovisioned for data-intensive operations, then productivity is improved, but loss of energy increases due to wasted costs
Solution Approach 1:
The patent implements dynamic resource allocation where networking resources are provisioned based on actual workload demands rather than static overprovisioning. The distributed shard architecture allows the system to efficiently utilize network bandwidth only when data-intensive operations are actually occurring, rather than maintaining constant overprovisioned capacity, thereby reducing wasted energy costs.
Data Source
AI summary
Systems and methods include storage of database table shards on a plurality of storage nodes. Each shard may be stored as multiple replicas, where each replica of a given shard is stored on a different storage node. In case of a failure of a given storage node, all shard replicas which were stored on the given storage node are copied from other storage nodes to new or existing storage nodes. A storage node may store shards of database tables associated with multiple tenants.