Table Replica Allocation in Replicated Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional replicated storage systems fail to balance data storage resources effectively, leading to unnecessary storage and increased costs due to inefficient allocation of table replicas across geographically disparate storage clusters.
Innovation Solution
A replicated storage system determines query loads and resource parameters to generate candidate table allocations, using a utility function to select optimal allocations that balance resource usage, ensuring tables with high query loads are replicated across multiple clusters and related tables are stored together, thereby reducing storage and network bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If tables are replicated across all storage clusters, then data availability is improved, but storage resource usage increases and becomes unbalanced
Solution Approach 1:
The patent applies local quality by making replication decisions specific to each table based on its query load characteristics. Instead of uniformly replicating all tables across all clusters, the system evaluates individual table access patterns and replicates only those tables that would benefit from multiple copies, thereby improving data availability for frequently accessed tables while avoiding unnecessary storage consumption for less accessed tables.
Solution Approach 2:
The system dynamically changes the replication parameter (number of replicas) based on query load metrics. Tables with high query loads are replicated across multiple storage clusters, while tables with low query loads are stored on fewer clusters. This parameter adjustment resolves the contradiction by adapting replication levels to actual usage patterns rather than applying a fixed replication strategy to all tables.
2Speed
If tables are replicated across multiple storage clusters, then query access speed is improved, but network bandwidth usage increases
Solution Approach 1:
The system applies partial action by replicating only the necessary portion of tables rather than all tables. Based on query load analysis, the system identifies which tables require replication for fast access and replicates only those specific tables across multiple clusters. This selective approach improves query access speed for critical tables while minimizing unnecessary network bandwidth consumption that would result from replicating all tables.
3Ease of operation
If conventional replicated storage systems allocate table replicas, then data accessibility is improved, but resource balancing fails and costs increase
Solution Approach 1:
The system implements feedback by continuously monitoring query loads on storage clusters and using this information to dynamically adjust table replication allocations. The feedback loop analyzes actual usage patterns and reallocates table replicas to achieve better resource balancing across clusters. This resolves the contradiction by enabling the system to maintain data accessibility while automatically optimizing resource distribution based on real-time performance data.
Data Source
AI summary
A replicated storage system allocates tables to storage clusters to balance the usage of resources in the replicated storage system. The replicated storage system determines the query load of the stored tables and generates candidate table allocations based on the query loads. The candidate table allocations are generated based on table allocation criteria based on the popularity of the tables or whether the tables are related to other tables. The replicated storage system determines resource parameters for each candidate table allocation and generates a resource usage score for each candidate table allocation based on a utility function that is weighted based on priorities of the resources used by the replicated storage system. The replicated storage system selects a table allocation from the candidate table allocations based on the resource usage scores of the candidate table allocations and allocates table replicas to the storage clusters.