Partition Expansion in Distributed Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face poor expansibility due to a limited number of partitions per physical node, limiting capacity expansion to only four times the original capacity.
Innovation Solution
A method and apparatus that dynamically increase the number of partitions per physical node by calculating an average value and multiplying it when below a threshold, updating the distributed hash table with new partition mappings to ensure even load distribution and capacity expansion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If capacity expansion is performed multiple times by adding new physical nodes, then system capacity increases, but the quantity of partitions in each physical node decreases, limiting further expansion
Solution Approach 1:
The patent divides each physical node into multiple virtual partitions, where each virtual partition can independently store data. This segmentation allows a single physical node to handle multiple partition assignments, enabling the system to scale beyond the original limitation where each node could only hold one partition per expansion cycle.
Solution Approach 2:
The patent introduces a new dimension to partition management by implementing virtual partitioning within physical nodes. Instead of only expanding horizontally by adding nodes, the system now has a vertical dimension where each node can accommodate multiple virtual partitions, effectively multiplying the partition capacity of each physical node.
2Device complexity
If the quantity of partitions in physical nodes is kept small for simplicity, then device complexity is reduced, but the system can only expand to a maximum of four times original capacity
Solution Approach 1:
The patent makes physical nodes universal by enabling each node to serve multiple partition assignments through virtual partitioning. Each physical node can now function as multiple logical partitions, allowing the same hardware infrastructure to support larger system capacities without proportionally increasing physical infrastructure or management complexity.
3Stability of the object's composition
If partitions are evenly distributed to physical nodes for load balancing, then load distribution is even, but the system lacks flexibility for multiple capacity expansions
Solution Approach 1:
The patent implements dynamic partition assignment where virtual partitions can be flexibly assigned to and reassigned from physical nodes based on system state. This dynamic approach maintains even load distribution while providing the flexibility to accommodate multiple capacity expansion scenarios, as the system can rebalance partitions across nodes as they are added or removed.
Data Source
AI summary
Embodiments of the present invention disclose a partition extension method and apparatus, which relate to the field of storage technologies, so that a quantity of partitions in a physical node can be increased during capacity expansion of a distributed storage system, thereby ensuring expansibility of the distributed storage system. A specific solution is as follows: acquiring an average value of quantities of partitions in all physical nodes in a distributed storage system; if the average value of the quantities of the partitions is less than a preset threshold, performing multiplication on a quantity of partitions in each physical node; and adding a mapping relationship between a newly added partition and a physical node in which the newly added partition is located into a pre-stored distributed hash table DHT. The present invention is applicable to a process of capacity expansion of the distributed storage system.