Distributed Storage Node Group Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In small-scale distributed storage systems, adding new nodes to expand the cluster while maintaining data redundancy and minimizing data movement is challenging, as existing methods require significant data redistribution and increase the risk of data loss due to node failures.
Innovation Solution
A method where a sliding window process gradually separates two node groups as new nodes are added, allowing for the creation of new node groups without extensive data movement by sharing some nodes between groups and maintaining exclusive assignment of others, thereby reducing the risk of data loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple nodes are added to expand the cluster, then the storage capacity and performance are improved, but the data redistribution and risk of data loss increase
Solution Approach 1:
The patent divides the cluster into multiple node groups (e.g., Group 1 and Group 2) with distinct node sets. Each group independently manages its data, allowing selective expansion without affecting other groups. This segmentation isolates failure domains, so that adding nodes to one group does not increase the risk of data loss across the entire cluster.
Solution Approach 2:
The patent implements dynamic node group configuration where nodes can be selectively assigned to different groups based on expansion needs. The system can dynamically adjust which nodes belong to which group, allowing flexible cluster growth while maintaining data redundancy and reliability through configurable group boundaries.
2Reliability
If node groups are created to improve data redundancy, then the reliability is improved, but the data movement and system complexity increase
Solution Approach 1:
The patent segments nodes into distinct groups with clear boundaries, making it straightforward to manage data redundancy within each group. Each group can independently implement redundancy mechanisms (such as RAID or replication) without affecting other groups, simplifying the overall system complexity compared to a single large group.
Solution Approach 2:
The patent applies local redundancy mechanisms within each node group tailored to the specific needs and characteristics of that group. Each group can have its own redundancy strategy (e.g., different replication factors or RAID configurations) optimized for local requirements, rather than applying a one-size-fits-all redundancy approach across the entire cluster.
3Quantity of substance
If nodes are added to expand the cluster, then the storage capacity is improved, but the data movement and time consumption increase
Solution Approach 1:
The patent segments the cluster into independent node groups, allowing new nodes to be added to specific groups without requiring data migration across the entire cluster. Data remains localized within groups, minimizing data movement time when expanding storage capacity by adding nodes to appropriate groups.
Solution Approach 2:
The patent pre-configures node groups with designated roles and data distribution patterns before expansion occurs. When nodes are added, the system can quickly integrate them into existing groups or create new groups without requiring extensive data redistribution, as the group structure and data organization are already established.
Data Source
AI summary
A distributed storage system includes a first node group consisting of a plurality of nodes and a second node group consisting of a plurality of nodes. Each of the first node group and the second node group distributes and stores data of an assigned object to a plurality of constituent nodes. The first node group includes at least one first node shared with the second node group and at least one second node not shared with the second node group. The second node group includes the at least one first node and at least one third node not shared with the first node group.