Replica Set Reclustering for Storage Capacity Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data replication systems face scalability limitations due to node storage capacity constraints, where data cannot be fully replicated across all nodes in a replica set, especially when nodes are geographically dispersed, making it difficult and costly to add storage capacity, and existing solutions like predefining a master cluster increase complexity and resource consumption.
Innovation Solution
A method and system for reclustering, where a computer determines node capacity thresholds, creates snapshots, partitions data, and initializes new replica sets, allowing for on-demand extensibility without predefining a master cluster, enabling data to be split and stored efficiently across multiple clusters while maintaining operational redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated across multiple nodes in a replica set, then fault tolerance and data accessibility are improved, but node storage capacity constraints prevent full replication when nodes are geographically dispersed
Solution Approach 1:
The patent applies segmentation by dividing the replica set into multiple shard sets, where each shard set contains a subset of the total data. This allows the system to scale beyond the storage capacity of individual nodes by distributing different data segments across different shard sets, thereby resolving the contradiction between maintaining fault tolerance through replication and overcoming node storage capacity constraints.
2Ease of operation
If a master cluster is predefined to manage replication, then data management is simplified, but system complexity and resource consumption increase
Solution Approach 1:
The patent implements self-service by enabling replica sets to autonomously manage their own data distribution across multiple shard sets without requiring a centralized master cluster. Each replica set can independently perform operations such as data sharding, replication, and failover, which reduces system complexity and resource consumption while maintaining ease of operation through automated self-management capabilities.
3Adaptability or versatility
If nodes are geographically dispersed for redundancy, then data accessibility is improved, but adding storage capacity becomes difficult and costly
Solution Approach 1:
The patent applies dimensionality change by organizing data across multiple hierarchical dimensions: replica sets contain multiple shard sets, and each shard set contains multiple shards. This multi-dimensional organization allows the system to scale storage capacity by adding new replica sets or shard sets rather than simply expanding individual node capacity, making geographic distribution more flexible and cost-effective while maintaining data accessibility.
Data Source
AI summary
A computer determines whether it has received user input or a node within a replica set has reached a capacity threshold. Based on receiving user input or determining that a node within a replica set has reached a capacity threshold, creating a snapshot of the data stored in the replica set and partitioning the data based on the created snapshot. The computer then initializing nodes within a new replica set and moves a partition from the original replica set to the new replica set before deleting the other partition from the old replica set.