Hash-Based Data Migration Timing for Distributed Storage Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed data storage systems face challenges in redistributing data across multiple nodes without causing high migration loads on specific nodes, which can lead to system downtime and accessibility issues during configuration transitions.
Innovation Solution
The method employs a migration function, such as a hash function, to determine the order and timing of data package migrations, spreading the load across nodes and time, allowing for concurrent access and minimizing peak loads by using migration values based on keys and optional counters for controlled migration timing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data migration is performed quickly to minimize transition time, then productivity is improved, but migration load on specific nodes increases excessively
Solution Approach 1:
The patent segments the data migration process by dividing data packages into groups based on migration values derived from hash functions. Each node migrates a specific segment of data at controlled intervals, preventing any single node from being overwhelmed by the entire migration load simultaneously.
Solution Approach 2:
The patent implements periodic action by using counters that increment over time and compare against migration values. Data packages are migrated at periodic intervals determined by these counters, spreading the migration load evenly across time periods rather than concentrating it in a single rapid operation.
2Manufacturing precision
If all data packages are migrated consecutively, then manufacturing precision is improved, but migration load concentration increases
Solution Approach 1:
The patent changes the parameter of migration timing by introducing migration values derived from hash functions and comparing them against incrementing counters. This parameter change transforms the migration process from a simple sequential operation to a controlled, distributed timing mechanism that maintains accuracy while reducing peak loads.
3Reliability
If client access is cut off during migration, then reliability of migration process is improved, but system availability deteriorates
Solution Approach 1:
The patent enables continuity of useful action by allowing client access to the storage system to continue during the migration process. The systematic migration approach ensures data integrity while maintaining system availability, as clients can access data from nodes that have not yet migrated or are in the process of migrating.
Data Source
Figure 1
Figure 2
AI summary
The present invention relates to a method of setting up a redistribution scheme for redistributing digital data packages 6 within a digital data storage system 4 comprising a plurality of nodes 5, wherein the data packages 6 are associated with respective keys 7 and are distributed among the nodes 5 according to a first distribution configuration within the digital data storage system 4, the method comprising: determining a second distribution configuration, in accordance with which it is intended to redistribute the data packages 6; applying a migration function to the respective keys 7 of each of the data packages 6, which function yields a migration value associated with each of the data packages 6; and assigning a migration time to each of the data packages 6 based on its associated migration value, at which times it is intended to migrate the respective data packages 6 to conformity with the second distribution configuration. The present invention is also related to a digital data storage system 4.