Storage Server Stripe Migration With Incremental Erasure Code Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data migration methods in storage servers, particularly in distributed and unified storage systems, face inefficiencies and load imbalances due to node failures and hardware changes, necessitating improved methods to ensure data reliability and efficient migration.
Innovation Solution
A data migration method that determines the number of strips to be migrated based on storage server information, selects strips according to a preset rule, and performs incremental updates of erasure codes to balance load and minimize data movement, using a partial strip migration approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data blocks are rearranged during disk insertion, then load balancing is achieved, but migration efficiency deteriorates
Solution Approach 1:
The patent divides the data storage structure into strips, where each strip contains multiple data blocks and check blocks. Instead of migrating all data blocks individually, the system migrates entire strips as units. This segmentation reduces the number of migration operations and improves efficiency while maintaining load balancing through the strip-based distribution mechanism.
Solution Approach 2:
The patent calculates the number of strips to be migrated based on the current disk configuration and storage information, rather than migrating all possible strips. This partial action approach ensures that only the necessary strips are migrated to achieve proper load balancing, avoiding unnecessary migration operations and improving overall efficiency.
2Reliability
If all data blocks are migrated to new disks, then load balancing is improved, but computational overhead increases
Solution Approach 1:
The system segments data into strips with embedded check blocks, allowing the migration process to work with these pre-organized units rather than individual data blocks. This reduces computational overhead by minimizing the number of separate encoding and migration operations required.
Solution Approach 2:
The patent performs preliminary calculations to determine the exact number of strips that need to be migrated based on current storage information and disk capacity. This preliminary planning avoids unnecessary computational operations during the actual migration process, reducing overall computational overhead while ensuring proper load distribution.
3Reliability
If erasure codes are updated for all strips, then data reliability is maintained, but migration time increases
Solution Approach 1:
The patent segments the storage system into strips, where each strip has its own check blocks. When migrating data to new disks, only the strips that need migration have their check blocks updated, while strips that remain in place maintain their original check blocks. This selective update approach reduces migration time while preserving data reliability through the erasure coding mechanism.
Solution Approach 2:
The system performs erasure code updates only for the partial set of strips that are being migrated, rather than updating all strips in the system. This partial action reduces the total time required for migration operations while maintaining data reliability for the migrated data through proper check block updates.
Data Source
AI summary
Disclosed are a data migration method, system and device for a storage server, and a non-transitory readable storage medium. The method includes: according to storage information of a storage server, determining the number n of strips to be migrated; selecting n stripes to be migrated from among c full stripes according to a preset migration rule, and selecting, from the n stripes to be migrated, data blocks to be migrated and migrating the data blocks to the newly inserted disks, so that after the migration is completed, the difference in stripe usage between any two disks in the storage server does not exceed 1; and determining data content of check data blocks of the strips so as to complete data migration of the storage server, and determining data content of check data blocks of the c-n strips by means of incremental update of an erasure code.