Data Replication Prioritizing Cold Blocks to Reduce Re-replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face challenges in shortening the initial replication completion time during data migration from a source storage device to a target storage device, as re-replication of overwritten data prolongs the process due to the need for additional transmission resources.
Innovation Solution
The method involves determining the overwritten probabilities of data blocks, classifying them as cold or hot, and replicating cold data blocks to the target storage device before hot data blocks, thereby reducing the need for re-replication resources during the initial replication period.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data blocks are replicated sequentially without classification, then all data can be replicated to the target storage device, but the initial replication completion time is prolonged due to re-replication of overwritten data
Solution Approach 1:
The patent applies preliminary action by calculating overwritten probabilities for all data blocks before replication starts, and classifying them into hot and cold categories in advance. This allows the replication process to prioritize cold data blocks (lower overwrite probability) for initial replication, reducing the need for re-replication during the initial phase and thereby shortening the initial replication completion time while maintaining eventual data completeness.
Solution Approach 2:
The patent segments the data blocks into two distinct categories: hot data blocks (higher overwrite probability) and cold data blocks (lower overwrite probability). This segmentation enables differential replication strategies where cold data blocks are replicated first with higher priority, while hot data blocks are handled differently, thus optimizing the replication timeline and reducing overall initial replication time.
2Reliability
If all data blocks are replicated with equal priority, then data completeness is maintained, but transmission resources are wasted on re-replicating overwritten data during initial replication period
Solution Approach 1:
The patent applies local quality by assigning different replication priorities and resource allocation to different data blocks based on their overwrite probabilities. Cold data blocks (lower overwrite probability) receive higher replication priority and more transmission resources during the initial phase, while hot data blocks (higher overwrite probability) receive lower priority. This localized differentiation optimizes transmission resource utilization by avoiding wasteful re-replication of data that is likely to be overwritten soon.
Solution Approach 2:
The patent changes the replication parameter from uniform treatment to differentiated treatment based on overwritten probability. By introducing the overwrite probability as a key parameter for decision-making, the system dynamically adjusts replication priorities and resource allocation, thereby reducing transmission resources spent on re-replicating data blocks that are likely to be overwritten before the initial replication completes.
3Reliability
If re-replication of overwritten data is performed continuously, then data consistency is maintained, but the initial replication process is prolonged due to additional transmission requirements
Solution Approach 1:
The patent applies preliminary action by pre-calculating overwritten probabilities and classifying data blocks before the replication process starts. This allows the system to prepare a replication plan that prioritizes cold data blocks, reducing the frequency and impact of re-replication operations during the initial phase. As a result, the initial replication completes faster (improved productivity) while still maintaining data consistency through targeted re-replication of hot data blocks when necessary.
Data Source
AI summary
Techniques for replicating data involve: determining overwritten probabilities of a plurality of data blocks of a source storage device. The techniques further involve: classifying the plurality of data blocks as cold data blocks or hot data blocks based on the overwritten probabilities in the plurality of data blocks. The techniques further involve: replicating at least a portion of the cold data blocks to a target storage device prior to the hot data blocks. Accordingly, transmission resources used to re-replicate overwritten data during an initial replication period can be reduced, thereby achieving efficient data replication.