Storage Data Rewriting Based on Physical Address Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems, particularly when used as secondary storage, successive versions of the same document lead to lower read speeds due to deduplication processes that affect the performance of newer versions.
Innovation Solution
A data processing method that determines the physical addresses of data in a storage system and assigns them into address ranges to identify data that needs rewriting, optimizing the amount of data rewritten to improve read performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication is performed on data in new versions to save storage space and reduce write time, then storage space utilization is improved, but read speed of new versions becomes lower than old versions
Solution Approach 1:
The system performs preliminary analysis of data distribution patterns before deduplication operations. By pre-calculating which data blocks are likely to be frequently accessed and rewriting them to consecutive physical addresses, the system prepares the storage structure in advance to minimize read latency while maintaining deduplication benefits.
Solution Approach 2:
The deduplication strategy is made dynamic by continuously monitoring data access patterns and adjusting the rewriting policy accordingly. The system dynamically determines which data blocks should be rewritten based on current distribution characteristics, allowing the balance between storage efficiency and read performance to adapt to changing workloads.
2Speed
If data is rewritten to improve read performance, then read speed is improved, but the amount of rewritten data increases write time and storage operations
Solution Approach 1:
Instead of uniformly rewriting all deduplicated data, the system applies local quality by selectively rewriting only those data blocks that exhibit poor distribution characteristics and are likely to cause read performance degradation. This targeted approach improves read performance for critical data while minimizing the overall write overhead.
Solution Approach 2:
The system changes the parameter of data distribution by rewriting scattered data blocks to consecutive physical addresses. This parameter transformation converts poorly distributed data into well-distributed data, improving read performance without requiring complete rewrites of all data.
3Speed
If all duplicate data is rewritten to ensure optimal read performance, then read performance is maximized, but storage space is wasted and write operations increase
Solution Approach 1:
The system applies partial action by rewriting only a subset of duplicate data blocks rather than all of them. By analyzing data distribution patterns, the system identifies the minimum necessary rewriting to achieve acceptable read performance, avoiding excessive rewriting that would waste storage space and increase write operations unnecessarily.
Data Source
AI summary
Embodiments of this application provide example data processing methods and example storage systems. One example method is performed by the storage system, and includes obtaining physical addresses of a plurality of pieces of first data stored in a hard disk, where each piece of first data is the same as at least one piece of second data in a plurality of pieces of second data written later or to be written. To-be-rewritten target first data in the plurality of pieces of first data is determined based on distribution of the physical addresses of the plurality of pieces of first data.