Storage Data Rewriting for Faster HDD Reads After Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems where duplicate data deletion (deduplication) is used, successive versions of the same document result in lower read speeds for new versions due to seek times and reduced data read bandwidth, especially in scenarios involving hard disk drives (HDDs) with scattered data storage.
Innovation Solution
A data processing method that determines and rewrites data based on the distribution of physical addresses, optimizing address ranges to minimize rewritten data and improve read/write performance by assigning data into address ranges with specific length criteria and sequence numbers to balance read/write bandwidths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication is performed on successive versions of the same document, then storage space is saved and write time is reduced, but read speed of the new version becomes lower than the old version
Solution Approach 1:
The patent segments the data into blocks and manages them through a mapping relationship between old versions and new versions. By segmenting the data structure and using versioned blocks, the system can maintain deduplication benefits while enabling efficient reads of new versions through the mapping mechanism that points to the latest version's location.
Solution Approach 2:
The patent performs preliminary actions by pre-establishing mapping relationships between old version data blocks and new version data blocks before read operations occur. This mapping structure is built in advance during write operations, so that when reads are needed, the system can directly access the correct data location without performing complex operations at read time.
2Productivity
If data is stored in scattered physical addresses to enable deduplication, then storage efficiency is improved, but read performance deteriorates due to increased seek times
Solution Approach 1:
The patent introduces a mapping relationship as an intermediary between the logical data structure and the physical scattered addresses. This mapping layer acts as a mediator that translates read requests for new versions into the correct physical locations, allowing the system to maintain scattered storage for efficiency while providing ordered access for performance.
Solution Approach 2:
The patent changes the addressing parameters by introducing versioned address mappings. Instead of directly accessing physical addresses, the system uses mapped addresses that incorporate version information. This parameter transformation allows the system to present a sequential access pattern to applications while the underlying physical storage remains scattered for deduplication efficiency.
3Speed
If all first data is rewritten to improve read performance, then read speed is improved, but write time and rewritten data amount increase
Solution Approach 1:
The patent applies local quality by selectively rewriting only the first data blocks that have scattered physical addresses and need reorganization, rather than rewriting all data. The system identifies specific local regions (address ranges) that require rewriting based on their distribution characteristics, applying the rewrite operation only where necessary to improve read performance.
Solution Approach 2:
The patent uses partial action by rewriting only a subset of the first data blocks rather than all of them. The system determines the minimum necessary rewriting based on the distribution of physical addresses and the number of address ranges that need reorganization, avoiding excessive rewriting operations that would waste time and resources.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Embodiments of this application provide a data processing method and a storage system. The method is performed by the storage system, and includes: obtaining physical addresses of a plurality of pieces of first data stored in a hard disk, where each piece of first data is the same as at least one piece of second data in a plurality of pieces of second data written later or to be written; and determining, based on distribution of the physical addresses of the plurality of pieces of first data, to-be-rewritten target first data in the plurality of pieces of first data. In a solution provided in embodiments of this application, data read performance is further improved.