Log-Structured Storage Garbage Collection Parity Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems using the log-structured scheme, garbage collection efficiency is hindered by parity reading requirements in RAID levels like RAID 5 or RAID 6, leading to decreased performance due to the need to read old parity and data during updates, which affects the efficiency of data management and storage.
Innovation Solution
The storage system employs a controller that divides RAID groups into stripes, uses address conversion information to invalidate old data, and executes a copy GC process by selecting non-additionally recordable areas as copy sources and destinations on a per-strip or per-stripe basis, padding data as necessary to optimize writing and reduce fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If parity reading is performed during garbage collection in RAID 5 or RAID 6 levels, then data integrity is maintained, but GC efficiency decreases due to additional read operations
Solution Approach 1:
The patent pre-calculates and stores parity information in advance during normal write operations. During garbage collection, this pre-computed parity data is directly utilized without requiring additional read operations from the storage devices, thereby maintaining data integrity while improving GC efficiency
Solution Approach 2:
The patent maintains continuous parity computation and updates it incrementally as data changes, rather than recomputing parity from scratch during GC. This continuous maintenance of parity information ensures reliability is preserved while avoiding the performance penalty of re-reading old parity data
2Quantity of substance
If old data is invalidated during update writing, de-duplication, or compression, then storage space is freed, but free area fragmentation increases
Solution Approach 1:
The patent segments the free area management into multiple independent regions or blocks. When fragmentation occurs, each segment can be managed and consolidated independently, allowing the system to maintain usable contiguous space even when overall fragmentation exists. This segmentation approach enables more flexible space allocation and reduces the impact of fragmentation on performance
3Loss of substance
If log-structured scheme is adopted for compression and de-duplication, then data reduction is achieved, but GC complexity increases due to address mapping requirements
Solution Approach 1:
The patent uses copy-on-write semantics where old data blocks are copied to new locations and validity is transferred through metadata updates rather than physical relocation. This copying approach simplifies the GC process by avoiding complex address remapping while still achieving data reduction through compression and de-duplication operations
Solution Approach 2:
The patent introduces an intermediary metadata layer that manages the mapping between logical and physical addresses. This intermediary structure abstracts the complexity of address mapping from the GC process itself, allowing GC to operate on a simplified view while the metadata layer handles the detailed address translation required for log-structured operations
Data Source
AI summary
A storage system selects from a plurality of physical areas constituting a physical address space as copy source physical areas, one or more non-additionally recordable physical areas each including a fragmented free area, and also selects a recordable physical area as a copy destination physical area. The storage system then writes one or more pieces of live data from the selected one or more copy source physical areas to the free area of the selected copy destination physical area on a per-strip or per-stripe basis, sequentially from the beginning of the free area. If the size of the write target data is such that it is not possible to write the write target data to the free area on a per-strip or per-stripe basis, then the storage system pads the write target data, and writes the padded write target data to the free area on a per-strip or per-stripe basis.