Erasure-Coded Storage Resync via Segment Usage Table
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
RAID-6 data storage systems face inefficiencies in write operations and capacity utilization due to the need for multiple read and write operations for fault tolerance, which can lead to increased latency and computational overhead, and conventional data recovery methods are inefficient due to the use of large bitmaps for tracking updates.
Innovation Solution
A storage system utilizing a meta object and a capacity object to perform full-stripe writes and maintain a segment usage table for efficient data resynchronization, eliminating the need for bitmaps and allowing exclusive or nearly exclusive full-stripe writes, thereby reducing latency and computational resources required for data recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RAID-6 uses block-level striping with multiple parity disks for fault tolerance, then data reliability is improved, but write efficiency and capacity utilization deteriorate
Solution Approach 1:
The invention segments the capacity object into multiple segments, each with its own stripe of data and parity information. This allows the system to perform full-stripe writes to multiple segments in parallel, improving write efficiency while maintaining the fault tolerance provided by distributed parity across all segments.
Solution Approach 2:
The invention merges multiple sequential write operations into a single full-stripe write operation that updates all blocks in a stripe simultaneously. This combining approach reduces the total number of operations required and improves write throughput while maintaining data integrity across all blocks.
2Ease of repair
If conventional data recovery methods use bitmaps to track updated blocks, then data recovery capability is improved, but device complexity and storage overhead increase
Solution Approach 1:
The invention extracts and eliminates the bitmap data structure from the system. Instead of using bitmaps to track updated blocks, the system relies on the segment usage table and transaction ID mechanism to determine which segments need resynchronization, thereby removing the complexity and storage overhead associated with bitmaps.
Solution Approach 2:
The segment usage table serves multiple functions: it tracks which segments have been written to, stores transaction IDs for ordering operations, and enables the system to determine resynchronization needs without requiring separate bitmap structures. This multi-functionality reduces overall system complexity.
3Productivity
If full-stripe writes are performed to update all blocks in a stripe, then write efficiency is improved, but computational resources for data recovery increase
Solution Approach 1:
The invention makes the resynchronization process dynamic by using transaction IDs to determine which segments need to be resynchronized based on the current state of the system. This dynamic approach allows the system to perform full-stripe writes efficiently while only processing the necessary segments for recovery, reducing unnecessary computational resources.
Solution Approach 2:
The system performs preliminary actions by maintaining the segment usage table and transaction IDs continuously during normal operation. This preliminary tracking enables the system to quickly identify which segments need resynchronization after a failure without performing unnecessary computational operations during the recovery process.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for resynchronizing data in a storage system. One of the methods includes determining that a particular disk of a capacity object of a storage system was offline for an interval of time, wherein the capacity object comprises a plurality of segments, and wherein the storage system comprises a segment usage table identifying a linked list of particular segments of the capacity object that are in use; determining a time point at which the particular disk went offline; determining one or more first segments of the capacity object that were modified after the time point, wherein determining one or more first segments comprises determining each segment of the segment usage table having a transaction ID that is larger than the time point; and resynchronizing, for each first segment, a portion of the particular disk corresponding to the first segment.