Segment Pre-allocation for Erasure Coded Storage Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face inefficiencies in write operations due to the need for partial stripe writes, which result in high write amplification and performance issues, especially when using erasure coding schemes like RAID-6, as they require significant NVRAM for caching or complex management of metadata with log-structured file systems.
Innovation Solution
Implementing a distributed storage system with two tiers of physical storage, using a LFS disk layout for capacity objects and an overwrite-based file system for metadata objects, allowing for full stripe writes and avoiding the complexities of metadata management in log-structured systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If erasure coding is used for fault tolerance, then storage overhead is reduced compared to mirroring, but write operation efficiency deteriorates due to partial stripe writes requiring read-modify-write sequences
Solution Approach 1:
The invention segments the write operation into two distinct phases: a metadata update phase that logs the intended write operation, and a data write phase that executes the actual write. This segmentation allows the system to decouple the metadata management overhead from the data write performance, enabling efficient partial stripe writes without the read-modify-write penalty.
Solution Approach 2:
The system performs preliminary actions by logging write operations to metadata before executing the actual data writes. This preliminary metadata update allows the system to prepare and coalesce multiple partial stripe writes into full stripe writes, improving write efficiency by avoiding repeated read-modify-write sequences.
2Productivity
If NVRAM is used to cache partial stripe writes, then write operation efficiency is improved by coalescing writes into full stripe writes, but system cost increases due to expensive NVRAM requirements
Solution Approach 1:
The invention replaces expensive NVRAM with cheaper, volatile memory resources that are already present in commodity servers. The system uses available RAM for write coalescing and relies on the log-structured metadata mechanism to ensure data integrity, eliminating the need for costly non-volatile memory while maintaining write efficiency.
Solution Approach 2:
The system creates a logical copy of the write operation in the metadata log before executing the physical write. This metadata copy allows the system to track and coalesce writes without requiring expensive caching hardware, using software-based mechanisms instead of hardware-based solutions.
3Productivity
If a log-structured file system is used to coalesce partial stripe writes, then write operation efficiency is improved, but implementation complexity increases due to metadata management requirements
Solution Approach 1:
The invention extracts the metadata management function from the traditional file system and implements it as a separate, simplified logging mechanism. By taking out the complex metadata management requirements and replacing them with a straightforward log-structured approach, the system achieves write coalescing without the implementation complexity of a full log-structured file system.
4Quantity of substance
If large segments are used in erasure coded objects, then storage efficiency is improved, but reliability deteriorates because failure of a single node causes loss of entire large segments
Solution Approach 1:
The invention segments large storage objects into smaller data blocks that are then erasure coded across multiple nodes. This segmentation ensures that failure of a single node only affects a portion of the data, not entire large segments, while still maintaining storage efficiency through optimized erasure coding schemes.
Data Source
AI summary
Techniques for supporting large segments when issuing writes to an erasure coded storage object in a distributed storage system are provided. In one set of embodiments, a node of the system can pre-allocate a segment of space in a capacity object of the storage object, receive a write request for updating a logical data block of the storage object, write data/metadata for the block to a record in a data log of a metadata object of the storage object, place the block in an in-memory bank, and determine whether the in-memory bank has become full. If so, the node can compute/fill-in one or more parity blocks for each stripe of the storage object in the in-memory bank and write, based on a next sub-segment pointer pointing to a free sub-segment of the pre-allocated segment, the contents of the in-memory bank via a full stripe write to the free sub-segment.