Erasure-Coded Logical Disk Blocks Without Read-Modify-Write
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing erasure coding schemes for storage systems incur significant performance costs, particularly during write operations, due to the need for read-modify-write operations, which amplify bandwidth requirements and latency, especially in systems with mismatched block sizes between logical and physical disks.
Innovation Solution
A system and method that utilize larger block sizes for logical disks and decompose them into smaller physical block sizes using erasure coding logic to avoid read-modify-write operations, allowing for efficient and cost-effective data redundancy with reduced data amplification during writes by mismatching block sizes and using erasure coding to reconstruct requests on reads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional erasure coding schemes are used with matched block sizes, then data redundancy is achieved, but write operations require read-modify-write which amplifies bandwidth requirements and increases latency
Solution Approach 1:
The patent applies asymmetry by deliberately mismatching the logical block size (LBS) and physical block size (PBS). Specifically, the LBS is set to a multiple of the PBS (e.g., LBS = 4KB, PBS = 512B), creating an asymmetric relationship that allows writes to bypass read-modify-write operations. This asymmetry enables the system to achieve both data redundancy through erasure coding and improved write performance by avoiding the need to read existing data before writing.
Solution Approach 2:
The patent segments the logical block into multiple physical blocks through the mismatched block size relationship. A single logical block of data is divided into multiple smaller physical blocks across different storage devices. This segmentation, combined with erasure coding, allows the system to maintain redundancy while enabling direct write operations without read-modify-write amplification.
2Stability of the object's composition
If read-modify-write operations are performed for every write, then data consistency is maintained, but bandwidth requirements are amplified and latency increases
Solution Approach 1:
The system performs preliminary action by pre-calculating and storing erasure coding parity information across multiple physical blocks. When a write operation occurs, the system can directly write to the appropriate physical blocks without needing to read existing data first, because the erasure coding structure is already in place. This preliminary setup enables direct writes while maintaining data consistency through the pre-established redundancy structure.
3Reliability
If full redundancy copies are stored, then data protection against failures is maximized, but storage costs increase
Solution Approach 1:
The patent applies parameter changes by modifying the redundancy ratio through erasure coding. Instead of storing full copies of data (100% redundancy), the system uses erasure coding to create partial redundancy blocks that provide equivalent protection with less storage overhead. For example, using an (n, k) erasure coding scheme where n > k, the system can tolerate up to (n-k) failures while using only (n/k) storage overhead, significantly reducing the quantity of stored data compared to full replication.
Data Source
AI summary
Disclosed are a method and device in a network for utilizing larger block sizes for a logical disk, and further decomposing into smaller physical block sizes for a redundant encoding by utilizing an erasure coding logic to avoid a read□modify□write operation on a plurality of write operations. The device includes a receiving module, an encoding module, and a transmitting module. The receiving module configured to obtain a read request or a write request of a large block size to a storage unit over a network. In an embodiment, the storage unit comprising a plurality of physical devices of a smaller blocks size that is a divisor of the large block size. The encoding module to encode the received requests redundantly to persist media failures by reconstructing the requests on reads using the redundancy. The transmitting module transmits either a request command to initiate a read operation or an acknowledgment data on completion of a write operation.