In-Line Erasure Coding Using Write-Ahead Log

The patent applies preliminary action by performing erasure coding operations on data chunks before they are committed to chunk files, during the write process itself. The EC configuration is applied in-line as data is being written to the write-ahead log, converting data chunks to data stripes and generating parity stripes immediately, rather than performing these operations later during a separate post-process phase. This eliminates the need for subsequent metadata updates and chunk file rewrites.

The patent applies segmentation by dividing data into variable-sized data chunks based on the actual write size, rather than using fixed large data stripe widths. Each data chunk is then processed through EC to generate the appropriate number of data stripes and parity stripes. This allows the system to maintain efficient EC processing by processing data in appropriately sized segments rather than forcing all data into large fixed-width stripes, thereby improving storage space utilization for small writes while maintaining processing efficiency.

The patent applies merging by consolidating the storage of data stripes and parity stripes into a unified write-ahead log structure, rather than distributing them across multiple separate chunk files. The write-ahead log sequentially stores data chunks that are converted to data stripes and parity stripes in-line, eliminating the need for complex metadata updates across multiple chunk files while maintaining fault tolerance through the EC configuration. This unified approach reduces metadata complexity and IOPS requirements while preserving reliability.