TCFS On-Disk Layout for Distributed Consensus I/O Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed consensus protocols in storage systems face issues with file fragmentation, high log entry accumulation, and inefficient sequential reading and writing operations, leading to substantial overhead and potential data corruption.
Innovation Solution
The Third Copy File System (TCFS) employs an optimized on-disk layout with error-correcting codes for snapshot and membership objects, atomic multi-block writes for log objects, and on-demand snapshotting to manage configuration updates efficiently, ensuring data integrity and reducing storage overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a generic file system layout is used to store configuration information, then the implementation is simple and generic, but file fragmentation occurs, log entries accumulate excessively, and sequential reading efficiency deteriorates
Solution Approach 1:
The patent segments the configuration information storage into distinct file types (membership file, snapshot file, log file) with dedicated layouts. Each file type is optimized for its specific purpose: membership files store cluster node information, snapshot files store configuration state snapshots, and log files store sequential log entries. This segmentation eliminates the fragmentation issues of generic file systems while maintaining implementation clarity through well-defined file structures.
Solution Approach 2:
The patent introduces a new dimension of organization by implementing a hierarchical file structure with specific directory layouts and file naming conventions. Configuration information is organized into dedicated directories (e.g., /etc/cluster, /var/lib/cluster) with structured file formats that include headers, data sections, and metadata. This dimensional organization enables efficient sequential access and eliminates the flat, fragmented structure of generic file systems.
2Reliability
If multiple write and commit operations are performed for each file update, then data and metadata are persisted, but partial commits occur leading to data corruption risk
Solution Approach 1:
The patent merges the data and metadata write operations into a single atomic commit process. Each configuration file update is performed as one indivisible operation that either completely succeeds or completely fails, eliminating partial commits. The file system implements journaling or transactional semantics where the entire update (data plus metadata) is written and committed atomically, ensuring consistency without requiring multiple separate write operations.
Solution Approach 2:
The patent implements protective mechanisms before write operations, such as creating backup copies of files before updating them, using write-ahead logging to record intended changes before execution, and employing checksums or error-correcting codes to detect corruption. These preemptive measures cushion against potential write failures or partial commits, ensuring data integrity without requiring complex recovery procedures.
3Ease of manufacture
If log entries are stored in a generic format, then storage is straightforward, but too many log entries accumulate before purging and sequential reading requires substantial overhead
Solution Approach 1:
The patent performs preliminary organization of log entries during the write operation itself. Log files are structured with predefined formats including entry headers, sequence numbers, timestamps, and delimiters that enable efficient parsing and sequential reading. The file system pre-allocates log file structures and implements circular buffers or ring buffers that automatically manage entry accumulation and purging, eliminating the need for complex post-processing of log data.
Solution Approach 2:
The patent changes the parameters of log storage by implementing fixed-size log entries with standardized formats, sequential numbering schemes, and predetermined retention policies. Log files use optimized data structures such as arrays or linked lists with fixed record lengths, enabling efficient sequential access and random deletion of specific entries. These parameter changes transform log storage from a generic, flexible format to a highly optimized, structured format that minimizes management overhead.
Data Source
AI summary
A layout of a file system is optimized to meet storage requirements of a distributed consensus protocol implemented on a plurality of nodes of a cluster. Illustratively, the file system has an on-disk layout representation that enables efficient input/output (I/O) operation performance. The on-disk layout of the file system embodies a plurality of file system objects: membership, snapshot and log objects. Each object is allocated an area or region (e.g., a contiguous storage space) on-disk. In addition, each object has two or more storage container instances, e.g., files. In the case of snapshot and membership objects, the two files of each object are used in a circular (alternating) fashion so that write operations directed to a first file storing a current copy of the snapshot/membership content may be performed without destroying a previous committed copy of the snapshot/membership content stored in a second file.