Persistent Memory Distributed-Journal File System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Journaling file systems face performance penalties in modern multi-threaded processors due to the need for constant synchronization of journal logs, which can become a bottleneck in data storage systems, especially with the increasing speed of near-memory storage devices, while maintaining crash immunity and data integrity.
Innovation Solution
A persistent memory-based distributed-journal file system is implemented, where each file is associated with a self-journal record, allowing for atomic operations to be executed without interrupting other operations, and maintaining high-speed data access by using direct memory mapping and volatile memory for frequently accessed resources datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a journal-based file system is used to ensure crash immunity and data integrity, then reliability is improved, but access speed and performance deteriorate due to constant synchronization requirements
Solution Approach 1:
The patent divides the single centralized journal into multiple distributed self-journal records, with each file having its own dedicated journal record. This segmentation allows parallel access to different file journals simultaneously, eliminating the synchronization bottleneck of centralized journaling while maintaining crash immunity through distributed logging of file operations.
Solution Approach 2:
The patent transitions from a single-dimension centralized journal structure to a multi-dimensional distributed journal architecture where journal records are spread across multiple files and locations. This dimensional change enables concurrent access from multiple threads without contention, as each thread can independently access and write to different file-specific journal records.
2Reliability
If a centralized journal is used to log all file operations, then data integrity is maintained, but the journal becomes a performance bottleneck in multi-threaded environments
Solution Approach 1:
The centralized journal is segmented into multiple distributed self-journal records associated with individual files. Each file operation logs to its corresponding file's self-journal record, allowing multiple threads to perform logging operations in parallel without contending for access to a single centralized journal, thereby increasing operation throughput while maintaining data integrity through distributed logging.
Solution Approach 2:
Each file maintains its own self-journal record, enabling the file to service its own logging requirements independently. This self-service approach eliminates the need for centralized journal coordination and synchronization, allowing files to autonomously log their operations and reducing the overall system bottleneck.
3Reliability
If extensive file system checks are performed after improper unmounting, then consistency is restored, but system downtime and access time increase
Solution Approach 1:
The patent performs preliminary consistency checks and recovery actions during the file system mounting process by analyzing self-journal records rather than performing extensive post-unmount checks. The distributed self-journal records enable quick identification and recovery of incomplete operations, allowing the file system to be mounted rapidly while maintaining consistency through targeted recovery based on pre-analyzed journal data.
4Reliability
If synchronous journal logging is implemented for every file operation, then crash immunity is ensured, but the journal synchronization becomes a performance bottleneck
Solution Approach 1:
The synchronous centralized journal logging is segmented into distributed asynchronous self-journal logging for each file. This segmentation allows multiple file operations to be logged concurrently without requiring synchronous coordination, as each file's self-journal record can be updated independently by different threads simultaneously, maintaining crash immunity through distributed logging while eliminating the synchronization bottleneck.
Data Source
AI summary
A persistent memory based distributed-journal file system, comprising:1) Direct mapped persistent memory unit hosting at least a subset of a file system which contains a plurality of files each associated with a metadata record and a self-journal record.2) A memory storing a program code.3) One or more processors coupled to the persistent memory unit and the memory for executing the program code which comprises:(a) Creating a resources dataset which maps characteristics of the file system by analyzing the files' metadata records.(b) Receiving an alteration request to alter altered file(s) of the plurality of files.(c) Determining if the alteration is an atomic alteration which is applied to the file system in an atomic operation.(d) Logging an indication of the alteration in the respective self-journal record of the altered file(s) for non-atomic alteration.(e) Applying the alteration in the file system.