Lazy Linking File Creation Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high latency associated with file creation in file systems, particularly exacerbated in network file systems and multi-tiered non-volatile memory integrated file server stacks, due to the need for synchronous block commits and higher read/write latencies in storage devices compared to non-volatile memory.
Innovation Solution
Implementing a method of lazy linking, where pre-created files and inodes are used from a pool, with the inode being initially linked to a hidden namespace and only subsequently linked to the target namespace upon occurrence of a predefined event, such as system recovery or intent log overflow, to defer the high-latency linking process and reduce overall file creation latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous block commits are performed during file creation in network file systems, then data consistency is ensured, but file creation latency increases significantly
Solution Approach 1:
The system pre-creates files and their associated inodes in advance, storing them in a pool before they are actually needed. When a file creation request arrives, the system can immediately allocate a pre-created file from the pool without performing synchronous block commits at that moment, thus reducing latency while maintaining data consistency through later commitment mechanisms
Solution Approach 2:
The patent introduces an intent log as an intermediary mechanism that records file creation intentions before actual file allocation. This intent log allows the system to defer the actual synchronous block commit operation to a later time when it can be performed more efficiently, separating the immediate response from the consistency-critical operation
2Stability of the object's composition
If file creation operations are performed using traditional file system methods, then proper file system structure is maintained, but the process involves high latency due to multiple synchronous operations
Solution Approach 1:
Files and inodes are pre-created and stored in a pool before actual use. This preliminary action maintains proper file system structure in advance, allowing rapid allocation when needed without performing multiple synchronous operations at the moment of file creation request
Solution Approach 2:
The file creation process is segmented into distinct phases: pre-creation of files in a pool, allocation from the pool upon request, and subsequent lazy linking. This segmentation allows the structure-maintaining operations to be performed in advance while the allocation phase operates rapidly
3Ease of operation
If inodes are immediately linked to target namespaces during file creation, then file accessibility is ensured, but the linking process contributes to overall creation latency
Solution Approach 1:
Inodes are pre-created and prepared in advance with their basic structure, but the actual linking to target namespaces is deferred. This preliminary preparation ensures inodes are ready for allocation while postponing the time-consuming linking operation to a later lazy linking phase
Solution Approach 2:
The system dynamically determines when linking should occur based on actual file creation requests. Instead of immediately linking all inodes, the system keeps them in a pool and performs linking dynamically at the point of use, optimizing the balance between accessibility and latency
Data Source
AI summary
The present subject matter relates to file creation. In one example a user request for creating a new file is received. An inode corresponding to a pre-created file is retrieved from a pre-created pool of files in response to receiving the user request. Further, the inode is associated with an in-memory dentry corresponding to the new file. Further, a log entry is created in an intent log to indicate intent of lazy linking of the pre-created file with a target directory in which the new file is to be created. Further, lazy linking of the pre-created file with the target directory is performed using a file system link operation upon occurrence of a predefined event.