Pre-allocating Data Object Replicas to Reduce Distributed File Creation Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed file sharing systems, creating a new file involves high latency due to synchronous operations in replicating files across multiple nodes, such as selecting and creating replicas, which can delay client application responses.
Innovation Solution
Implementing a method where a node pre-allocates replicas for data objects, allowing it to satisfy creation requests without synchronous replica allocation, by selecting nodes with sufficient free space and creating empty replicas in advance, which can then be used to reduce latency in data object creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If replicas are allocated synchronously when a client requests new file creation, then the file system ensures data redundancy and availability, but the client application experiences high latency
Solution Approach 1:
The system pre-allocates empty replica structures and reserves disk space on target nodes before actual file creation requests are made. This preliminary preparation of replica containers allows the system to quickly instantiate files by simply copying data to pre-reserved locations, rather than performing allocation operations during the critical file creation path.
2Reliability
If multiple operations are performed to create file replicas (selecting nodes, creating replicas, linking replicas), then the system maintains proper file structure and redundancy, but the creation process becomes complex and time-consuming
Solution Approach 1:
The patent separates the file creation process into distinct phases: pre-allocation of replica structures (infrastructure preparation) and actual file instantiation (data population). By segmenting these operations, the system can prepare the replica framework in advance without blocking the critical file creation path, reducing both complexity and latency.
Solution Approach 2:
The system introduces pre-allocated empty replica structures as intermediaries between the file creation request and the actual data storage. These pre-created placeholder structures serve as intermediaries that absorb the complexity of node selection and replica setup, allowing the client to experience a simple, fast file creation operation while the background processes handle the complex replica management.
3Loss of time
If replicas are pre-allocated in advance, then the latency for file creation is decreased, but disk space must be reserved before actual need
Solution Approach 1:
The system performs partial pre-allocation by reserving only the metadata structures and disk space allocations necessary to enable fast file creation, rather than fully populating replicas in advance. This partial preparation approach reduces the disk space burden while still achieving the latency reduction benefit, as the actual file data is only copied to pre-allocated structures when needed.
Data Source
AI summary
A system and method for pre-allocating replicas for a distributed file sharing system. Creating a new file may involve creating a plurality of replicas for the file on a plurality of nodes. In one embodiment nodes in the system may pre-allocate sets of file replicas, where the pre-allocated replicas can be used to satisfy requests to create new files. Pre-allocating the file replicas may decrease the latency of file creation requests by enabling the requests to be satisfied without performing replica allocation in response to the requests.