Distributed File System Offset Modification for Contiguous Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional disk file systems and network file systems face inefficiencies in storage space utilization and read performance when striping files across multiple storage servers, as they often result in unused preallocated space and unnecessary read ahead operations due to non-contiguous data portions.
Innovation Solution
A distributed file system method that modifies the offsets of file portions before writing them to storage servers, ensuring contiguous arrangement within file entries, thereby optimizing storage space and read performance by using a file striping module to divide and distribute files across multiple storage servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional disk file systems stripe files across multiple storage servers, then storage capacity is increased, but storage space utilization deteriorates due to unused preallocated space
Solution Approach 1:
The patent segments files into multiple portions and distributes them across different storage servers. By modifying offsets to ensure contiguous arrangement of file portions within each server's file entry, the system enables efficient use of preallocated space while maintaining the benefits of distributed storage capacity.
2Quantity of substance
If traditional network file systems stripe files across multiple storage servers, then storage capacity is increased, but read performance deteriorates due to non-contiguous data portions
Solution Approach 1:
The patent segments files into multiple portions distributed across storage servers, then applies offset modification to ensure contiguous arrangement within each server's file entry. This segmentation approach maintains distributed storage capacity while improving read performance through contiguous data layout.
Solution Approach 2:
The patent performs preliminary offset modification before writing file portions to storage servers. By pre-adjusting offsets to ensure contiguous arrangement, the system eliminates the need for read-ahead operations and improves read performance without sacrificing distributed storage capacity.
3Quantity of substance
If traditional file systems allocate space for striped files, then storage capacity is increased, but write performance deteriorates due to inefficient preallocation usage
Solution Approach 1:
The patent segments files into portions and distributes them across storage servers with modified offsets. This segmentation enables efficient preallocation usage on each server, improving write performance while maintaining increased storage capacity through distribution.
Solution Approach 2:
The patent performs preliminary offset modification before file writing operations. By pre-configuring contiguous offsets, the system enables efficient preallocation usage during write operations, improving write performance without compromising distributed storage capacity.
Data Source
AI summary
A processing device executing a client of a distributed file system receives an initial file, wherein the distributed file system comprises a plurality of nodes, each of the plurality of nodes comprising a local file system that interfaces with the distributed file system. The processing device causes the plurality of nodes to generate file entries of the initial file. The processing device divides the initial file into a plurality of portions, each of the plurality of portions having initial offsets within the initial file. The processing device assigns subsets of the plurality of portions to the plurality of nodes, wherein each of the plurality of nodes is assigned a different subset. The processing device causes the plurality of nodes to write the portions of the assigned subsets at modified offsets within the file entries, wherein the modified offsets cause the portions of the assigned subsets to be contiguous.