File System Cache Deduplication for Shared Block Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern file systems face inefficiencies due to duplicate I/O requests and content in page caches when multiple files access shared data blocks, leading to issues like 'boot storms' and increased storage costs, especially in environments with virtual machine images and file system checkpoints.
Innovation Solution
Implementing a cache management system that uses deduplication information to identify shared data blocks and maintain a cache configured to store these blocks, allowing the operating system to service requests using shared file identifiers without accessing storage devices, thereby reducing duplicate requests and content in the cache.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the file system processes each file access request independently without checking for shared blocks, then the simplicity of the access process is maintained, but the storage device capacity is burdened by duplicate requests for the same data blocks
Solution Approach 1:
The system performs preliminary deduplication analysis before processing file access requests. By pre-identifying shared data blocks and creating a deduplication information data structure that maps shared blocks to multiple file identifiers, the system prepares the groundwork for efficient request handling. When a request arrives, the system checks this pre-prepared information to determine if the requested block is shared, avoiding redundant storage device accesses.
Solution Approach 2:
The patent introduces an intermediary cache structure that sits between the file system and the storage device. This cache stores deduplication information including shared block identifiers and their associated file identifiers. When a file access request is received, the system queries this intermediary cache to check if the requested block is shared, and if so, retrieves it from the cache rather than accessing the storage device directly, thus reducing storage device burden.
2Productivity
If the system sends multiple requests to the storage device for the same shared data block accessed through different file names, then the file system maintains simple access logic, but the system performance deteriorates due to repeated I/O operations
Solution Approach 1:
The system performs preliminary deduplication analysis before processing file access requests. By pre-identifying shared data blocks and creating a deduplication information data structure that maps shared blocks to multiple file identifiers, the system prepares the groundwork for efficient request handling. When a request arrives, the system checks this pre-prepared information to determine if the requested block is shared, avoiding redundant storage device accesses.
Solution Approach 2:
The patent uses different identifiers (file identifiers versus shared block identifiers) to distinguish between unique file access and shared block access. By tagging data blocks with shared block identifiers and maintaining a mapping between file identifiers and shared block identifiers, the system can quickly recognize when multiple files reference the same underlying data, thereby optimizing the access path to avoid redundant I/O operations.
3Speed
If the cache stores duplicate copies of shared data blocks for different files, then the file access speed is improved, but the cache memory usage increases unnecessarily
Solution Approach 1:
The patent merges multiple references to the same shared data block into a single cache entry. By using shared block identifiers to index cache entries and maintaining a mapping between file identifiers and shared block identifiers, the system ensures that when a shared block is loaded into cache, it is stored once with the shared block identifier. Subsequent requests for the same block through different files are served from the same cache entry, eliminating duplicate storage while maintaining fast access.
Solution Approach 2:
The patent makes the cache universally applicable to multiple files by using shared block identifiers instead of file-specific identifiers. A single cache entry with a shared block identifier can serve multiple files that reference the same underlying data block. This multi-functional cache structure allows the system to service requests from different files through a unified cache interface, maximizing cache utility while minimizing memory consumption.
Data Source
AI summary
Techniques relating to data deduplication at the file system level are disclosed. A system may include a block cache configured to store data blocks indicated as being shared by multiple files. The system may be coupled to a storage configured to permanently store data for the multiple files. The computer system may receive a file request, and, upon determining that the file request includes shared data, the computer system may service the request wholly or in part using the block cache. When multiple requests including multiple file identifiers (but referring to the same underlying shared data) are received, a shared (common) file identifier may be used in looking up data for the multiple requests, reducing block cache storage, and improving block cache performance and overall system operation.