File System Deduplication Headers for Application Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
File systems do not effectively share information about duplicate file portions, leading to unnecessary processing and resource wastage when applications access files with duplicate content, as they fail to track and utilize deduplicated data across multiple requests.
Innovation Solution
Creating and maintaining data structures with headers that identify duplicate file portions, allowing the file system to provide this information to applications, enabling them to skip processing already deduplicated content and reducing redundant reads and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If file systems perform deduplication internally to reduce storage requirements, then storage space is reduced, but applications cannot access deduplication information and must perform redundant reads and processing
Solution Approach 1:
The patent introduces data structures (headers, mapping information) as intermediaries that carry deduplication information between the file system and applications. These data structures act as a mediator that enables applications to access deduplication information without compromising the internal deduplication mechanism, thus resolving the information loss problem while maintaining storage efficiency
Solution Approach 2:
The patent segments deduplication information into separate data structures (headers associated with file portions) that can be independently provided to applications. This segmentation allows the file system to maintain internal deduplication while externally exposing only the necessary information about duplicate portions, enabling applications to make informed processing decisions
2Ease of operation
If applications read and process all file portions without deduplication information, then processing is simple, but processing power and time are wasted on duplicate content
Solution Approach 1:
The patent implements preliminary action by having the file system pre-compute and store deduplication information in data structures before applications request files. This advance preparation allows applications to quickly identify and skip duplicate portions without performing complex analysis during processing, thus reducing processing time while maintaining operational simplicity
Solution Approach 2:
The patent establishes a feedback mechanism where the file system provides deduplication information (headers mapping file portions to content) back to applications. This feedback enables applications to adjust their processing behavior by skipping already-deduplicated content, thereby reducing wasted processing time while keeping the operation model simple
3Ease of operation
If multiple copies of duplicate content are maintained, then application access is straightforward, but disk usage and storage consumption increase
Solution Approach 1:
The patent uses virtual copying through data structures rather than physical duplication. The headers create virtual copies of content references that allow applications to access deduplicated content efficiently without actually storing multiple physical copies, thus maintaining ease of access while minimizing disk usage
Data Source
AI summary
A method of maintaining and providing information relating to file deduplication. A first portion of a first file and a second portion of a second file that contain a first content are identified. A first header associated with the first portion is created. The first header identifies the first portion and the second portion containing the first content. The first header is appended to a storage location of the first content of the first portion to form a first data structure for the first file. The first data structure is stored. The first data structure is provided to an application requesting the first file so that duplicate data processing can be avoided by the application. The first data structure is updated when the first file or the second file are altered. A similar process may occur to generate a data structure for the second file.