File System Deduplication via Evaluation-Based Re-chunking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file systems with deduplication features generate inefficient stub chunks during re-chunking, leading to reduced deduplication efficiency and wasted storage space, especially when file contents are modified.
Innovation Solution
A method that defines a calculation range within a file based on chunk types, generates an evaluation value to determine whether to re-chunk and deduplicate marked chunks, thereby preventing inefficient re-chunking and reducing stub chunks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If re-chunking is performed on modified chunks, then the file system can handle content changes, but stub chunks are generated that reduce deduplication efficiency and waste storage space
Solution Approach 1:
The patent segments the file into chunks and further divides modified chunks into sub-chunks during re-chunking. This segmentation allows the system to identify and handle only the modified portions rather than treating entire chunks as stubs, thereby reducing storage space waste while maintaining adaptability to content changes.
Solution Approach 2:
The patent applies different processing strategies to different parts of the file based on their characteristics. Modified chunks are re-chunked into sub-chunks with evaluation values calculated for each, while unmodified chunks are handled differently. This local quality approach ensures that re-chunking operations are applied selectively to minimize stub chunk generation while still handling modifications effectively.
2Adaptability or versatility
If re-chunking is performed on modified chunks, then content changes can be processed, but deduplication efficiency is reduced due to stub chunk generation
Solution Approach 1:
The patent performs preliminary evaluation of modified chunks by calculating evaluation values for each sub-chunk before finalizing the re-chunking process. This preliminary action allows the system to identify which sub-chunks are likely to duplicate existing chunks, enabling proactive decisions that maintain deduplication efficiency while still processing content modifications.
Solution Approach 2:
The patent uses evaluation values as feedback to guide the re-chunking process. By calculating and comparing evaluation values of sub-chunks against existing chunks, the system receives feedback on potential duplications and adjusts its re-chunking strategy accordingly, thereby maintaining high deduplication efficiency while handling content changes.
3Adaptability or versatility
If conventional re-chunking is performed, then modified chunks can be handled, but storage space is wasted and deduplication effect is poor
Solution Approach 1:
The patent segments modified chunks into smaller sub-chunks during re-chunking, allowing for more granular analysis and comparison with existing chunks. This segmentation increases the likelihood of finding exact matches for duplication, thereby improving storage space utilization while maintaining the ability to handle modified chunks.
Solution Approach 2:
The patent changes the parameter of chunk size during re-chunking by dividing modified chunks into sub-chunks of appropriate sizes based on evaluation values. This parameter change optimizes the balance between handling modifications and maximizing deduplication opportunities, thereby improving storage space utilization without sacrificing adaptability.
Data Source
AI summary
A method for deduplication applicable to a file chunked into a plurality of deduplicated chunks is provided and includes: defining a calculation range in the file according to types of the chunks in the file, where the calculation range includes a plurality of consecutive chunks in the file; generating an evaluation value according to the types of the chunks in the calculation range to determine whether to mark the chunks in the calculation range; and re-chunking and deduplicating the marked chunks in the file. A computer-readable medium and a file system corresponding to the method for deduplication are also provided.