Storage Device Deduplication and Compression Grain Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage technologies require the deduplication and compression grains to be the same, leading to limitations such as decreased deduplication ratios due to large grains and decreased compression ratios due to small grains.
Innovation Solution
The method involves performing deduplication based on a first grain and compression based on a second grain, where the size of the second grain is greater than the size of the first grain, allowing for different grains to be used for each process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large grain is used for deduplication, then deduplication coverage is improved, but deduplication ratio decreases
Solution Approach 1:
The patent segments the grain size parameter into two distinct values: a first grain size for deduplication and a second grain size for compression. This segmentation allows each process to operate at its optimal grain size independently, resolving the contradiction between deduplication coverage and deduplication ratio.
Solution Approach 2:
The patent changes the parameter of grain size from a single fixed value to two different values (first grain size and second grain size) for different processes. This parameter change enables deduplication to use a larger grain for better coverage while compression uses a smaller grain for better ratio.
2Quantity of substance
If a small grain is used for compression, then compression ratio is improved, but compression overhead increases
Solution Approach 1:
The patent segments the grain size parameter into two distinct values: a first grain size for deduplication and a second grain size for compression. This segmentation allows compression to use an optimally small grain size for better compression ratio without being constrained by deduplication requirements.
Solution Approach 2:
The patent performs deduplication as a preliminary action before compression. By removing duplicates first at a larger grain size, the amount of data requiring compression is reduced, thereby lowering the compression overhead despite using a smaller compression grain size.
3Quantity of substance
If different grains are used for deduplication and compression, then overall reduction ratio is improved, but system complexity increases
Solution Approach 1:
The patent introduces a parameter change from a single grain size to two grain sizes (first and second grain sizes) for different processes. This controlled parameter change improves overall reduction ratio while maintaining manageable system complexity through clear separation of concerns.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides a data processing method and a storage device, and belongs to the field of storage technologies. In this application, the storage device performs deduplication and compression based on different grains, deduplicates data based on a large grain, and compresses the data based on a small grain. Therefore, a limitation that a deduplication grain and a compression grain need to be the same is removed. A deduplication ratio decrease caused by an excessively large grain and a compression ratio decrease caused by an excessively small grain are avoided to some extent, to improve an overall reduction ratio of deduplication and compression.