Data Reduction Assessment for Compression-Deduplication Overlap
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems lack an efficient method to determine the optimal data reduction techniques, such as compression and deduplication, to maximize storage savings, often resulting in overlapping benefits that do not significantly increase overall reduction when both methods are used together.
Innovation Solution
A method is developed to assess data reduction statistics, including compression and deduplication ratios, to determine the most effective data reduction option by calculating the overlap and unique savings, allowing for automatic or user-driven selection of the best option among enabling only compression, only deduplication, both, or neither, and presenting this information graphically to users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If both compression and deduplication are enabled simultaneously, then data reduction is performed, but the benefits overlap and do not significantly increase overall reduction
Solution Approach 1:
The system calculates data reduction statistics for partial actions (compression only, deduplication only) and compares them with the expected excessive action (both together). By computing the overlap statistic, the system determines whether the excessive action provides sufficient additional benefit to justify its complexity, applying partial action when overlap is high and excessive action when overlap is low.
2Quantity of substance
If data reduction statistics are calculated and analyzed, then optimal data reduction options can be determined, but additional processing time and CPU cycles are consumed
Solution Approach 1:
The system performs preliminary calculations of data reduction statistics (compression ratio, deduplication ratio, and overlap) before committing to a data reduction strategy. This preliminary analysis allows the system to predict the effectiveness of different approaches and select the optimal one in advance, avoiding wasted processing time on ineffective reduction methods.
Solution Approach 2:
The system uses calculated statistics as feedback to determine the optimal data reduction configuration. By continuously monitoring compression ratios, deduplication ratios, and overlap metrics, the system can adaptively adjust its data reduction strategy to maximize storage savings while minimizing processing overhead.
3Productivity
If the system automatically determines the best data reduction option, then storage efficiency is maximized, but user control and decision-making are reduced
Solution Approach 1:
The system performs self-service by automatically calculating data reduction statistics and determining the optimal configuration without requiring user intervention. It computes compression ratios, deduplication ratios, and overlap metrics, then autonomously selects the best data reduction option, freeing users from complex decision-making while maximizing storage efficiency.
Solution Approach 2:
The system acts as an intermediary between raw data and storage allocation by introducing a layer of statistical analysis and decision logic. This intermediary layer calculates reduction statistics and determines optimal configurations, mediating between user storage needs and system efficiency goals without requiring direct user control over the technical details.
Data Source
AI summary
Techniques for determining data reduction options may include: receiving data reduction statistics for a data set including a first value of a first statistic denoting an amount of data reduction obtained for the data set when compression is enabled, a second value of a second statistic denoting an amount of data reduction obtained for the data set when deduplication is enabled, and a third value of a third statistic denoting an overlap in data reduction contribution when both compression and deduplication are enabled; and determining, in accordance with the data reduction statistics, a first setting denoting a current data reduction option enabled for the data set. A Venn diagram provided on a user interface display may illustrate data reduction benefits for the data set based on the data reduction statistics. Data reduction benefits for the data set may be reassessed to determine whether to modify the current data reduction option.