Adaptive Data De-duplication for Low Latency Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data de-duplication technologies are not suitable for portable devices like notebook computers and smartphones due to high input/output latency issues caused by complex algorithms, which degrade data input/output speed.
Innovation Solution
A method and apparatus for adaptive data de-duplication that determines the de-duplication unit based on access properties such as access time, modification time, sequential, and random access counts, using hash algorithms to generate unique identifiers and manage data blocks, thereby optimizing de-duplication rates for low latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If complex data de-duplication algorithms are applied to portable devices, then data de-duplication rate is improved, but data input/output speed deteriorates significantly
Solution Approach 1:
The patent applies dynamics by making the de-duplication unit size adjustable and adaptive rather than fixed. The system dynamically selects de-duplication unit sizes based on data characteristics and access patterns, allowing optimization between de-duplication rate and I/O speed for different scenarios. This resolves the contradiction by enabling the system to adapt its behavior rather than being constrained by a single algorithmic approach.
Solution Approach 2:
The patent changes the parameter of de-duplication unit size to resolve the contradiction. By varying the unit size parameter according to data types and access patterns, the system can achieve high de-duplication rates for suitable data while maintaining fast I/O speed for time-sensitive operations. This parameter adjustment allows the system to optimize performance based on specific conditions rather than applying a fixed complex algorithm.
2Device complexity
If fixed de-duplication unit size is used, then processing simplicity is improved, but adaptability to different data characteristics deteriorates
Solution Approach 1:
The system transitions from a static fixed unit size to a dynamic adaptive unit size that changes based on data characteristics. The de-duplication unit size is determined adaptively according to access patterns and data properties, allowing the system to maintain simplicity in implementation while achieving high adaptability to different data types and access scenarios.
3Productivity
If large de-duplication unit size is used, then processing speed is improved, but de-duplication rate deteriorates
Solution Approach 1:
The patent resolves this contradiction by making the de-duplication unit size a variable parameter rather than a fixed value. The system adjusts the unit size parameter based on data characteristics and access patterns, allowing it to use larger units for high-speed processing when appropriate while using smaller units to maximize de-duplication rates when beneficial. This parameter adaptation enables optimization of both processing speed and de-duplication rate for different scenarios.
Data Source
AI summary
Disclosed are a method for data de-duplication and an apparatus for the same. The method may comprise obtaining access property of data based on input request or output request for the data, determining de-duplication unit of the data based on the access property, and performing de-duplication on the data based on the de-duplication unit. Thus, data de-duplication rate may be determined adaptively based on input/output characteristics of data. Also, data de-duplication may be performed based on the determined data de-duplication rate so as to provide low input/output latency.