Deduplication Level Assignment for I/O Requests
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the field of data deduplication, storage service providers face challenges in efficiently managing I/O requests and reducing storage consumption, particularly due to increased access operations and the need to differentiate between compressed and uncompressed data blocks to optimize deduplication processes.
Innovation Solution
A method and device that assign a deduplication level to I/O requests based on file stream properties, such as compression status, to determine the appropriate deduplication approach, and utilize a cache to store fingerprints for quick identification of subsequent requests, thereby optimizing resource usage and deduplication efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deduplication is performed on all data blocks regardless of compression status, then deduplication coverage is improved, but computing resource consumption increases
Solution Approach 1:
The patent applies local quality by differentiating deduplication processing based on file stream properties. Compressed data blocks are identified through property inspection (magic numbers, file types) and assigned a first deduplication level that skips actual deduplication, while uncompressed blocks receive a third deduplication level that performs full deduplication. This localized differentiation optimizes computing resource usage while maintaining effective deduplication coverage where needed.
Solution Approach 2:
The patent implements preliminary action by inspecting file stream properties before performing deduplication. The system determines compression status, assigns appropriate deduplication levels, and makes skip decisions in advance based on these properties. This preliminary classification prevents wasted computing resources on compressed blocks that cannot be deduplicated, while ensuring uncompressed blocks receive proper deduplication processing.
2Use of energy by moving object
If deduplication level assignment is based on file stream properties, then computing resources are saved, but processing complexity increases
Solution Approach 1:
The patent reduces processing complexity during actual deduplication by performing preliminary property inspection and deduplication level assignment before the deduplication process. The system extracts file stream properties, determines compression status, and assigns deduplication levels in advance. This preliminary action simplifies the main deduplication loop to only need to check the assigned level and either skip or process accordingly, rather than analyzing each block in detail.
Solution Approach 2:
The patent introduces an intermediary deduplication level assignment mechanism that acts as a mediator between property inspection and deduplication execution. The assigned deduplication level serves as an intermediary decision variable that simplifies the control logic, allowing the system to handle different file types and compression states through a unified level-based framework rather than complex conditional processing.
3Speed
If I/O requests are processed without differentiation, then processing speed is maintained, but storage consumption increases
Solution Approach 1:
The patent applies local quality by differentiating I/O request processing based on file stream properties and assigned deduplication levels. The system inspects properties locally for each file stream, assigns appropriate deduplication levels, and applies different processing strategies accordingly. This localized differentiation reduces storage consumption by avoiding duplication of compressed blocks while maintaining processing speed through efficient property-based classification.
Data Source
AI summary
Embodiments of the present disclosure provide a method and device for deduplication. Specifically, the method may comprise obtaining a property of a file stream, the property of a file stream including a file type or a magic number identifying a format of a protocol or a file. The method further includes in response to receiving an I/O request for a data block of the file stream, assigning a deduplication level to the I/O request based on the property of the file stream. Moreover, the method further includes deduplicating the data block of the file stream based on the deduplication level assigned to the I/O request. In addition, a corresponding device and computer program product are provided.