Write Request Tagging for Inline Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deduplication methods in data storage systems often result in high disk access and over-provisioning of storage space, especially when busy with critical data storage functions, and not all processing paths support inline deduplication.
Innovation Solution
The technique selectively tags write requests for copy operations and directs them through a processing path that supports inline deduplication, avoiding the creation of duplicate copies by using a digest cache to store and retrieve data efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If background deduplication is used, then storage space is saved through duplicate removal, but disk access increases and critical storage functions are interfered with
Solution Approach 1:
The patent applies preliminary action by performing deduplication checks at the time of write request processing rather than waiting for background processes. The system checks if data blocks are duplicates when they arrive at the storage system, and if so, skips writing them to the data log, thereby preventing duplicate storage from the outset without requiring subsequent background deduplication operations.
2Quantity of substance
If background deduplication is used, then duplicate data is removed, but storage space is over-provisioned during initial writes
Solution Approach 1:
The system performs deduplication determination preliminarily at write time by checking if incoming data blocks are duplicates before they are written to storage. This preliminary check prevents the creation of duplicate copies during initial writes, eliminating the need for subsequent background deduplication and avoiding storage space over-provisioning.
Solution Approach 2:
The patent ensures continuous deduplication action by integrating the deduplication check into the normal write request processing flow. The system continuously monitors incoming write requests and performs deduplication checks as part of the standard processing sequence, maintaining consistent deduplication behavior without interruption or delay.
3Adaptability or versatility
If multiple processing paths are used for write requests, then system flexibility is improved, but not all paths support inline deduplication
Solution Approach 1:
The patent applies local quality by making different processing paths have different deduplication capabilities. The first processing path is configured to support inline deduplication by performing duplicate checks before writing to the data log, while the second processing path does not support deduplication. This allows the system to optimize each path according to its specific function and requirements.
Solution Approach 2:
The system segments write request processing into distinct paths with different characteristics. The first processing path is dedicated to operations that benefit from inline deduplication (such as copy operations), while the second path handles operations where deduplication is not needed or is less critical. This segmentation allows each path to be optimized independently.
Data Source
AI summary
A storage technique includes selectively tagging a set of write requests by applying tags to write requests that are pursuant to copy operations and not applying tags to at least some write requests that are not pursuant to copy operations. For each of the set of write requests, the technique further includes performing a decision operation configured to (i) direct the write request down a first processing path that supports inline deduplication in response to the write request satisfying a set of criteria and (ii) direct the write request down a second processing path that does not support inline deduplication in response to the write request failing to satisfy the set of criteria. The set of criteria include whether the write request has a tag applied, such that the decision operation preferentially directs write requests down the first processing path in response to the write requests being tagged.