Data De-duplication via Pointer Redirection in Thin Provisioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems lack efficient automatic management of data throughout its lifecycle, leading to inefficient storage and accessibility issues due to changing data value and access frequency, as well as unnecessary storage of duplicate data.
Innovation Solution
A method and system for de-duplicating data by using pointers to map logical volumes to physical storage, where duplicate data sets are identified and stored only once, with pointers updated to reference the shared data chunk, and physical storage space is allocated and deallocated accordingly, allowing for efficient storage and access based on data value and frequency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple copies of data are stored to ensure accessibility and availability, then data reliability is improved, but storage space is wasted due to duplication
Solution Approach 1:
Multiple logical volumes that contain duplicate data are merged by having their pointers reference the same physical storage location. When data is written to one logical volume, the same physical location serves all logical volumes, eliminating redundant storage while maintaining data availability across multiple volumes.
Solution Approach 2:
A single physical storage location serves multiple logical volumes simultaneously. The same physical data chunk can be accessed by multiple logical volume pointers, allowing one physical storage unit to fulfill the storage needs of multiple logical volumes and eliminating the need for separate physical copies.
2Speed
If data is stored on fast and expensive storage devices for immediate accessibility, then data access speed is improved, but storage cost increases
Solution Approach 1:
The storage system dynamically allocates data between different storage tiers based on access patterns. Frequently accessed data is automatically placed on fast storage devices, while less frequently accessed data is moved to slower, cheaper storage devices. The pointer redirection mechanism enables this dynamic reallocation without requiring manual intervention.
Solution Approach 2:
The system changes the storage location parameter of data based on access frequency. By monitoring how often data is accessed and redirecting pointers to appropriate storage tiers, the system adapts the physical storage location to match the access requirements, optimizing both speed and cost.
3Productivity
If thin provisioning is used to allocate physical storage space on demand, then storage efficiency is improved, but data de-duplication capability deteriorates
Solution Approach 1:
The storage system segments data into fixed-size chunks and maintains a chunk map that tracks the physical location of each data chunk. This segmentation approach allows the system to efficiently identify and redirect duplicate data chunks while working within the thin provisioning framework, where physical space is allocated only when needed.
Solution Approach 2:
A pointer redirection mechanism acts as an intermediary between logical volume pointers and physical storage locations. This intermediary layer enables de-duplication by intercepting write operations and redirecting pointers to existing data chunks, while maintaining compatibility with thin provisioning's on-demand allocation model.
Data Source
AI summary
A system for de-duplicating data includes providing a first volume including at least one pointer to a second volume that corresponds to physical storage space, wherein the first volume is a logical volume. A first set of data is detected as a duplicate of a second set of data stored on the second volume at a first data chunk. A pointer of the first volume associated with the first set of data is modified to point to the first data chunk. After modifying the pointer, no additional physical storage space is allocated for the first set of data.