Archiving Deduplicated Cloud Data by Segment Access Patterns
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transferring deduplicated data to an archival tier in cloud storage is challenging due to shared segments across multiple files and varying access times, making it difficult for cloud storage providers to automatically move objects based on access or creation time, as some segments may remain accessible while others are inactive, leading to inefficient cost management and potential duplication in archival storage.
Innovation Solution
A method that automatically transfers deduplicated data from a standard storage class to a cost-effective archival storage class based on access patterns of segments, using metadata to track last referenced and accessed times, ensuring only inactive segments are moved, and preventing duplication in the archival tier, thereby controlling costs and optimizing storage utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If cloud storage providers automatically move objects to archival tier based on access or creation time, then storage cost is reduced, but deduplicated data with shared segments cannot be properly managed leading to potential duplication and inefficiency
Solution Approach 1:
The patent segments deduplicated data into individual segments that can be independently tracked and managed. Each segment within a cloud object is monitored separately for access patterns, allowing the system to determine when entire objects are eligible for archival based on all their segments being inactive, rather than managing the object as a single unit. This resolves the contradiction by enabling cost-effective archival while maintaining data management accuracy through granular control.
Solution Approach 2:
The patent implements a feedback mechanism that continuously monitors access patterns of deduplicated segments and uses this information to dynamically determine archival eligibility. The system tracks when segments are last accessed and only moves cloud objects to archival tier when all their segments meet the inactivity threshold, preventing premature archival of actively used data. This feedback loop ensures storage cost reduction without compromising data management reliability.
2Ease of operation
If all segments of a cloud object are moved to archival tier together, then data management is simplified, but actively accessed segments are incorrectly archived leading to accessibility issues
Solution Approach 1:
The patent divides the cloud object into individually trackable segments and implements separate access pattern monitoring for each segment. This segmentation allows the system to identify when all segments of an object are inactive before initiating archival, providing simple automated management while ensuring actively accessed segments remain accessible. The approach maintains ease of operation through automated decision-making while preserving data accessibility through granular segment-level tracking.
Solution Approach 2:
The system enables cloud objects to effectively self-determine their archival eligibility based on their own segment access patterns. Each cloud object is automatically evaluated against archival criteria by monitoring its segments' activity, and the decision to archive or retain is made autonomously without manual intervention. This self-service mechanism simplifies data management while ensuring reliability, as objects are only archived when their own segments indicate inactivity.
3Device complexity
If segments are tracked at cloud object level, then management overhead is reduced, but precision in determining archival eligibility is insufficient leading to either premature archival or missed archival opportunities
Solution Approach 1:
The patent segments the tracking and management process into individual segment-level operations. Instead of treating cloud objects as monolithic units, the system tracks each segment's access patterns separately, providing precise measurement of actual data usage. This segmentation enables accurate determination of archival eligibility by evaluating each segment independently, then aggregating results to make informed decisions about entire cloud objects, thereby improving measurement precision without excessive complexity.
Solution Approach 2:
The patent implements tracking at the segment level (more granular than the cloud object level) to achieve better precision in archival eligibility determination. By monitoring segments individually rather than treating the entire object as a single unit, the system captures more detailed access pattern information. This partial action approach (tracking only necessary segments) provides the precision needed to accurately determine when objects are ready for archival, avoiding both premature and missed archival opportunities.
Data Source
AI summary
A method, apparatus, and system manages an object in a storage provider that provides a plurality of storage classes of storage. The method may include receiving a request for transfer of the object comprising segments from a first storage class to a second storage class of the storage provider; storing metadata associated with the object; determining that each of the segments satisfies a data storage policy based on the access characteristics of each of the segments within the object, wherein the determining includes: determining that none of the segments are referenced by the new segment for a predetermined time period; determining that none of the segments are accessed for the predetermined time period from the last accessed time instance; and in response to determining that each of the segments satisfies the data storage policy, transferring the object from the first storage class to the second storage class of the storage provider.