Time-Based Data Retirement in De-Duplicated Archival Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data backup and archiving methods are resource-intensive and costly due to the need for multiple stages of data storage, leading to increased complexity and difficulty in locating and deleting archived data, especially when using offline and unsearchable indexed-based archive storage systems.
Innovation Solution
A system and method for time-based data retirement in de-duplicated archival storage, where data snapshots are stored with metadata, duplicates are eliminated through deduplication, and archive storage locations are linked to backup references, allowing for efficient storage and deletion of data based on expiration times, reducing the need for duplicate storage and computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is backed up to multiple storage locations across multiple stages, then data protection and retention are improved, but storage costs and system complexity increase
Solution Approach 1:
The patent combines multiple backup stages into a unified deduplicated storage system where data is stored once and referenced multiple times across different retention periods. The archive storage system merges backup data and archive data into a single deduplicated repository, eliminating the need for separate storage locations and reducing system complexity while maintaining data protection across multiple stages.
Solution Approach 2:
The archive storage location serves multiple functions simultaneously: it stores both active backup data and archived data, provides deduplication services for both, and supports multiple retention policies. This multi-functional approach eliminates the need for separate specialized storage systems for backup and archive operations.
2Quantity of substance
If data is archived in offline storage, then storage costs are reduced, but data retrieval and deletion operations become more difficult and resource-intensive
Solution Approach 1:
The system pre-computes and stores metadata indices for archived data during the archiving process, including information about data locations, retention periods, and deduplication relationships. This preliminary indexing enables efficient retrieval and deletion operations without requiring brute-force scanning of archived data later.
Solution Approach 2:
The patent introduces a catalogue and metadata layer as an intermediary between the offline archive storage and the user/system. This catalogue stores references and indices that enable efficient location and management of archived data without requiring direct access to or scanning of the actual archived data, thus maintaining ease of operation while using cost-effective offline storage.
3Reliability
If brute force approach is used to locate archived data for deletion, then data can be found, but computing resources and time are excessively consumed
Solution Approach 1:
The system pre-computes and stores metadata indices during the archiving process that include information about data locations, retention periods, and deduplication relationships. When deletion is needed, the system queries this pre-computed metadata rather than scanning archived data, dramatically reducing the time and computing resources required for deletion operations.
4Reliability
If duplicate data is stored across multiple backup stages, then data protection is enhanced, but storage costs and computing resources increase
Solution Approach 1:
The patent merges backup storage and archive storage into a single deduplicated storage system. Data is stored once in the archive storage location and referenced by both backup and archive operations. The deduplication mechanism identifies and eliminates duplicate data blocks, ensuring that identical data is not stored multiple times across different stages, thus reducing storage space while maintaining data protection.
Data Source
AI summary
Data from computing devices is backed-up regularly, storing a snapshot of the data and corresponding metadata in a data store. The backup data are stored for a relatively short period of time before being archived to long-term storage. For snapshots with files with identical data that are not archived together, archive storage space and computing resources may be conserved by not storing duplicates of the data. When the data is added to the archive storage, the archive storage location is added to backup reference entries for other files with identical data. When all files referencing an archive storage location are expired from the backup data store, an archive retention period is initiated, and an entry is added to a time-based archive expiration database indicating the storage location and an expiration time for the archived data. At the expiration time, the archived data is designated for deletion from the archive.