Three-Tiered Data Storage System with Deduplication and Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The ongoing storage of multiple versions of data in tiered data storage systems incurs significant economic and computing costs due to the large amount of data and the need to maintain version and location information, despite declining storage costs.
Innovation Solution
A three-tiered data storage system is implemented, where data is managed through checkpoints, snapshots, and archives, with policies defining storage times and locations, and differencing techniques are used to store only changed portions of data, reducing redundancy and storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple versions of data are stored for backup and resiliency, then data reliability is improved, but storage cost and computing cost increase
Solution Approach 1:
The patent merges multiple versions of the same data into a single storage location using deduplication techniques. Instead of storing identical data copies separately, the system combines them by identifying duplicate data blocks and storing only one instance, thereby reducing storage cost while maintaining data reliability through version tracking.
Solution Approach 2:
The patent implements a universal storage mechanism that handles multiple data versions, deduplication, and retrieval operations through a single integrated system. The storage system provides multi-functionality by simultaneously supporting version control, space optimization, and data recovery without requiring separate specialized systems for each function.
2Reliability
If multiple versions of data are stored, then data resiliency is improved, but the amount of data to be managed increases
Solution Approach 1:
The patent extracts the essential version information and metadata from the actual data storage management process. By separating version tracking and deduplication logic into distinct management layers, the system reduces the complexity of managing multiple data versions while maintaining resiliency through streamlined version control mechanisms.
Solution Approach 2:
The patent implements automated pruning mechanisms that discard obsolete data versions that are no longer needed for resiliency, while recovering and retaining only the necessary versions. This reduces the amount of data to be managed and lowers computing costs by eliminating redundant version information.
3Quantity of substance
If differencing techniques are used to store only changed portions, then storage space is reduced, but data reconstruction complexity increases
Solution Approach 1:
The patent performs preliminary indexing and cataloging of data blocks during the storage process, creating a map of data locations and versions before retrieval is needed. This preliminary action stores metadata about data block relationships, which simplifies the reconstruction process by providing pre-computed information about how to assemble recovered data from differenced portions.
Data Source
AI summary
An efficient data storage system is described. An agent software application on computing devices in a first tier processes snapshot backups and pushes them to an appliance software application on a server in a second tier. The appliance software application processes archive backups and pushes them to cloud storage in a third tier. A cloud application on a management server receives storage policy specifications from customers and promulgates the policies to the agent software application and the appliance software application. The policy specifications include time periods and retention set information for the backups. When a retention set has been exceeded, the storage system is pruned to remove file references to unneeded files and delete data files no longer referenced in storage sets in the retention set.