Object Storage Snapshot Management via Storlet Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud storage systems lack efficient control over managing point in time copies of data, leading to increased costs and time consumption in backup and recovery operations, especially when dealing with large volumes of incremental backups and the need to preserve snapshot chains.
Innovation Solution
Implementing a system that allows for the manipulation of point in time copies of data within an object storage system, using directives to collapse, roll forward, or delete snapshots, thereby reducing the need for on-premises processing and minimizing data transfer across expensive or congested networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple point in time copies are kept at frequent intervals in cloud storage, then data recovery options are improved, but storage costs and data management complexity increase
Solution Approach 1:
The patent implements dynamic retention policies that automatically adjust the number and age of retained snapshots based on configurable parameters such as maximum snapshot age, minimum retention count, and storage capacity thresholds. This allows the system to maintain adequate recovery options while preventing uncontrolled growth of stored data copies.
Solution Approach 2:
The system performs self-managed snapshot consolidation and deletion operations based on predefined policies without requiring manual intervention. The automated policy engine continuously evaluates stored snapshots and executes retention rules, reducing both storage costs and management overhead while maintaining recovery capabilities.
2Reliability
If multiple incremental backups are downloaded and restored sequentially from cloud storage, then complete data recovery is achieved, but recovery time and network bandwidth consumption increase
Solution Approach 1:
The system pre-processes and consolidates multiple incremental snapshots into a single cumulative snapshot before storage, so that during recovery operations, only one snapshot needs to be downloaded and applied rather than sequentially processing multiple incremental backups. This dramatically reduces both recovery time and network bandwidth consumption.
Solution Approach 2:
The patent merges multiple incremental backup data sets into a single consolidated snapshot that contains all changes accumulated across the incremental sequence. This consolidation allows the recovery process to retrieve and apply one unified data set rather than multiple separate incremental updates, eliminating the time-consuming sequential restoration process.
3Reliability
If flashcopy bitmaps are preserved to maintain restore capability, then data integrity is maintained, but storage space is consumed and deletion operations are restricted
Solution Approach 1:
The system extracts and removes unnecessary flashcopy bitmap data structures when snapshots are consolidated or deleted, allowing flexible management of snapshot chains without being constrained by preserved bitmap integrity requirements. This enables straightforward deletion operations while maintaining data integrity through the consolidated snapshot approach.
Solution Approach 2:
The patent implements selective discarding of intermediate snapshot data structures (including flashcopy bitmaps) when consolidation occurs, with the understanding that the consolidated snapshot preserves the necessary data integrity information. This allows deletion operations to proceed freely on individual snapshots while the consolidated version maintains recoverability.
4Ease of operation
If on-premises processing is used to manage point in time copies, then control over snapshot operations is maintained, but network bandwidth is consumed and processing overhead increases
Solution Approach 1:
The patent shifts snapshot management operations from the on-premises environment to the cloud storage environment, where consolidation and deletion operations are executed directly on stored data without requiring data transfer to local systems. This dimensional shift eliminates network bandwidth consumption for management operations while maintaining full control through cloud-based policy enforcement.
Solution Approach 2:
The system implements self-service snapshot management capabilities within the cloud storage environment, where the storage system autonomously performs consolidation and deletion operations based on configured policies without requiring on-premises processing or data transfer. This reduces network overhead while maintaining operational control through policy configuration.
Data Source
AI summary
A system, according to one embodiment, includes: a processor, as well as logic that is integrated with the processor, executable by the processor, or integrated with and executable by the processor. The logic is configured to: send, by the processor, point in time copies of data to an object storage system. A directive for manipulating the point in time copies of the data are also set to the object storage system by the processor. Moreover, the point in time copies of the data are manipulated by a storlet on the object storage system according to the directive.