Application-Based Packing for Cloud Backup Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based object storage systems face inefficiencies in managing backup data, leading to increased storage costs due to uniform data handling and lack of data-specific storage techniques, particularly in incremental backup infrastructures.
Innovation Solution
A system for application-based packing of backup data into objects of specialized size, allowing for intelligent allocation within cloud-based object storage by determining whether files are stored as individual or shared objects, and storing metadata to facilitate full restore capabilities, thereby optimizing storage space and reducing costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored as immutable objects to meet retention requirements, then data protection is improved, but storage space consumption increases
Solution Approach 1:
The system segments backup data into immutable and mutable portions, storing only the immutable portions as immutable objects in object storage while keeping mutable portions in traditional storage, thereby reducing storage space consumption while maintaining data protection for retained data
Solution Approach 2:
The system applies different storage qualities to different portions of backup data based on their retention status and mutability characteristics, storing immutable data in object storage with high durability while keeping mutable data in traditional storage with faster access
2Adaptability or versatility
If universal storage APIs are used for backup operations, then system compatibility is improved, but storage resource allocation efficiency deteriorates
Solution Approach 1:
The system introduces a storage gateway as an intermediary between backup software and object storage, which translates universal backup API requests into optimized object storage operations, enabling both system compatibility and efficient storage resource allocation
3Productivity
If incremental backup infrastructure is implemented, then backup efficiency is improved, but storage management complexity increases
Solution Approach 1:
The storage gateway automatically performs synthetic full backup operations by retrieving incremental backup data from object storage and reconstructing full backup sets, eliminating the need for manual full backup operations and reducing storage management complexity
Data Source
AI summary
Described is a system (and method) for application-based packing for storing backup data within an object storage. More particularly, the system may efficiently pack backup files into objects having a specialized size by accounting for the source and size of the backup file. For example, the system may determine whether backup files associated with a particular application may be stored within individual objects or packed together within a shared object. Accordingly, by accounting for such criteria the system may perform an intelligent allocation of backup data within a cloud-based object storage. The system may be implemented as part of an incremental backup infrastructure. Accordingly, when allocating the backup files amongst objects, the system may also store specialized metadata to preserve the ability to perform a reconstruction of data (e.g. synthetically) as if a full backup was performed.