Cloud Storage SDKs for Native Data Format Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Companies face challenges in efficiently managing and protecting growing volumes of data across their networks while minimizing productivity impact and optimizing storage costs, particularly in migrating data to lower-cost storage and ensuring data availability and accessibility.
Innovation Solution
The implementation of an information management system that facilitates the storage of data sets, including backup and secondary copies, in cloud storage libraries using a format native to the cloud, with storage policies guiding data transfer and metadata addition through cloud SDKs, enabling efficient data recovery and management across various storage formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is stored in cloud storage libraries using cloud native formats, then data accessibility and scalability are improved, but data management complexity increases
Solution Approach 1:
The patent introduces cloud storage SDKs as intermediary components that mediate between the information management system and cloud storage libraries. These SDKs handle the complexity of cloud native format operations, providing simplified interfaces for data transfer, metadata management, and storage operations, thereby improving accessibility without exposing the underlying complexity to users
Solution Approach 2:
The system segments data management operations into distinct functional components: data transfer operations, metadata addition operations, and storage operations. Each operation is handled by specific modules (data transfer module, metadata module, storage module), which independently manage specific aspects of cloud storage, reducing overall system complexity while maintaining versatility
2Quantity of substance
If data is migrated to cloud storage, then storage costs are optimized, but data recovery efficiency may be reduced
Solution Approach 1:
The system performs preliminary actions by maintaining detailed metadata about stored data in the cloud, including data block locations, file structures, and access patterns. This pre-organized information enables rapid data recovery operations without requiring full data retrieval from cloud storage, significantly reducing recovery time while maintaining cost-effective cloud storage utilization
Solution Approach 2:
The patent implements a hybrid storage strategy where critical data blocks are copied and cached in local storage systems, while complete data sets are maintained in cloud storage. This copying approach allows fast recovery of frequently accessed data from local caches while maintaining economical cloud storage utilization for complete data sets
3Productivity
If cloud SDKs are used to transfer data blocks, then data transfer efficiency is improved, but system complexity increases
Solution Approach 1:
The cloud storage SDKs are designed as universal, multi-functional components that handle multiple operations (data transfer, metadata management, authentication, storage) through unified interfaces. This multi-functionality improves data transfer efficiency by optimizing all operations through a single standardized interface while containing system complexity within the SDK layer, preventing complexity from propagating to the rest of the information management system
Data Source
AI summary
Systems and methods that facilitate an information management system to store sets of data, such as backup or secondary copies of production data, to cloud storage libraries in a format native to the cloud storage library are disclosed. The systems and methods create storage policies for the sets of data that identify the cloud storage libraries to which the sets of data are to be stored. Via the storage policies, one or more media agents or servers transfer data blocks of the sets of data to cloud storage SDKs associated with the cloud storage libraries. The cloud storage SDKs then add external metadata, such as metadata associated with a native format for the cloud library to the data blocks, before the data blocks are stored to the cloud storage library.