Cloud Archiving via File Layout APIs for Compressed Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current archiving approaches to cloud storage require decompression and re-compression of data, which is computationally expensive and inefficient, especially when using scarce compute resources on storage appliances, and can result in data transmission failures over lower speed network links.
Innovation Solution
The method involves retrieving the layout of a file on a storage device to identify locations of compressed data blocks, allowing for the direct transfer of already compressed data to cloud storage without decompression and re-compression, leveraging file layout APIs to manage the retrieval and migration of compressed data blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is decompressed and re-compressed for cloud archiving, then data can be transferred to cloud storage, but computational resources are excessively consumed
Solution Approach 1:
The patent applies preliminary action by compressing data before storage on the storage appliance. When archiving to cloud storage, the already-compressed data is directly transferred without requiring decompression and re-compression. This preliminary compression action eliminates the need for repeated compression operations during archiving, significantly reducing computational resource consumption while maintaining data transfer reliability
Solution Approach 2:
The patent uses copying by creating a reference or pointer to the compressed data blocks already stored on the appliance rather than physically moving and re-compressing the data. The file layout API retrieves metadata about compressed block locations, and the system copies these references to the cloud storage platform, avoiding expensive decompression and re-compression operations while ensuring data integrity
2Productivity
If data is processed for transmission to cloud locations, then archiving can be completed, but processing time increases
Solution Approach 1:
Data is compressed in advance before being stored on the appliance. During archiving operations, this pre-compressed data can be directly transferred to cloud storage without requiring time-consuming decompression and re-compression steps, significantly reducing the overall archiving time and improving productivity
Solution Approach 2:
The patent extracts only the necessary metadata (file layout information identifying compressed block locations) from the storage appliance rather than extracting and processing the actual data blocks. This extraction approach minimizes processing time while still enabling efficient data transfer to cloud storage
3Use of energy by moving object
If compressed data is directly transferred from storage appliance, then computational resources are saved, but data layout complexity must be managed
Solution Approach 1:
The patent introduces a file layout API as an intermediary layer between the storage appliance and the archiving system. This API handles the complexity of retrieving and interpreting compressed data block locations, providing a simplified interface that returns metadata about data layouts. The intermediary absorbs the complexity management burden, allowing the rest of the system to efficiently transfer compressed data without directly dealing with layout complexity
Data Source
AI summary
A method comprises retrieving a layout of a file on a storage device, wherein the layout identifies a plurality of locations on the storage device where a plurality of compressed blocks of data are stored. In the method, the plurality of compressed blocks of data are retrieved from the plurality of locations identified in the layout for transmission to and storage in a cloud storage platform.