Chunk Headers Enable Compressed Data Backup Without Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional backup systems are inefficient in handling compressed data, as they require decompression to be usable, which undermines the purpose of compression in reducing resource consumption, and are often unfamiliar with various compression methods, leading to increased network bandwidth and storage capacity usage.
Innovation Solution
The system organizes data into a self-contained datastream with chunk headers that store information about data manipulation techniques, allowing backup applications to process compressed data without decompression, and enables efficient storage and restoration by managing chunk headers and maps, which indicate the compression method and size of data chunks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If compressed data is decompressed to be usable by backup applications, then backup applications can process the data, but network bandwidth and storage capacity consumption increase significantly
Solution Approach 1:
The patent introduces chunk headers as an intermediary data structure that enables backup applications to process compressed data without full decompression. The chunk headers contain metadata about compression methods and data organization, allowing the backup application to navigate and manage compressed data directly, thus avoiding the bandwidth and storage overhead of decompression while maintaining processing capability
Solution Approach 2:
The patent segments compressed data into chunks with associated headers, allowing backup applications to work with individual chunks rather than requiring complete decompression of entire data streams. This segmentation enables selective processing and reduces the amount of data that needs to be handled at once, decreasing bandwidth and storage requirements
2Adaptability or versatility
If traditional backup systems handle compressed data without understanding compression methods, then they can process any compressed data, but they require significantly more storage capacity and network bandwidth
Solution Approach 1:
The chunk headers serve as an intermediary layer between the compressed data and the backup application, containing information about compression methods without requiring the application to understand or implement those methods. This allows the system to maintain compression awareness while keeping the backup application simple and versatile
Solution Approach 2:
The patent performs preliminary organization of compressed data into chunks with descriptive headers before the backup process. This preliminary action embeds compression metadata upfront, enabling efficient storage and retrieval without requiring the backup application to analyze or decompress the data, thus reducing storage capacity requirements
Data Source
AI summary
A method, article of manufacture, and apparatus for efficiently recovering information are disclosed. In some embodiments, this comprises receiving a datastream and analyzing the datastream for chunk headers, data chunks and chunk maps. The chunk headers are then used to determine the data manipulation technique of the data chunks. In some embodiments, the chunk headers contain information about the size of the manipulated and un-manipulated data chunks. Data is then reconstructed from the datastream analysis and data manipulation technique determination. The reconstructed data is then stored in a computer readable medium. In some embodiments, the reconstructed data may be stored in multiple destinations. In some embodiments, the storage of the reconstructed data in multiple destinations may be simultaneous.