Self-Contained Datastream for Backup Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data backup methods suffer from significant disk overhead and inefficiency due to redundant data storage and the fragmented nature of deduplicated data, especially when dealing with large datasets like those on tape storage systems.
Innovation Solution
A datastream engine organizes deduplicated data into a self-contained format with file headers, metadata, and blocks, minimizing the need for multiple reads and allowing for efficient storage and retrieval by maintaining all necessary components in a single datastream, reducing redundant storage and improving restore operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If data is deduplicated by dividing into blocks and storing in hash tables or binary trees, then data redundancy is reduced, but disk overhead increases due to constant disk access for searching
Solution Approach 1:
The patent applies preliminary action by organizing deduplicated data blocks into a sequential datastream format before storage, with file headers and metadata prepared in advance. This eliminates the need for constant disk searching during restore operations, as the sequential structure allows direct access to required blocks without hashing or tree traversal.
Solution Approach 2:
The patent replaces the mechanical disk-based hash table or binary tree search system with a sequential datastream structure. Instead of using disk-based indexing mechanisms that require constant seeking and accessing, the data is organized in a linear sequence that can be processed more efficiently, reducing mechanical disk overhead.
2Ease of operation
If deduplicated data is stored in hash tables or binary trees, then data retrieval is possible, but searching efficiency decreases as database size grows
Solution Approach 1:
The patent segments the deduplicated data into a structured datastream format with file headers, metadata sections, and sequential data blocks. Each segment is self-contained with necessary information for identification and retrieval, eliminating the need for searching through large hash tables or binary trees while maintaining efficient data access capability.
3Reliability
If traditional backup methods are used to store copies of data, then data safety is ensured, but storage capacity is wasted due to redundant data
Solution Approach 1:
The patent merges multiple deduplicated data blocks into a single consolidated datastream structure that maintains data integrity and safety. By combining the blocks with file headers and metadata in a unified sequential format, the system ensures data reliability while eliminating the storage waste associated with traditional redundant copying methods.
Data Source
AI summary
A method, article of manufacture, and apparatus for efficiently backing up information is disclosed. In an embodiment, this comprises receiving data and analyzing the received data. In an embodiment, the data may be deduped. File headers, corresponding file metadata and corresponding file blocks are then created based on analyzing the data. In an embodiment, the data is discarded. The file headers, corresponding file metadata and corresponding file blocks are then organized into a self contained datastream. The self contained datastream is then stored in a storage device. File headers are used to indicate the start of a new file in the self contained datastream. File blocks contain the information necessary to reconstruct the files in the datastream. File metadata describes how to reconstruct the file, such as which file blocks are required for the file, and the location of each file block in the self contained datastream, among others.