Metadata Object for Archive Data Container Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network storage systems face inefficiencies in restoring specific data containers from object-based storage systems, particularly in managing and retrieving data containers with deduplication and compression, which can be resource-intensive and complex.
Innovation Solution
The solution involves generating a metadata object that includes a first metadata signature and an updated payload signature, using deduplication and compression techniques to efficiently track and restore data containers by storing placement information of data blocks within an intermediate logical object, allowing for targeted restoration with minimal network bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data containers are stored with deduplication and compression in object-based storage, then storage efficiency is improved, but restoration complexity and resource usage increase
Solution Approach 1:
The system pre-generates a metadata object during the archiving process that contains a signature map of all data containers. This preliminary action stores the structural information and block placement details beforehand, so that during restoration, the system can directly use this pre-computed metadata to quickly locate and restore specific data containers without having to parse through the entire archived structure, thus reducing restoration complexity while maintaining storage efficiency
Solution Approach 2:
The patent introduces a metadata object as an intermediary between the archived data containers and the restoration process. This metadata object contains a signature map that maps data container signatures to their block locations and placement information. During restoration, this intermediary metadata structure enables direct access to specific data containers without requiring complex parsing of the deduplicated and compressed data structures, thereby simplifying the restoration process while preserving the storage efficiency benefits of deduplication and compression
2Reliability
If all data containers are restored from an archive, then complete data recovery is achieved, but network bandwidth consumption increases
Solution Approach 1:
The patent extracts only the necessary metadata object from the archive during restoration. The metadata object contains the signature map with block placement information for all data containers. By extracting and using only this compact metadata structure rather than restoring or processing the entire archived data, the system enables selective restoration of individual data containers or specific blocks, significantly reducing network bandwidth consumption while maintaining the ability to recover complete data when needed
3Speed
If metadata includes detailed placement information for all blocks, then restoration speed is improved, but metadata size and storage overhead increase
Solution Approach 1:
The patent segments the metadata structure into a compact signature map format. Instead of storing detailed placement information for all blocks in a monolithic structure, the metadata is segmented into signature entries that map data container signatures to their block locations. This segmentation allows the system to store only the essential placement information needed for restoration in a compressed format, reducing metadata size while maintaining fast restoration capability through efficient signature-based lookup
Data Source
AI summary
Methods and systems for an object based storage are provided. As an example, a method for generating a metadata object for an archive data container having a plurality of data containers is disclosed. The method includes generating a first metadata signature for the archive data container using an archive data container identifier, a number of data containers within the archive data container, and placement information of each data container within the archive data container; assigning a plurality of blocks for storing data for the plurality of data containers at an object based storage to an intermediate logical object; updating a payload signature with placement information of the plurality of blocks within the intermediate logical object; and placing the first metadata signature and the updated payload signature within the metadata object, wherein the metadata object is used to retrieve location information for a specific data container within the archive data container.