Block-Based File System Object Storage Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current backup and restore systems face inefficiencies in managing and storing large volumes of data, particularly in handling duplicate data and scaling storage solutions, as they often rely on block-based systems that lack metadata association and struggle to efficiently utilize object storage.
Innovation Solution
A block-based file system is enhanced to support object storage by using file handles and offsets to generate object identifiers, storing metadata and data containers in a stripe-aligned manner, and implementing deduplication to remove duplicates, thereby enabling efficient data management and scaling through both local and cloud-based storage tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If block-based storage systems are used for backup, then data redundancy and protection are improved, but the ability to scale and manage large volumes of data efficiently deteriorates
Solution Approach 1:
The patent implements a dual-mode storage system where the same storage infrastructure can operate in both block-based mode (for data protection and redundancy) and object-based mode (for scalable data management). The storage system includes a block device interface and an object storage interface that share underlying storage resources, allowing seamless adaptation between different operational requirements without needing separate storage systems.
Solution Approach 2:
The patent introduces a storage management layer that acts as an intermediary between the block-based file system and the object storage system. This layer includes components such as a block device driver, object storage interface, and data synchronization mechanism that translate and coordinate operations between the two storage paradigms, enabling efficient data movement and management across different storage modes.
2Quantity of substance
If deduplication is implemented to remove duplicate data, then storage efficiency is improved, but processing time and computational resources increase
Solution Approach 1:
The patent implements deduplication at the time of data ingestion and during incremental backup operations, rather than performing comprehensive deduplication only during backup. The system continuously tracks data blocks and identifies duplicates as they are written to storage, maintaining a block index structure that enables real-time deduplication decisions. This preliminary action reduces the computational burden during actual backup operations.
Solution Approach 2:
The patent divides the deduplication process into manageable segments by processing data in blocks and using a block-based indexing structure. The storage system breaks down large files into smaller blocks, creates fingerprints for each block, and stores only unique blocks in the object storage system. This segmentation approach allows parallel processing and reduces the overall computational complexity of deduplication operations.
3Productivity
If stripe-aligned storage is used for metadata and data containers, then data organization and retrieval efficiency are improved, but storage flexibility and adaptability may be reduced
Solution Approach 1:
The patent implements a dynamic storage allocation system where stripe alignment is applied selectively based on the type of data being stored and the operational requirements. The storage management layer can dynamically adjust the alignment strategy for different containers and data types, switching between stripe-aligned and flexible allocation as needed. This dynamic approach maintains the performance benefits of stripe alignment for structured data while preserving flexibility for varying storage requirements.
Data Source
AI summary
A computer-implemented method for storing backup data includes receiving, by a computing system, data associated with a file to be backed up; performing, by the computing system, de-duplication on the file to remove duplicated data and to generate unique backup data; receiving, by the computing system, a file handle based on opening a block-based storage file; storing, by the computing system, a metadata container in the block-based storage file stripe aligned; and storing, by the computing system, a data container in the block-based storage file stripe aligned, wherein the metadata container and the data container are associated with a set of objects, and wherein the data container includes the unique backup data.