Unified Data Management via Torrent Metadata Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems face challenges in integrating and managing data across multiple storage systems, particularly when these systems lack unified metadata, requiring time-consuming and costly data transfer processes to rebuild metadata for unified access.
Innovation Solution
Acquire a torrent file from the storage system, which contains data digests of data blocks, and generate metadata in a format supported by the unified management system for data-block-level access, allowing efficient integration and access without the need to download and upload the entire file.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transferred between different storage systems to achieve unified management, then data accessibility across storage systems is improved, but data transfer time and cost increase
Solution Approach 1:
The patent segments the data management process by separating metadata operations from actual data transfer. It extracts and manages metadata (torrent files, data digests, block information) independently from the bulk data, allowing unified management without requiring physical data movement between storage systems.
Solution Approach 2:
The patent introduces a unified management system that acts as an intermediary layer between different storage systems. This system manages metadata and coordinates access to data blocks across multiple storage systems without requiring direct data transfer between them, thereby maintaining data accessibility while avoiding transfer time penalties.
2Adaptability or versatility
If data is transferred between different storage systems to achieve unified management, then data accessibility across storage systems is improved, but transfer cost increases
Solution Approach 1:
The patent segments the data management process by separating metadata operations from actual data transfer. It extracts and manages metadata (torrent files, data digests, block information) independently from the bulk data, allowing unified management without requiring physical data movement between storage systems.
Solution Approach 2:
The patent creates a virtual copy of data access pathways through metadata rather than physical copying of data. The unified management system uses torrent files and metadata to represent and manage data blocks across different storage systems, eliminating the need for expensive data transfer while maintaining access capability.
3Adaptability or versatility
If metadata is rebuilt by downloading and uploading entire files, then unified metadata format is achieved, but processing time and resource consumption increase
Solution Approach 1:
The patent extracts only the necessary metadata components (torrent files, data digests, block information) from the storage systems without requiring download and upload of entire files. This extraction approach achieves unified metadata format while dramatically reducing processing time and resource consumption.
Solution Approach 2:
The patent performs preliminary actions by collecting and organizing metadata (torrent files containing data digests and block information) before unified management is needed. This preliminary metadata preparation enables efficient unified management without requiring subsequent file-level processing.
Data Source
AI summary
A method, a device, and a computer program product for data management are provided in embodiments of the present disclosure. A method for data management comprises: acquiring, from a storage system, a torrent file associated with downloading of a target file, the target file being stored in the storage system, and the torrent file comprising at least a plurality of data digests of a plurality of data blocks of the target file; generating metadata for the plurality of data blocks based on the torrent file, the metadata being in a format supported by a unified management system, and the unified management system being configured for data access across the storage system and at least one other storage system; and storing the metadata for data-block-level access to the plurality of data blocks through the unified management system.