Object-to-File Metadata Migration With Integrity Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Metadata stored in object storage is susceptible to deletion and/or modification during data migration from one container to another, especially when migrating between different cloud providers, leading to data loss and integrity issues, particularly when migrating to file storage systems.
Innovation Solution
A novel data migration strategy that involves directly injecting object storage metadata into file storage properties, using a supplemental data structure to preserve metadata integrity and allowing targeted migration of specific metadata portions without accessing the underlying object data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional data migration is performed from object storage to file storage, then data transfer is achieved, but metadata integrity is compromised and data loss occurs
Solution Approach 1:
The patent introduces a supplemental data structure as an intermediary carrier that preserves object storage metadata during migration to file storage. This mediator structure includes fields for storing original object keys, metadata key-value pairs, and mappings between source and target locations, enabling faithful transmission of metadata without loss or modification.
Solution Approach 2:
The patent creates a copy of the metadata structure within the supplemental data structure, replicating the original object storage metadata in a format compatible with file storage. This copying process preserves all metadata attributes including content type, last modified date, creation date, and object lock mode, ensuring no data loss during migration.
2Adaptability or versatility
If metadata is migrated during data transfer, then data portability is improved, but metadata modification and deletion occur
Solution Approach 1:
The patent performs preliminary actions by pre-creating the supplemental data structure with dedicated fields for metadata storage before the actual migration begins. This pre-prepared structure guides the migration process to preserve metadata integrity while achieving portability across different storage systems.
Solution Approach 2:
The patent implements feedback mechanisms through validation processes that verify metadata integrity after migration. The system checks that all metadata fields are properly transferred and maintain their original values, allowing for correction of any modifications or deletions that may have occurred during transfer.
3Quantity of substance
If full metadata migration is performed, then data completeness is improved, but migration time and computing resources increase
Solution Approach 1:
The patent segments the metadata migration process into distinct components: object data migration and metadata migration. The supplemental data structure separates these operations, allowing metadata to be transferred independently and efficiently without requiring access to or processing of the entire object data, thus reducing migration time while maintaining completeness.
Solution Approach 2:
The patent extracts only the necessary metadata fields from the object storage system and places them into the supplemental data structure. This extraction process selectively transfers only the required metadata attributes (content type, timestamps, lock modes) rather than migrating all object data, significantly reducing migration time and computing resource requirements.
Data Source
AI summary
Systems and methods for novel uses and/or improvements to data migration. In particular, systems and methods for data migration of metadata stored in object storage from one container to another, especially in instances when the metadata is destined for ingestion by an artificial intelligence application. The systems and methods ensure that all metadata (e.g., metadata stored in object storage) is preserved during data migration, including metadata such as content type, object lock mode, object lock retain until date, and/or custom metadata.