Data Migration via Compressed Segment Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data migration methods between primary and secondary storage systems are inefficient, as they require decompression of data, leading to increased processing demands and network resource usage, resulting in slower performance during migration and recall operations.
Innovation Solution
A method and device that utilize a second interface specifically for migrating compressed data from the primary storage system to the secondary storage system, eliminating the need for decompression and enabling faster data transfer, along with the implementation of partial recall of compressed data through logical segment mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is decompressed during migration, then data accessibility is improved, but processing time and network resource usage increase
Solution Approach 1:
The system performs preliminary actions by maintaining metadata that pre-records the mapping relationships between compressed data segments and their corresponding uncompressed data positions. This allows the system to directly locate and retrieve specific data segments without performing full decompression, thus resolving the contradiction between data accessibility and processing time.
Solution Approach 2:
The patent divides compressed data into multiple segments and maintains separate metadata records for each segment, mapping them to their corresponding uncompressed positions. This segmentation allows selective retrieval of specific data portions without processing the entire compressed data set, reducing both processing time and network resource usage while maintaining accessibility.
2Productivity
If compressed data is migrated directly without decompression, then migration speed is improved, but data retrieval complexity increases
Solution Approach 1:
The system creates and maintains a metadata copy that records the mapping relationships between compressed data segments and their uncompressed equivalents. This metadata copy serves as an index that simplifies data retrieval operations, allowing the system to migrate compressed data directly while maintaining simple retrieval processes through the metadata index.
3Ease of operation
If full decompression is performed for data recall, then data usability is improved, but network bandwidth consumption increases
Solution Approach 1:
The system implements partial action by retrieving and decompressing only the specific compressed data segments that are currently needed, rather than performing full decompression of all migrated data. The metadata enables precise identification of required segments, reducing network bandwidth consumption while maintaining data usability for the specific operations at hand.
Data Source
AI summary
Embodiments of the present disclosure relate to a method and device for migrating data. The method comprises identifying cold data in a primary storage system. The method further comprises, in response to determining that the cold data is in a non-compression state, obtaining the cold data from the primary storage system via a first interface, the first interface being configured for a user to access the primary storage system. The method further comprises obtaining, in response to determining the cold data is in a compression state, the cold data in the compression state from the primary storage system via a second interface that is different from the first interface. The method further comprises migrating the obtained cold data from the primary storage system to a secondary storage system.