Logical Map for Seamless On-Disk Format Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise storage systems face disruptions when new features are introduced, requiring data conversion from one incompatible on-disk format to another, such as from a RAID-6 array with 4 MB data stripes to one with 1 TB data stripes, which can be complex and disruptive.
Innovation Solution
The implementation of an object manager that creates conversion metadata, including a logical map and pointers, to manage the conversion of data objects from an old format to a new format, allowing for concurrent read and write operations without taking the data object offline, using a B-tree data structure for efficient insertion and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data conversion from old format to new format is performed, then storage system can support new features, but IO operations are disrupted and system availability is reduced
Solution Approach 1:
The patent segments the data storage system into multiple components: old format data objects, new format data objects, and a logical map that coordinates between them. This segmentation allows the system to maintain both old and new formats simultaneously, enabling format conversion without disrupting IO operations. The logical map divides the address space into regions that can be accessed independently, allowing readers to access data in either format while writers migrate to the new format.
Solution Approach 2:
The logical map serves as an intermediary between the old and new data formats. It contains translation information that maps logical block addresses to physical locations in either the old or new format. This intermediary layer abstracts the format difference from IO operations, allowing the system to support both formats without requiring changes to the IO interface or disrupting ongoing operations.
2Adaptability or versatility
If data conversion is performed offline, then format compatibility is achieved, but productivity and user access are reduced
Solution Approach 1:
The patent enables continuous IO operations during the format conversion process. Readers can continuously access data in either the old or new format through the logical map, while writers continuously migrate data to the new format. This continuous operation eliminates the need for offline conversion, maintaining productivity and user access throughout the migration process.
3Ease of operation
If conversion metadata is created to manage format transition, then seamless conversion is enabled, but device complexity increases
Solution Approach 1:
The patent adds a logical mapping dimension between the old and new data formats. Instead of directly converting physical storage structures, it creates a logical layer that maps logical block addresses to physical locations in either format. This additional dimensional layer simplifies the conversion management by providing a unified interface while handling the complexity of format translation in the background.
Data Source
AI summary
A logical map represents fragments from separate versions of a data object. Migration of data from a first (old) version to the second (new) version happens gradually, where write operations go to the new version of the data object. The logical map initially points to the old data object, but is updated to point to the portions of the new data object as write operations are performed on the new data object. A background migration copies data from the old data object to the new data object.