Live vSAN File System Upgrade via Metadata Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for upgrading virtual storage area network (vSAN) on-disk formats from extent-based to log-structured file systems require downtime or maintain dual file systems, which are impractical and costly.
Innovation Solution
The solution involves an upgrade manager that concatenates components of the old and new file systems, pre-populates metadata, and redirects I/O operations during the transition, allowing for live upgrades without disrupting user input/output operations or causing significant performance degradation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If offline upgrade is performed by reading content from previous file system and inserting into new file system, then file system upgrade is achieved, but system downtime occurs and operation is disrupted
Solution Approach 1:
The patent implements a dynamic upgrade approach where the file system transitions from static offline upgrading to dynamic live upgrading. The system allows simultaneous operation of old and new file systems, with I/O operations dynamically redirected based on data location. This enables continuous system operation during the upgrade process, eliminating the need for complete system downtime while ensuring upgrade completeness.
Solution Approach 2:
The patent applies preliminary action by pre-creating the new file system structure and pre-populating its metadata before actual data migration begins. The upgrade manager pre-establishes the target file system framework, pre-allocates storage resources, and pre-configures metadata structures, so that when the upgrade is triggered, the system can seamlessly switch to the new file system without interruption, thus avoiding downtime.
2Ease of operation
If dual file systems are maintained to store new data in newer file system and serve old data from previous file system, then continuous operation is enabled, but continuous cost of forwarding and maintaining two file systems incurs
Solution Approach 1:
The patent extracts the file system management functionality into a separate upgrade manager component that operates independently from the core storage operations. This upgrade manager selectively manages the transition between old and new file systems, extracting only the necessary upgrade operations while leaving the main storage system unchanged. This reduces complexity by centralizing upgrade logic rather than maintaining complex dual file system integration throughout the entire storage subsystem.
Solution Approach 2:
The upgrade manager acts as an intermediary between the old extent-based file system and the new log-structured file system. It mediates I/O operations by intercepting requests, determining whether data resides in the old or new file system, and redirecting operations appropriately. This intermediary approach simplifies the dual file system architecture by providing a single point of control for managing the transition, rather than requiring complex integration logic throughout the storage stack.
3Productivity
If live upgrade is implemented with concatenation of old and new file system components, then continuous operation is maintained, but metadata management complexity increases
Solution Approach 1:
The patent segments the file system metadata into distinct components: old file system metadata, new file system metadata, and upgrade manager control structures. Each segment is independently managed and can be processed separately. The upgrade manager maintains separate metadata structures for each file system version, allowing parallel management without requiring complex integration of all metadata into a single structure. This segmentation reduces overall metadata management complexity by dividing it into manageable, independent units.
Solution Approach 2:
The upgrade manager creates and maintains copies of necessary metadata structures for both the old and new file systems simultaneously. Rather than attempting to transform or convert metadata in place, the system creates parallel metadata copies that can be independently validated and managed. This copying approach simplifies metadata management during the transition by allowing the system to work with identical metadata structures for both file system versions, reducing the complexity of handling format conversions and compatibility issues.
Data Source
AI summary
An example method of upgrading a distributed storage object from a first version to a second version includes: querying metadata of a first component configured according to the first version of the distributed storage object, the metadata defining extents of data on a disk group of the first component; populating, for a second component configured according to the second version of the distributed storage object, logical and middle maps based on the metadata such that initial entries in the logical map point to initial entries in the middle map, and the initial entries in the middle map point to physical addresses of the disk group of the first component; and reading the data from the disk group of the first component and writing the data to a disk group of the second component while updating the initial entries in the middle map.