Virtual Disk Cloning via Snapshot Metadata in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage techniques in data centers face challenges such as storage manageability issues, high costs, and inefficiencies in provisioning and managing storage capacity, particularly with the rise of virtualization, where many server applications need to access disks within a central storage node, leading to performance bottlenecks and long provisioning times.
Innovation Solution
The system provides incremental scalability, a user-friendly management console for provisioning virtual disks, and fine-grained policy management, allowing unique properties and policies for each virtual disk, enabling efficient data protection and recovery, and supporting various protocols like iSCSI and NFS, with features like snapshotting and cloning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a central storage node is used to consolidate storage disks, then storage manageability is improved, but performance bottlenecks occur due to the sheer number of requests from virtual machine applications
Solution Approach 1:
The patent divides the centralized storage node into multiple distributed storage nodes (first storage node, second storage node, etc.), each capable of independently handling storage requests. This segmentation distributes the request load across multiple nodes, preventing performance bottlenecks while maintaining centralized management capabilities through the management module.
2Quantity of substance
If additional storage nodes are added to increase capacity, then storage capacity is improved, but system complexity and management overhead increase
Solution Approach 1:
The patent creates storage nodes with universal functionality where each node can serve multiple purposes: data storage, request processing, and dynamic role assignment. The management module can dynamically designate nodes as source or target nodes based on cloning operations, eliminating the need for specialized hardware or complex configuration for different storage functions.
Solution Approach 2:
The storage system implements self-service capabilities through automated provisioning and cloning operations. The management module automatically handles resource allocation, data replication, and node configuration when cloning virtual disks, reducing manual intervention and management overhead while scaling capacity.
3Reliability
If traditional storage provisioning methods are used, then data protection and replication are achieved, but provisioning time for new virtual disks becomes excessively long (up to four weeks)
Solution Approach 1:
The patent implements rapid virtual disk provisioning through cloning operations that create copies of existing virtual disks. Instead of provisioning disks from scratch or through lengthy manual processes, the system clones proven templates or existing disks, instantly providing protected, pre-configured storage resources that inherit data protection policies from the source.
Solution Approach 2:
The system performs preliminary actions by pre-configuring storage templates with data protection policies, replication settings, and performance parameters before actual provisioning needs arise. When provisioning is required, these pre-prepared templates can be rapidly cloned and deployed, eliminating lengthy configuration processes while maintaining comprehensive data protection.
4Reliability
If monolithic hardware boxes are purchased with separate products for compression, replication, and de-duplication, then data protection functionalities are achieved, but cost and configuration complexity increase
Solution Approach 1:
The patent merges previously separate data protection functionalities (compression, replication, de-duplication) into an integrated software-based system running across distributed storage nodes. The management module coordinates these functions across nodes, eliminating the need for separate hardware appliances while maintaining comprehensive data protection capabilities through unified policy management.
Data Source
AI summary
An administrator provisions a virtual disk in a remote storage platform and defines policies for that virtual disk. A virtual machine writes to and reads from the storage platform using any storage protocol. Virtual disk data within a failed storage pool is migrated to different storage pools while still respecting the policies of each virtual disk. Snapshot and revert commands are given for a virtual disk at a particular point in time and overhead is minimal. A virtual disk is cloned utilizing snapshot information and no data need be copied. Any number of Zookeeper clusters are executing in a coordinated fashion within the storage platform, thus increasing overall throughput. A timestamp is generated that guarantees a monotonically increasing counter, even upon a crash of a virtual machine. Any virtual disk has a “hybrid cloud aware” policy in which one replica of the virtual disk is stored in a public cloud.