Acceleration Pool for Block Volume Creation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud computing, the creation of new volumes from snapshots is often slow due to limited data transfer rates, leading to long latencies, especially when multiple volumes are created in parallel, which can result in increased chances of 'get faults' and reduced system availability.
Innovation Solution
The implementation of a dedicated acceleration pool using read-only block storage replicas of a snapshot, where source volumes are assigned to users to accelerate the creation of new volumes, reducing the burden on the snapshot and minimizing bandwidth overload, allowing for faster replication and parallel creation of volumes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data is transferred directly from snapshot to new volumes, then storage capacity is preserved, but data transfer rate is limited and creation time is long
Solution Approach 1:
The patent introduces an acceleration pool as an intermediary storage layer between the snapshot and new volumes. Data is first copied from the snapshot to the acceleration pool, then from the acceleration pool to new volumes. This intermediary structure enables parallel data transfer operations and significantly increases the overall data transfer rate, reducing volume creation time from hours to minutes.
Solution Approach 2:
The patent segments the data transfer process into two distinct phases: (1) snapshot to acceleration pool, and (2) acceleration pool to new volumes. This segmentation allows multiple new volumes to be created in parallel by distributing data blocks across multiple acceleration pool volumes, thereby increasing throughput and reducing total creation time.
2Productivity
If multiple volumes are created in parallel from a snapshot, then system throughput is improved, but bandwidth overload increases and get faults occur more frequently
Solution Approach 1:
The acceleration pool serves as a buffer intermediary that decouples the snapshot from multiple new volume creation operations. By pre-copying data to the acceleration pool, the system can serve multiple parallel volume creation requests without overloading the snapshot, thereby maintaining high throughput while preventing get faults and ensuring system reliability.
Solution Approach 2:
The system performs preliminary data copying from the snapshot to the acceleration pool before actual volume creation requests are fulfilled. This preliminary action prepares the data in advance, allowing subsequent parallel volume creations to proceed quickly and reliably without competing for snapshot bandwidth, thus preventing get faults.
3Speed
If a dedicated acceleration pool is implemented, then volume creation speed is increased, but system complexity increases
Solution Approach 1:
The acceleration pool is designed as a universal storage structure that can serve multiple functions: (1) accelerating volume creation from snapshots, (2) serving as a cache for frequent access patterns, and (3) supporting parallel operations. This multi-functionality justifies the added complexity by providing broad system benefits across different operational scenarios.
Solution Approach 2:
The system creates simplified copies of snapshot data in the acceleration pool using a standardized data structure and format. This copying mechanism is automated and managed through uniform interfaces, which masks the underlying complexity from users while enabling fast parallel volume creation. The copy operation itself is streamlined through chunk-based data movement and parallel processing.
Data Source
AI summary
Generally described, one or more aspects of the present application correspond to techniques for new block store volumes of data from object storage snapshots of an origin volume by creating an intermediary “acceleration pool” of source volumes reserved for supporting the new volume creation process. The source volumes may be prevented from supporting user I/O or computing instances in order to reserve their bandwidth for the transfer of data to the new volumes. In addition, the source volumes can be stored on block store servers to increase the speed with which their data can be transferred to the new volumes. The acceleration pool can be partitioned among users accessing the pool in order to maintain availability even when some users heavily utilize the acceleration pool, with particular source volumes intelligently selected from among a user's partition in order to optimize the speed at which the new volumes are created.