Hybrid Page-Block Storage for Cloud File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud file systems often compromise on factors like cost, latency, IOPs, throughput, reliability, and scale due to their design, which typically uses a single storage technology for both reads and writes, such as page storage technology that has relatively low IOPs and throughput.
Innovation Solution
The solution involves using different storage technologies for reads and writes, specifically employing page storage for writes and block storage for reads. This approach allows for the use of storage technologies with attributes more beneficial for each operation, enhancing speed and reliability while reducing costs and preserving session consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If page storage technology is used for both reads and writes in conventional cloud file systems, then session consistency is maintained, but IOPs and throughput are limited due to the inherent characteristics of page storage
Solution Approach 1:
The storage system is segmented into two distinct storage technologies: page storage for write operations and block storage for read operations. This segmentation allows each storage type to be optimized for its specific function, with page storage providing sequential write efficiency and block storage providing random read performance, thereby resolving the contradiction between productivity and reliability.
Solution Approach 2:
Different storage qualities are applied to different operations: page storage with its sequential access characteristics is used locally for write operations, while block storage with its random access characteristics is used locally for read operations. This local quality optimization enables high IOPs and throughput for writes while maintaining session consistency through the file system's page management on the client side.
2Productivity
If a single storage technology is used for both reads and writes, then system complexity is reduced, but performance factors such as speed, reliability, cost, and scale are compromised
Solution Approach 1:
The storage architecture dynamically selects different storage technologies based on the operation type: page storage is used for write operations and block storage is used for read operations. This dynamic approach allows the system to optimize performance for each operation type while maintaining a unified file system interface, thereby improving speed without significantly increasing perceived complexity for users.
Solution Approach 2:
The cloud file system serves multiple functions by integrating both page storage and block storage backends. It provides sequential write capability through page storage and random read capability through block storage, while maintaining a single unified interface for users. This multi-functionality enables the system to achieve high performance for both read and write operations without requiring separate file systems.
3Speed
If page storage is used for writes during user sessions, then session consistency is preserved, but read speed is limited due to the same storage technology constraints
Solution Approach 1:
The read and write operations are segmented to use different storage technologies: block storage for reads and page storage for writes. This segmentation enables fast random reads from block storage while maintaining session consistency through the file system's page-based write operations, thereby resolving the contradiction between read speed and session consistency.
Solution Approach 2:
The file system acts as an intermediary that translates between the block storage interface used for reads and the page storage interface used for writes. During user sessions, the file system manages page-based writes to preserve session consistency, while allowing direct block-based reads for high speed, effectively mediating between the two storage technologies.
4Productivity
If block storage is used for reads, then read performance is improved, but the ability to ensure files are stored in designated order is reduced compared to page storage
Solution Approach 1:
The system segments write operations to use page storage which maintains sequential order, while read operations use block storage for performance. This segmentation allows write efficiency to be maintained through ordered page writes, while read performance is improved through block storage, resolving the contradiction between write efficiency and file storage order.
Data Source
AI summary
Techniques are described herein that are capable of converting pages written in page storage during a user session to page-embedded blocks in block storage for reading. Blocks of first data are read from block storage during a user session. Pages of second data are written to page storage during the user session. The pages of the second data indicate changes to be made with regard to at least a subset of the blocks of the first data in the block storage. At a time instance at which no pages are being written to the page storage, the pages of the second data are transferred from the page storage to the block storage by converting the pages of the second data, which are configured to have a page format associated with the page storage, to page-embedded blocks, which are configured to have a block format associated with the block storage.