Multiqueued Cloud Storage Access via Union Virtual Block Device
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Single-threaded client-side software for cloud storage access limits performance and scalability due to resource utilization of only one CPU core, leading to inefficient file system request processing under load.
Innovation Solution
Implementing a multiqueued system by creating multiple mount points, instantiating block devices, and creating a union virtual block device that splits and delegates requests across multiple block devices, allowing for parallel processing and merging of results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single-threaded client-side software is used for cloud storage access, then the code execution is simple and easy to implement, but the performance is limited due to utilization of only one CPU core
Solution Approach 1:
The patent divides the single-threaded file system into multiple independent single-threaded file systems, each handling a specific queue of requests. This segmentation allows parallel processing across multiple CPU cores while maintaining the simplicity of single-threaded architecture within each file system instance.
Solution Approach 2:
The patent introduces a virtualization layer that acts as an intermediary between the client and the multiple single-threaded file systems. This virtualization layer manages request routing and coordination, enabling parallel processing without requiring complex multi-threaded synchronization logic in the file system code itself.
2Adaptability or versatility
If a single-queued file system is used, then the implementation is simpler, but the scalability is poor when the system is under load
Solution Approach 1:
The single queue is segmented into multiple independent queues, with each queue handled by a separate single-threaded file system instance. This allows the system to scale horizontally by adding more queue instances, improving adaptability under load while keeping individual queue management simple.
Solution Approach 2:
The patent transitions from a single-dimensional queue structure to a multi-dimensional architecture where multiple queues operate in parallel across different CPU cores. This dimensional expansion enables better scalability without increasing the complexity of individual queue management.
3Productivity
If a single process handles all file system requests, then the code execution is straightforward, but the number of requests processed per second is limited
Solution Approach 1:
The single request-handling process is segmented into multiple independent process instances, each capable of handling file system requests in parallel. This segmentation increases throughput by utilizing multiple CPU cores while maintaining straightforward process execution logic within each instance.
Solution Approach 2:
The patent creates multiple copies of the single-threaded file system process, each running independently on different CPU cores. This copying approach multiplies the request processing capacity while preserving the simplicity of the original single-process implementation.
Data Source
AI summary
Systems and methods are disclosed herein for multithreaded access to cloud storage. An exemplary method comprises creating a plurality of mount points by mounting, by a hardware processor, a plurality of file systems on a computer system, creating an image file on each of the plurality of mount points, instantiating, for each of the plurality of mount points, a block device on the image file, creating a union virtual block device that creates one or more stripes from each block device, delegating a request for accessing the union virtual block device, received from a client, to one or more block devices and merging a result of the request from each of the one or more block devices and providing the result to the client.