Customized Virtual Disk Architecture for Heterogeneous Storage Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data centers struggle to effectively handle applications with heterogeneous workloads due to their inability to provide customized hardware configurations, often resulting in sub-optimal performance or loss of economies of scale purchasing benefits.
Innovation Solution
A computing system architecture that utilizes a small number of storage server SKUs with distinct hardware configurations, allowing for the construction of customized virtual storage disks backed by different types of storage devices, such as HDDs and SSDs, to meet the specific needs of each application, while maintaining economies of scale purchasing power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a conventional data center uses coarse hardware configurations (e.g., all HDD or all SSD), then the hardware procurement is simple and economies of scale are maintained, but the storage performance is sub-optimal for applications with heterogeneous workloads
Solution Approach 1:
The storage system is segmented into multiple storage tiers (HDD and SSD) with distinct performance characteristics. The system allows applications to be allocated to different storage tiers based on their workload requirements, enabling fine-grained control over storage performance while maintaining manageable hardware complexity through standardized server SKUs.
Solution Approach 2:
The system dynamically allocates applications to different storage configurations based on their declared workload requirements. The storage allocation is not fixed but can be adjusted at runtime, allowing the system to adapt to heterogeneous workload needs while maintaining a relatively simple underlying hardware architecture.
2Productivity
If a cloud operator provides customized hardware configurations for each application, then storage performance is optimized for each application type, but the cloud operator loses economies of scale purchasing power
Solution Approach 1:
The system uses a small set of universal storage server SKUs that can serve multiple application types. Each SKU is designed to be multi-functional, supporting different storage configurations (HDD/SSD combinations) that can be allocated to applications based on their needs. This universality maintains economies of scale purchasing power while enabling optimized storage performance for diverse workloads.
Solution Approach 2:
The system changes storage parameters (type, capacity, performance characteristics) by allocating different combinations of storage devices to different application SKUs. Rather than customizing hardware at the physical level, the system achieves customization through parameter changes in the virtual storage configuration, maintaining standardized hardware procurement while enabling optimized performance.
3Ease of operation
If the cloud operator allocates storage based on coarse descriptions (high capacity or low latency), then the allocation process is simple, but applications with nuanced storage requirements receive sub-optimal hardware
Solution Approach 1:
The system creates virtual storage configurations that copy the desired storage characteristics into the virtual environment. Applications can specify their storage requirements, and the system creates virtual storage volumes that replicate the needed performance and capacity characteristics, providing precise matching without requiring complex physical hardware customization.
Solution Approach 2:
The system introduces a virtual storage layer as an intermediary between the physical storage devices and the applications. This virtual layer translates coarse application requirements into precise storage allocations, mediating between the simplicity of allocation processes and the precision of performance matching by abstracting the complexity in the virtual domain.
Data Source
AI summary
A computing system architecture that facilitates constructing a virtual disk that is customized for an application is described herein. An exemplary computing system having such architecture includes a first plurality of homogeneous storage servers, each storage server in the first plurality of storage servers comprising respective data storage devices of a first type. The exemplary computing system also includes a second plurality of homogeneous storage servers, each storage server in the second plurality of storage servers comprising respective data storage devices of a second type. A virtual disk that is customized for an application is constructed by mapping a linear (virtual) address space to portions of storage devices across the first plurality of storage servers and the second plurality of storage servers. The storage servers are accessible over a full bisection bandwidth network.