Multi-Instance 2LM Memory Tiering for Cloud VM Cost-Speed Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-based memory systems lack the ability to efficiently manage performance and cost tradeoffs for multi-instance virtual machines, as existing two-level memory systems are not designed for on-demand pricing tiers based on priority level access to system resources, making them unsuitable for cloud environments.
Innovation Solution
A multi-instance 2-Level-Memory (2LM) architecture with a virtual memory management module (vMMU) that dynamically manages DRAM and SCM across virtual machines, prioritizing memory access based on cost and performance levels, allowing for flexible performance and cost balancing through memory swapping and caching policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If SCM is used to bridge the performance and cost gap between flash memory and DRAM, then cost efficiency is improved, but performance (latency) deteriorates compared to DRAM
Solution Approach 1:
The memory system is segmented into multiple priority levels (first priority, second priority, third priority) with each level having different performance-cost characteristics. High-priority instances access faster memory (DRAM), while lower-priority instances access slower but cheaper memory (SCM), resolving the contradiction by serving different performance needs simultaneously
Solution Approach 2:
Different quality levels of memory service are provided locally to different instances based on their priority. The system provides high-performance service to high-priority instances and cost-effective service to lower-priority instances, allowing each instance to receive appropriate memory performance matching its needs
2Device complexity
If conventional single-instance 2LM systems are used, then hardware/software trade-off is simplified, but adaptability to multi-instance cloud environments deteriorates
Solution Approach 1:
The memory management module is designed with universal functionality to handle multiple instances with different priority levels in a single system. It implements a unified architecture that can serve both high-priority and lower-priority instances, making it adaptable to cloud environments while maintaining manageable complexity through standardized protocols
Solution Approach 2:
The system dynamically adjusts memory allocation and access rights based on instance priority levels. The memory management module can flexibly manage resources across multiple instances with different performance requirements, enabling adaptability to varying cloud workloads while maintaining a consistent management framework
3Speed
If high priority instances access faster memory, then performance is improved, but cost increases
Solution Approach 1:
The memory resource pool is segmented into different priority tiers. High-priority instances are allocated access to faster DRAM memory, while lower-priority instances access slower SCM memory. This segmentation allows the system to optimize performance for critical workloads while controlling costs for less critical workloads
Solution Approach 2:
Different memory performance quality is provided locally to different instance types based on their priority classification. The system matches memory service quality to instance requirements, ensuring high performance where needed while avoiding unnecessary expenditure on high-performance memory for lower-priority instances
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A multi-instance 2-Level-Memory (2LM) architecture manages access by processing instances having different memory usage priorities to memory having different performance and cost levels. The 2LM architecture includes a virtual memory management module that manages access by respective processing instances by creating memory instances based on specified memory usage priority levels and specified virtual memory sizes and defining policies for each usage priority level of the created memory instances. In response to a virtual memory request by a processing instance, the virtual memory management module determines whether a virtual memory size at a designated usage priority level requested by a processing instance can be satisfied by a policy of a created first memory instance and, if not, selects another memory instance that can satisfy the requested virtual memory size at the designated usage priority level and swaps out the first memory instance in favor of the other memory instance.