Distributed VM Placement via Message Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized virtual machine placement systems face scalability issues in large networks, are prone to single points of failure, and can lead to complex management and increased bandwidth usage due to race conditions and dynamic resource requirements.
Innovation Solution
A distributed message queue system where hosts subscribe to queues based on resource compatibility, allowing for efficient VM placement by matching VM requests with hosts that have the necessary resources, reducing the need for continuous updates and minimizing network traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized placement engine is used to manage VM placement, then placement decisions can be made with full information about host capabilities, but the system becomes non-scalable and creates a single point of failure in large networks with thousands of hosts
Solution Approach 1:
The patent divides the centralized placement engine into multiple distributed placement agents, one on each host. Each agent independently makes placement decisions for its local host based on received VM requests and host capability information, eliminating the single point of failure and enabling horizontal scaling across thousands of hosts while maintaining placement decision quality through distributed intelligence
Solution Approach 2:
The patent introduces a new dimensional approach by organizing placement decisions hierarchically: global placement policy configuration at the management plane, local placement agent execution at the host level, and message queue coordination for cross-host decisions. This multi-dimensional architecture allows the system to scale horizontally while maintaining centralized coordination where needed
2Reliability
If a centralized placement engine continuously updates host capacity information, then accurate placement decisions can be made, but bandwidth consumption increases and the system becomes impractical for large data centers with dynamic resource requirements
Solution Approach 1:
Each placement agent maintains local knowledge of its host's capability and capacity information, eliminating the need for continuous centralized updates. The agent receives VM placement requests and independently determines whether the local host can satisfy them, reducing network bandwidth consumption while maintaining accurate placement decisions through localized information processing
Solution Approach 2:
The system pre-configures host capability information locally at each placement agent before VM placement requests arrive. This preliminary local caching of host specifications allows agents to immediately evaluate VM requests without real-time network queries, reducing bandwidth consumption while maintaining placement accuracy through pre-available information
3Productivity
If multiple concurrent schedulers are used to handle VM placement requests, then processing capacity increases, but race conditions may result in placement failures
Solution Approach 1:
The patent introduces message queues as intermediary components between multiple concurrent placement agents and the VM placement requests. The message queue system provides ordered message processing, acknowledgment mechanisms, and retry logic that prevent race conditions and placement failures while allowing multiple agents to concurrently process requests, maintaining both high productivity and reliability
Data Source
AI summary
Machines, systems and methods for deploying virtual machines (VMs), the method comprises maintaining N message queues corresponding to N VM flavors, wherein the N VM flavors are associated with N distinguishable configuration requirements for supporting placement of at least a first VM on at least a first host with limited resources; wherein a first host from among a plurality of hosts subscribes to a first message queue from among the N message queues, in response to determining that the first host has the resources that match the respective configuration requirements for supporting the first VM placed on the first message queue; receiving a VM placement request for the first VM, wherein the VM placement request comprises information indicating a first VM flavor associated with a first message queue; communicating arrival of the VM request to one or more hosts subscribing to the first message queue.