Distributed Virtual Machine Resource Aggregation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional scale-up solutions and distributed frameworks face challenges in efficiently processing small datasets due to high configuration complexities, programming model difficulties, and runtime overheads, while traditional scale-up machines are costly and inefficient for small-scale data processing tasks.

Innovation Solution

Implementing a many-to-one virtualization method using QEMU to aggregate resources across multiple physical machines, enabling CPU, memory, and I/O virtualization, and providing a distributed shared memory system for consistent memory access and time synchronization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If distributed frameworks are used to process small datasets, then resource utilization improves, but configuration complexity and runtime overhead increase

Engineering Contradiction:

Improveresource utilizationVSAvoidconfiguration complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system segments the virtual machine into multiple parts distributed across different physical machines. Each physical machine hosts a portion of the virtual machine's memory and processing resources, allowing the virtual machine to leverage distributed resources while maintaining a unified virtual interface that abstracts away the complexity of distribution.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a distributed shared memory system as an intermediary layer between the virtual machine and the underlying distributed physical machines. This intermediary provides a unified memory address space that automatically handles memory management, synchronization, and coordination across multiple machines, eliminating the need for complex distributed programming models.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If scale-up machines are used for small datasets, then performance improves, but cost increases

Engineering Contradiction:

Improveprocessing performanceVSAvoidmachine resources

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The system merges resources from multiple commodity physical machines to create a unified virtual machine that provides scale-up performance. By combining the memory and processing power of multiple machines into a single virtualized environment, the system achieves high-performance processing for small datasets without requiring expensive dedicated scale-up hardware.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The virtual machine provides a universal interface that can run on various underlying hardware configurations. The same virtual machine image can be executed on different physical machine combinations, allowing flexible resource allocation and eliminating the need for specialized scale-up machines while maintaining consistent performance characteristics.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Adaptability or versatility

If distributed frameworks are used, then resource aggregation improves, but programming model complexity increases

Engineering Contradiction:

Improveresource aggregationVSAvoidprogramming model simplicity

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system creates a virtual copy of a single-machine environment that runs across distributed hardware. The virtual machine replicates the appearance and behavior of a conventional scale-up machine, allowing applications to run without modification. This copying approach enables resource aggregation from distributed machines while maintaining the simplicity of single-machine programming models.

Inventive Principle:

Principle #26Copying

4Adaptability or versatility

If virtualization is implemented across multiple machines, then resource utilization improves, but time synchronization complexity increases

Engineering Contradiction:

Improveresource utilizationVSAvoidsynchronization complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The distributed shared memory system acts as an intermediary that automatically handles time synchronization between multiple physical machines. The system implements hardware-supported virtualized memory that provides unified access semantics, automatically managing clock synchronization and memory consistency across the distributed infrastructure without requiring complex software synchronization protocols.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10853119B2GiantVM for resource aggregation

Publication Date: 2020.12.01 SHANGHAI JIAOTONG UNIV

AI summary

Described herein is a method for resource aggregation (many-to-one virtualization), comprising: virtualizing CPU by QEMU in a distributed way; organizing a plurality of memories scattered over different machines as pages to providing consistent memory view for guest OS; and performing time synchronization between different machines.