Virtual Node Deployment for Cluster Apps in Single Machines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Migrating cluster-based applications to single machine environments is challenging due to the need for significant source code changes and resource reallocation, which hinders efficient performance in multicore big-memory systems.
Innovation Solution
The deployment of cluster-based applications is facilitated by creating virtual nodes within a single machine environment, using OS containers to manage resource allocation and inter-thread communication, where each virtual node is assigned a core subset, IP address, and in-memory file system, allowing access to shared memory, thereby simulating a distributed cluster setup.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If cluster-based applications are migrated to single machine environments, then performance gains are achieved in multicore big-memory systems, but significant source code changes and resource reallocation are required
Solution Approach 1:
The patent creates virtual nodes that copy the appearance and behavior of distributed cluster nodes within a single machine environment. Each virtual node is assigned a core subset, IP address, and in-memory file system, creating a virtual copy of the distributed cluster architecture that allows applications to run without modification
Solution Approach 2:
The patent introduces a virtualization layer as an intermediary between the application and the physical hardware. This layer includes virtual nodes, in-memory file systems, and resource management components that mediate between the application's expectations of a distributed cluster and the actual single-machine environment
2Productivity
If cluster-based applications are migrated to single machine environments, then performance gains are achieved, but extensive modifications to deployment scripts and source code are needed
Solution Approach 1:
The patent copies the distributed cluster deployment structure into a virtualized single-machine environment, creating virtual nodes that replicate the original cluster's architecture. This allows deployment scripts to remain unchanged as they interact with virtual nodes that mimic the expected distributed environment
3Loss of time
If virtual nodes are created with in-memory file systems, then memory access time is minimized and latency in inter-node communications is reduced, but resource allocation complexity increases
Solution Approach 1:
The patent assigns each virtual node its own in-memory file system that is localized to that specific virtual node, optimizing memory access for each node independently. The resource management application manages these localized file systems and core subsets, allocating resources efficiently while maintaining the complexity benefits of virtualization
Data Source
AI summary
Examples relate to deploying distributed applications using virtual nodes. In some examples, virtual nodes are created and are each assigned a core subset of a number of processing cores, an Internet protocol (IP) address, and an in-memory file system configured to provide access to a portion of physically shared memory. At this stage, a distributed application that is configured to be deployed to a plurality of machine nodes is deployed to the plurality of virtual nodes. On a first virtual node, a reference to a first dataset stored in physically shared memory is sent to a second virtual node, where the physically shared memory is accessible to each of the plurality of virtual nodes. Next, on the second virtual node, the first dataset is accessed through the in-memory file system of the first virtual node.