Live-Migration of Applications in Distributed Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for managing multiple applications on a cluster of servers fail to effectively address redundancy and load-balancing issues in high-latency networked systems, as they either incur high costs, impose performance overhead, or are limited by geographical constraints and the risk of data corruption.
Innovation Solution
A server configured to operate in either a master or slave role, with logic for replicating data across a cluster, autonomously switching roles based on events such as failure or load changes, using snapshot replication and heartbeats to maintain data consistency and balance load across servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If virtualisation is used to replicate every change in memory and system state to a second physical host over a high-speed LAN, then automatic failover capability is improved, but performance overhead increases and costs increase
Solution Approach 1:
The patent applies partial replication by selectively copying only critical data structures (filesystem superblocks, inode information, allocation maps) rather than every memory change. This partial action approach provides sufficient failover capability while reducing the performance overhead and resource consumption associated with full memory state replication.
Solution Approach 2:
The invention extracts and replicates only the essential filesystem metadata and data structure information needed for failover, separating this critical data from the complete system state. This extraction approach enables automatic failover while minimizing the overhead of replicating unnecessary data.
2Productivity
If a shared storage backend is used for classical cluster load balancing, then load distribution is improved, but the storage system becomes a single point of failure and costs increase
Solution Approach 1:
The patent segments the storage dependency by creating independent filesystem images on each server's local storage. Instead of all servers depending on a single shared storage backend, each server maintains its own segmented copy of the filesystem data, eliminating the shared storage single point of failure while enabling load distribution across the cluster.
Solution Approach 2:
The invention uses copying to create redundant filesystem images on each server's local storage rather than accessing a shared storage backend. This copying approach allows any server to take over another's workload immediately without shared storage access, improving reliability while maintaining load balancing capability.
3Productivity
If manual intervention is used to move applications between servers during load spikes, then resource utilization can be adjusted, but recovery time increases and operational complexity increases
Solution Approach 1:
The patent implements self-service automation where the system automatically detects load imbalances and triggers application migration without manual intervention. The load balancer monitors server load and autonomously relocates applications from overloaded servers to underutilized ones, reducing recovery time while maintaining optimal resource utilization.
Solution Approach 2:
The invention incorporates feedback mechanisms where the load balancer continuously monitors server performance metrics and uses this information to automatically trigger migration decisions. This closed-loop feedback system enables rapid response to load changes, reducing recovery time compared to manual monitoring and intervention.
4Reliability
If quorum, fencing or STONITH mechanisms are implemented to prevent split-brain scenarios, then data corruption is prevented, but system complexity increases and operational costs increase
Solution Approach 1:
The patent uses copying to create independent filesystem image replicas on each server's local storage. This copying approach inherently prevents split-brain data corruption because each server works with its own consistent copy, eliminating the need for complex quorum mechanisms to coordinate access and prevent conflicting writes to shared storage.
Solution Approach 2:
The invention segments the filesystem data into independent images stored locally on each server rather than using shared storage. This segmentation approach prevents data corruption in split-brain scenarios by ensuring each server operates on its own data copy, eliminating the need for fencing or STONITH mechanisms to prevent concurrent access conflicts.
Data Source
AI summary
A method and apparatus for distribution of applications amongst a number of servers, ensuring that changes to application data on a master for that application are asynchronously replicated to a number of slaves for that application. Servers may be located in geographically diverse locations; the invention permits data replication over high-latency and lossy network connections and failure-tolerance under hardware and network failure conditions. Access to applications is mediated by a distributed protocol handler which allows any request for any application to be addressed to any server, and which, when working in tandem with the replication system, pauses connections momentarily to allow seamless, consistent live-migration of applications and their state between servers. Additionally, a system which controls the aforementioned live- migration based on dynamic measurement of load generated by each application and the topological preferences of each application, in order to automatically keep servers at an optimum utilisation level.