Live-Migration of Applications in Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for managing multiple applications on a cluster of servers fail to effectively address redundancy and load-balancing issues in high-latency networked systems, as they either incur significant overhead, are geographically limited, or require expensive shared storage setups, and manual intervention for load balancing can lead to delayed recovery and data corruption.
Innovation Solution
A server configured to operate in either a master or slave role within a cluster, capable of autonomously replicating data and switching roles based on events such as failures or load changes, using snapshot replication and heartbeats to maintain data consistency and balance load across servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If virtualisation is used to replicate every change in memory and system state to a second physical host, then redundancy is improved, but performance overhead increases and geographical flexibility deteriorates
Solution Approach 1:
The patent extracts only the essential data changes (filesystem modifications) rather than replicating every memory and system state change. This selective replication approach reduces the overhead significantly while maintaining redundancy, as the system only tracks and replicates actual data modifications through inotify watchers and snapshot mechanisms.
Solution Approach 2:
The system takes preliminary snapshots of the filesystem before and after changes, and only replicates the differences between these snapshots. This preliminary action of capturing baseline states allows the system to efficiently determine what needs replication without continuously monitoring and copying all system changes.
2Reliability
If synchronous replication is used to ensure data consistency across servers, then data integrity is improved, but network bandwidth consumption increases and geographical flexibility deteriorates
Solution Approach 1:
The system takes preliminary snapshots before replication and only transmits the differences between snapshots rather than complete data sets. This preliminary capture of baseline states significantly reduces the quantity of data that needs to be transmitted over the network while maintaining consistency.
Solution Approach 2:
Instead of synchronous replication, the system uses asynchronous copying of filesystem differences between snapshots. This allows data to be replicated without blocking the primary operation, reducing network bandwidth consumption while still ensuring eventual consistency across replicas.
3Adaptability or versatility
If shared storage backend is used to support live migration, then application mobility is improved, but cost increases and single point of failure risk increases
Solution Approach 1:
The patent extracts the storage dependency from the live migration mechanism by using local filesystem snapshots and differences rather than requiring a shared storage backend. This extraction eliminates the need for complex shared storage infrastructure while maintaining application mobility capabilities.
Solution Approach 2:
The system introduces filesystem snapshots and difference tracking as intermediaries between the application data and the migration process. These intermediaries allow data to be transferred and replicated without requiring a shared storage backend, simplifying the architecture while enabling live migration.
4Productivity
If manual intervention is used to move applications between servers, then resource overhead is reduced, but recovery time increases
Solution Approach 1:
The system implements self-service automation where the replication and failover processes occur automatically without manual intervention. The master-slave architecture with automatic failure detection and data replication enables the system to self-heal and recover from failures, reducing both resource overhead and recovery time.
Solution Approach 2:
The system uses feedback mechanisms through inotify watchers and snapshot tracking to automatically detect changes and trigger replication and failover processes. This feedback-driven automation eliminates manual intervention while maintaining resource efficiency, as the system only activates replication when actual changes occur.
5Productivity
If load balancer appliances with shared storage are used, then load distribution is improved, but cost increases and geographical flexibility deteriorates
Solution Approach 1:
The patent extracts the load balancing functionality from dedicated appliances and integrates it directly into the application servers. Each server maintains its own state through local filesystem snapshots, eliminating the need for external load balancer appliances and shared storage while maintaining load distribution capabilities.
Solution Approach 2:
The system makes each server multi-functional by combining application hosting, data replication, and failover capabilities in a single unified architecture. This eliminates the need for specialized load balancer appliances, reducing device complexity and cost while maintaining load distribution through the master-slave replication mechanism.
Data Source
AI summary
A method and apparatus for distribution of applications amongst a number of servers, ensuring that changes to application data on a master for that application are asynchronously replicated to a number of slaves for that application. Servers may be located in geographically diverse locations; the invention permits data replication over high-latency and lossy network connections and failure-tolerance under hardware and network failure conditions. Access to applications is mediated by a distributed protocol handler which allows any request for any application to be addressed to any server, and which, when working in tandem with the replication system, pauses connections momentarily to allow seamless, consistent live-migration of applications and their state between servers. Additionally, a system which controls the aforementioned live-migration based on dynamic measurement of load generated by each application and the topological preferences of each application, in order to automatically keep servers at an optimum utilization level.