Intelligent Data Propagation in Distributed Environments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In highly distributed environments, the speed of cloning virtual machines (VMs) across numerous host devices is suboptimal due to network and datastore bottlenecks, leading to inefficient data propagation.
Innovation Solution
The method involves determining distance values between datastores, performing graph cluster analysis to create proximity-based clusters, and using a shortest path tree to prioritize data copying paths, while suspending and resuming copying based on datastore performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is copied concurrently to all destination datastores from a single source datastore, then the cloning operation can proceed in parallel, but network and datastore bottlenecks cause suboptimal copying speed
Solution Approach 1:
The system performs preliminary actions by calculating shortest path trees and identifying optimal source datastore selections before initiating the cloning operation. This allows the system to pre-determine the best data propagation paths and prepare multiple potential source datastores, enabling more efficient concurrent copying operations that avoid network and datastore bottlenecks.
Solution Approach 2:
The system introduces intermediary mechanisms including shortest path tree calculations and performance threshold monitoring as mediators between the cloning operation and data propagation. These intermediaries optimize the copying process by determining optimal paths and managing concurrent operations, thereby improving cloning speed while reducing deployment time.
2Productivity
If the entire file is copied from source to destination, then the copying process is simple, but the suboptimal copying speed persists regardless of copying method
Solution Approach 1:
The system segments the data copying process into multiple optimized paths using shortest path tree calculations. Instead of a single monolithic copy operation, the data is divided into multiple parallel copying streams through different paths, each managed independently. This segmentation increases propagation speed while the systematic approach to path selection keeps the overall process manageable.
Solution Approach 2:
The system dynamically adjusts the copying process by monitoring performance thresholds and adapting the data propagation paths in real-time. The shortest path tree is recalculated and updated based on actual copying performance, allowing the system to dynamically optimize the copying strategy. This dynamic approach improves propagation speed while adapting the complexity based on actual system conditions.
Data Source
AI summary
Exemplary methods, apparatuses, and systems that can intelligently copy data to a plurality of datastores are described. In one embodiment, a distance value of a path between each datastore is determined. Based on the distance values, a graph cluster analysis creates clusters of the datastores within close proximity to one another. Also, a shortest path tree determines the most efficient paths available for copying data from a source datastore to one or more destination datastores. The source datastore is designated as the root of the shortest path tree, and the one or more destination datastores are designated as the vertices of the tree. After each child vertex of the source datastore is ordered in descending order according to a number of unique clusters to which descendants of the child vertex belong, the data is copied from the source datastore to the one or more destination datastores in the descending order.