Compute Node Installation Stream Sharing for Cluster Boot
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale compute clusters with diskless compute nodes face challenges in booting up quickly due to network saturation caused by the transfer of large installation images, which limits scalability and prolongs boot times, especially when using existing methods like file synchronization, peer-to-peer file sharing, and multicast network transport mechanisms.
Innovation Solution
The implementation of installation stream sharing, where helper compute nodes replicate and distribute installation streams to other client compute nodes, minimizing network congestion by selecting nodes to share installation streams in a way that avoids network saturation and eliminates the need for complex protocols like multicast.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If installation images are transferred to each compute node during boot, then the root file system can be provisioned, but network saturation occurs and boot times are prolonged
Solution Approach 1:
Multiple compute nodes are merged into a single installation stream receiver. The first compute node receives the installation image from the second compute node, and then relays it to subsequent nodes. This combining approach reduces the number of separate image transfers from thousands to just a few, eliminating network saturation while maintaining boot functionality.
Solution Approach 2:
The installation distribution process is segmented into hierarchical levels. The second compute node receives the image from the server, the first compute node receives it from the second node, and then the first node distributes it to multiple other nodes. This segmentation allows parallel distribution without overwhelming the network at any single point.
2Productivity
If traditional file synchronization or peer-to-peer sharing is used, then installation images can be distributed, but device complexity and protocol requirements increase
Solution Approach 1:
Compute nodes automatically perform installation stream sharing without requiring complex external coordination protocols. Each node that receives an installation stream automatically becomes a distributor to other nodes, eliminating the need for sophisticated peer-to-peer negotiation, multicast routing, or file synchronization protocols. The system self-organizes through simple receive-and-relay operations.
Data Source
AI summary
A technique includes a first compute node of a cluster of nodes receiving a communication from a cluster maintenance node of the cluster instructing the first compute node to provide an installation image that is received by the first compute node to a second compute node of the cluster. The first node and the second node are peers. The technique includes sharing the first installation stream that communicates the image to the first compute node. Sharing the first installation stream includes, while the first compute node is receiving the first installation stream, the first compute node providing a second installation stream to communicate parts of the image received via the first installation stream to the second compute node.