Parallel-prefix broadcast optimization via rank-based node segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Broadcasting the results of a parallel-prefix operation across thousands of compute nodes in a parallel computer is challenging due to substantial data communications costs, especially in large groups, where existing methods are inefficient and costly.
Innovation Solution
Configuring each compute node with a parallel-prefix contribution buffer and results buffer, where nodes determine if they contribute to the operation, using identity elements or their contributions, and performing the operation in parallel to yield results, which are then stored, optimizing data communications through a global combining network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional broadcast methods are used to distribute parallel-prefix results to all compute nodes, then all nodes receive the complete results array, but data communications overhead and costs become substantial
Solution Approach 1:
The patent segments the broadcast operation into selective contributions based on node rank. Each compute node contributes its parallel-prefix result only to specific lower-rank nodes according to a predetermined pattern, rather than broadcasting to all nodes. This segmentation reduces the total data communication volume while ensuring all nodes receive necessary information for correct operation.
Solution Approach 2:
The patent implements local quality by having each compute node's contribution behavior depend on its specific rank. Nodes with different ranks contribute to different sets of destinations, creating a differentiated communication pattern where each node's local contribution strategy is optimized for its position in the hierarchy, thereby reducing overall communication overhead.
2Productivity
If all compute nodes participate in the parallel-prefix operation, then comprehensive computation is achieved, but data communications costs increase substantially
Solution Approach 1:
The patent segments the computation participation by rank. Not all nodes perform all operations equally - lower-rank nodes receive results from higher-rank nodes and may contribute to even lower-rank nodes. This segmented participation maintains computational completeness while reducing redundant communications.
Solution Approach 2:
The patent uses preliminary action through the predetermined contribution pattern based on node ranks. The communication pattern is pre-determined by the rank assignment, allowing nodes to know in advance which nodes they should contribute to and which results they should expect, optimizing the communication-computation balance before execution begins.
Data Source
AI summary
A parallel-prefix broadcast for a parallel-prefix operation on a parallel computer includes: configuring, on each node, a parallel-prefix contribution buffer for storing the node's parallel-prefix contribution; configuring, on each node, a parallel-prefix results buffer for storing results of a operation, the results buffer having a position for each node that corresponds to node's rank; and repeatedly for each position in the results buffer: processing in parallel by each node, including: determining, by the node, whether the current position in the results buffer is to include the node's contribution, if the current position is not to include the contribution, contributing the identity element, and if the current position is to include the contribution, contributing the contribution, performing, by each node, the operation using the contributed identity elements and the contributed contributions, yielding a result from the operation, and storing, by each node, the result in the position in the results buffer.