Parallel Barrier Synchronization Reducing Local Overhead

Solution Overview

Problem

In parallel computing, global barrier operations often require multiple local barriers, increasing overhead and reducing efficiency, as existing methods do not effectively minimize the number of local barriers needed during global synchronization.

Innovation Solution

Each compute node determines if it is a master task; if not, it joins a single local barrier, and if it is a master task, it joins the global barrier only after all other tasks on the node have joined the local barrier, thereby reducing the number of local barriers needed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple local barriers are used for global barrier operation, then all tasks can be synchronized, but computational overhead increases and efficiency decreases

Engineering Contradiction:

Improvesynchronization reliabilityVSAvoidbarrier operation efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent merges multiple local barrier operations into a single local barrier operation. Instead of having each task join multiple separate local barriers before the global barrier, the invention combines these into one unified local barrier that all tasks join together, thereby reducing overhead while maintaining synchronization reliability.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces a preliminary action where master tasks first collect counts of non-master tasks from all compute nodes, then use this information to determine when to join the global barrier. This preliminary counting action enables master tasks to synchronize at the optimal moment after all non-master tasks have joined the single local barrier, eliminating the need for multiple local barriers.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If multiple local barriers are used for global barrier operation, then task synchronization is ensured, but computational overhead increases

Engineering Contradiction:

Improvetask synchronizationVSAvoidbarrier operation complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent merges multiple local barrier operations into a single local barrier operation. Instead of having each task join multiple separate local barriers before the global barrier, the invention combines these into one unified local barrier that all tasks join together, thereby reducing overhead while maintaining synchronization reliability.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent extracts the synchronization function from multiple separate local barriers and consolidates it into a single local barrier operation. By taking out the essential synchronization requirement and implementing it in one unified barrier rather than multiple separate barriers, the invention reduces complexity while maintaining the same synchronization reliability.

Inventive Principle:

Principle #2Taking out (Extraction)

3Productivity

If master tasks join global barrier immediately, then global synchronization is achieved, but local task completion is not ensured

Engineering Contradiction:

Improveglobal barrier speedVSAvoidlocal task completion

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent introduces a preliminary action where master tasks first collect counts of non-master tasks from all compute nodes, then use this information to determine when to join the global barrier. This preliminary counting action enables master tasks to synchronize at the optimal moment after all non-master tasks have joined the single local barrier, eliminating the need for multiple local barriers.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses feedback from non-master tasks to determine when master tasks should join the global barrier. Each non-master task provides feedback by joining the single local barrier, and master tasks use this feedback information (task counts) to determine the optimal moment to join the global barrier, ensuring all local tasks are complete before global synchronization occurs.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS8910178B2Performing a global barrier operation in a parallel computer

Publication Date: 2014.12.09 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Executing computing tasks on a parallel computer that includes compute nodes coupled for data communications, where each compute node executes tasks, with one task on each compute node designated as a master task, including: for each task on each compute node until all master tasks have joined a global barrier: determining whether the task is a master task; if the task is not a master task, joining a single local barrier; if the task is a master task, joining the global barrier and the single local barrier only after all other tasks on the compute node have joined the single local barrier.