Method and apparatus for a hierarchical synchronization barrier in a multi-node system

Abstract

A hierarchical barrier synchronization of cores and nodes on a multiprocessor system, in one aspect, may include providing by each of a plurality of threads on a chip, input bit signal to a respective bit in a register, in response to reaching a barrier; determining whether all of the plurality of threads reached the barrier by electrically tying bits of the register together and “AND”ing the input bit signals; determining whether only on-chip synchronization is needed or whether inter-node synchronization is needed; in response to determining that all of the plurality of threads on the chip reached the barrier, notifying the plurality of threads on the chip, if it is determined that only on-chip synchronization is needed; and after all of the plurality of threads on the chip reached the barrier, communicating the synchronization signal to outside of the chip, if it is determined that inter-node synchronization is needed.

Description

FIELD[0001]The present application generally relates to computer architecture and more particularly to a synchronization barrier in a multi-node system.BACKGROUND[0002]In a multiprocessor system, synchronization between multi-nodes is frequently needed to synchronize all nodes. Synchronization is used to indicate that calculation on all nodes has reached a certain point. When synchronizing the nodes in a multi-node system, no node can continue processing until all nodes reach the synchronization point. This approach is used when, for example, partial results are calculated on all nodes in one phase of calculation, and then all partial results have to accumulate into a global result which is needed in the following phase of calculation. It is also used when successive phases of a calculation need to proceed in lock step across all nodes.[0003]Each node in a multi-node system can have one or more processor cores. One or more processor cores can be located on the same chip (i.e., integ...

AI Technical Summary

Benefits of technology

[0012]An apparatus for a hierarchical barrier synchronization of cores and nodes on a multiprocessor system, in one aspect, may include a plurality of cores arranged in an integrated circuit. A register may be operable to store input bit signals received from each of said plurality of cores. A control logic circuit may be operable to electrically tie and perform a Boolean “AND” function on said stored input bit signals to determine whether said plurality of cores all achieved barrier. The control logic circuit may be further operable to determine whether only on-ch...

Problems solved by technology

Barrier synchronization for on-chip and off-chip synchronization by using a BSR register introduces overhead to at least one processor, which needs to poll the BSR register until all processors reached the barrier.

To poll a register, a number of instructions has to be to executed to determine that synchronization is achieved and communicate this status on-chip and / or off-chip, resulting in a power consuming, energy-inefficient system and causing long latency for synchronization.

Images