Joining a sync zone

Abstract

A set of configurable synchronization groups (which may be called synchronization zones) is defined. Any processor may belong to any synchronization zone. Each processor contains a register that indicates which synchronization zone it belongs to. If a processor does not belong to a synchronization zone, it continuously asserts its synchronization request for that synchronization zone to the synchronization controller. If a processor belongs to a synchronization zone, it asserts its synchronization request for that synchronization zone only when it reaches a synchronization point for that synchronization zone that is indicated in its compiled code set.

Description

[Technical Field]

[0001] The present disclosure relates to data processing devices including multiple processors, and more particularly to coordinating synchronization involving processors of the multiple processors. [Background technology]

[0002] In connection with processing data for complex or large amounts of applications, a processing device may be provided to perform the processing of that data. The processing device may function as a work accelerator where processing of specific data is offloaded from the host system. Such a processing unit may have dedicated hardware for performing specific types of processing.

[0003] As an example, one computing field in which such dedicated accelerator subsystems can be used is found in machine intelligence. As is well known to those skilled in the art of machine intelligence, machine intelligence algorithms center around performing iterative updates to a "knowledge model," which can be represented by a graph of multiple interconnected nodes. The implementation of each node involves processing data, and the interconnections of the graph correspond to the data exchanged between the nodes. Typically, at least some of the processing of each node can be performed independently of some or all other nodes in the graph; therefore, large graphs expose significant opportunities for multithreading. Thus, processing devices specialized for machine intelligence applications can have a high degree of multithreading. Some forms of parallelism can be achieved by configuring multiple processor tiles on the same chip (i.e., the same die), with each processor tile including its own separate execution units and memory (including program memory and data memory). Thus, separate portions of program code can be executed in parallel in different tiles.

[0004] When a processing device including multiple processors operating in parallel is provided, a technique is needed to prevent code executed on one processor from executing before data on which that code depends, provided by other code executed on another processor. There are several possible approaches to achieving this, one of which is described herein as an example as "BSP" bulk synchronous parallelism. According to BSP, each processing unit executes a computation phase and an exchange phase in alternating cycles. During the computation phase, each processing unit executes one or more computational tasks locally on the processing unit but does not communicate any results of the computation with other processing units. During the exchange phase, each processing unit is allowed to exchange one or more results of the computation from the previous computation phase with one or more other processing units. Furthermore, according to the BSP principle, barrier synchronization is placed at the junction point where the computation phase transitions to the exchange phase, the junction point where the exchange phase transitions to the computation phase, or both.

[0005] To coordinate synchronization (e.g., barrier synchronization) between processors, a central synchronization controller is provided that receives synchronization requests from each set of processors to be synchronized together and may return a synchronization acknowledgment once synchronization requests have been received from all processors that participate in the synchronization together. Summary of the Invention [Problem to be solved by the invention]

[0006] In some cases, all processors belonging to a processing device may participate in synchronization. However, at some synchronization points, some of the processors may not have data to exchange with other processors. Therefore, it has been proposed to allow some processors of a processing device to operate asynchronously with other processors of the processing device. However, at a later point in time, two or more groups of processors that may behave asynchronously with each other may be required to synchronize together at a synchronization point. Therefore, it is desirable to provide a mechanism that allows different groups of tiles to synchronize flexibly. [Means for solving the problem]

[0007] According to a first aspect, there is provided a data processing device comprising: a plurality of processors; and a synchronization controller including circuitry configured to receive requests from the processors to join synchronization and to return an acknowledgement to the processors in response to receiving the request, each of the processors comprising: an execution unit configured to execute a set of computer readable instructions held in a memory of the respective processor; and a register storing, for each of a set of configurable synchronization groups, an indication as to whether the respective processor belongs to the respective configurable synchronization group, wherein for a first of the processors, the indication for a first of the configurable synchronization groups indicates that the first of the processors does not belong to the first of the configurable synchronization groups, and an indication for a second of the configurable synchronization groups includes circuitry configured to assert to the synchronization controller a request to join synchronization of the first one of the configurable synchronized groups in response to an instruction for a first one of the processors indicating that the first one of the processors belongs to a second one of the configurable synchronized groups, and the first one of the processors does not belong to the first one of the configurable synchronized groups, and the circuitry of the first one of the processors is configured to assert to the synchronization controller a request to join synchronization of the second one of the configurable synchronized groups in response to an execution unit of the first one of the processors reaching a synchronization point of the second one of the configurable synchronized groups indicated in the set of computer-readable instructions for the first one of the processors.

[0008] A configurable set of synchronization groups (which may be called synchronization zones) is defined. Any processor may belong to any synchronization zone. Each processor includes a register indicating the synchronization zone to which it belongs. If a processor does not belong to a synchronization zone, it continuously asserts a synchronization request for that synchronization zone to the synchronization controller. If a processor belongs to a synchronization zone, it asserts its synchronization request for that synchronization zone only when it reaches a synchronization point for that synchronization zone indicated in its compiled code set. In this way, only when all processors belonging to a particular synchronization zone have reached a synchronization point, the synchronization controller has received synchronization requests for that synchronization zone from all processors in the device (including processors belonging to that zone and processors not belonging to that zone) and can proceed to cause synchronization acknowledgments to be sent to the processors of the device.

[0009] In some embodiments, the circuitry of the synchronization controller is configured to issue a corresponding acknowledgement to each of the processors in response to all of the processors of the data processing device issuing a request to join synchronization of a first one of the configurable synchronization groups.

[0010] In some embodiments, the synchronization controller circuitry is configured to, in response to a request to join synchronization of a first one of the configurable synchronization groups, cause the processor to issue a further request to the external synchronization controller to synchronize with further processors belonging to further devices, and subsequently, in response to receiving a further acknowledgment of the further request from the external synchronization controller, return a corresponding acknowledgment to each of the processors.

[0011] In some embodiments, the data processing device includes an external synchronization controller that includes storage that stores a set of configuration settings for a first synchronization group of the configurable synchronization groups, and circuitry configured to exchange one or more additional requests and one or more additional acknowledgments with the further device in response to a further request received from the synchronization controller, depending on the configuration settings of the first synchronization group of the configurable synchronization groups.

[0012] In some embodiments, for each of the processors belonging to a first one of the configurable synchronous groups, an execution unit of the respective processor is configured to issue a request to join synchronization of the first one of the configurable synchronous groups upon reaching a first barrier synchronization implemented between the processors belonging to the first one of the configurable synchronous groups, and for the first one of the processors, the respective execution unit is configured to proceed with computation or data exchange without waiting at the first barrier synchronization while the execution unit of each of the processors belonging to the first one of the configurable synchronous groups is paused and waiting at the first synchronization barrier.

[0013] In some embodiments, the synchronization of a second one of the configurable synchronization groups is a second barrier synchronization, and the execution unit of a first one of the processors is configured to proceed beyond the second barrier synchronization in response to receiving an acknowledgment to a request to join the synchronization of the second one of the configurable synchronization groups.

[0014] In some embodiments, the execution unit of a first one of the processors is configured to proceed beyond the second barrier synchronization by entering an exchange phase in which the first one of the processors at least one of sends or receives data.

[0015] In some embodiments, the execution unit of a first one of the processors is configured to, after asserting a request to join synchronization of the first one of the configurable synchronization groups, execute an update instruction to update an indication for the first one of the configurable synchronization groups to specify that the first one of the processors belongs to the first one of the configurable synchronization groups.

[0016] In some embodiments, the execution unit of a first one of the processors is configured to, after asserting a request to join synchronization of a second one of the configurable synchronization groups, execute an update instruction to update an indication for the second one of the configurable synchronization groups to specify that the first one of the processors does not belong to the second one of the configurable synchronization groups.

[0017] In some embodiments, the data processing device includes an aggregation circuit configured to provide a first aggregate synchronization request to the synchronization controller in response to all of the processors providing respective requests to participate in synchronization of a first one of the configurable synchronization groups, and to provide a second aggregate synchronization request to the synchronization controller in response to all of the processors providing requests to participate in synchronization of a second one of the configurable synchronization groups, and the circuitry of the synchronization controller configured to return an acknowledgement to the processor in response to each of the first aggregate synchronization request and the second aggregate synchronization request.

[0018] In some embodiments, the data processing device includes a first synchronization request wire connected to a first one of the processors and a second synchronization request wire connected to the first one of the processors, wherein circuitry of the first one of the processors is configured to assert a request to join synchronization of a first one of the configurable synchronization groups by asserting a signal on the first synchronization request wire, and circuitry of the second one of the processors is configured to assert a request to join synchronization of a second one of the configurable synchronization groups by asserting a signal on the second synchronization request wire.

[0019] In some embodiments, the circuitry of a first one of the processors includes a first multiplexer configured to select a first input and output a first signal representing a request to join synchronization of the first one of the configurable synchronization groups in response to an indication that the first one of the processors does not belong to a first one of the configurable synchronization groups.

[0020] In some embodiments, the circuitry of a first one of the processors includes a second multiplexer configured to select a second input to output a second signal controlled by the execution unit in response to an indication that the first one of the processors belongs to a second one of the configurable synchronous groups.

[0021] In some embodiments, the execution unit of a first one of the processors is configured, upon reaching a synchronization point, to execute a synchronization instruction to cause a second signal to be set to a state representing a request to join synchronization of a second one of the configurable synchronization groups.

[0022] In some embodiments, a second one of the configurable synchronization groups includes further processors belonging to one or more further data processing devices, and a first one of the processors is configured to participate in synchronization of the second one of the configurable synchronization groups by exchanging data with one or more of the further processors.

[0023] In some embodiments, an execution unit of a first one of the processors is configured to, upon reaching a synchronization point of a second one of the configurable synchronization groups, execute a synchronization instruction to cause assertion of a request to join the synchronization of the second one of the configurable synchronization groups.

[0024] In some embodiments, circuitry of a first one of the processors is configured to receive a control signal from an execution unit of the first one of the processors indicating that the execution unit has reached a synchronization point, and to translate the control signal into a request to join synchronization of a second one of the configurable synchronization groups.

[0025] In some embodiments, a first one of the processors includes a first interface configured to receive a first acknowledgment signal of a first one of the configurable groups of synchrony, and circuitry of the first one of the processors is configured to invert the first acknowledgment signal to generate a request to join synchronization of the first one of the configurable groups of synchrony, and a second one of the processors includes a second interface configured to receive a second acknowledgment signal of a second one of the configurable groups of synchrony, and circuitry of the second one of the processors is configured to invert the second acknowledgment signal to generate a request to join synchronization of the second one of the configurable groups of synchrony.

[0026] In some embodiments, the data processing device is an integrated circuit.

[0027] According to a second aspect, there is provided a method implemented in a data processing device including a plurality of processors, the method comprising: storing, in each of the processors, for each of a set of configurable synchronization groups, an indication as to whether the respective processor belongs to the respective configurable synchronization group; and executing a set of computer readable instructions retained in a memory of the respective processor, wherein, for a first one of the processors, the indication for the first one of the configurable synchronization groups indicates that the first one of the processors does not belong to the first one of the configurable synchronization groups and the indication for a second one of the configurable synchronization groups indicates that the first one of the processors belongs to the second one of the configurable synchronization groups; the method further comprising: receiving a request to join synchronization from a processor and returning an acknowledgment to the processor in response to receiving the request; asserting, in a first one of the processors, a request to join synchronization of a first one of the configurable synchronization groups to a synchronization controller in response to an indication for a first one of the configurable synchronization groups indicating that the first one of the processors does not belong to the first one of the configurable synchronization groups; and asserting, in response to an execution unit of the first one of the processors reaching a synchronization point of a second one of the configurable synchronization groups indicated in the set of computer-readable instructions for the first one of the processors, a request to join synchronization of a second one of the configurable synchronization groups to the synchronization controller.

[0028] In some embodiments, the method includes, in the synchronization controller, issuing a corresponding acknowledgement to each of the processors of the data processing device in response to all of the processors issuing a request to join synchronization of a first one of the configurable synchronization groups.

[0029] In some embodiments, the method includes, in the synchronization controller, in response to a request to join synchronization of a first synchronization group of the configurable synchronization groups, the processor issuing a further request to the external synchronization controller to synchronize with further processors belonging to further devices, and subsequently, in response to receiving a further acknowledgment of the further request from the external synchronization controller, returning a corresponding acknowledgment to each of the processors.

[0030] In some embodiments, the method includes storing, in the external synchronization controller, a set of configuration settings for a first synchronization group of the configurable synchronization groups, and in response to a further request received from the synchronization controller, exchanging one or more additional requests and one or more additional acknowledgments with the further device according to the configuration settings of the first synchronization group of the configurable synchronization groups.

[0031] In some embodiments, the method includes, for each of the processors belonging to a first one of the configurable synchronization groups, upon reaching a first barrier synchronization implemented between the processors belonging to the first one of the configurable synchronization groups, issuing a request to join synchronization of the first one of the configurable synchronization groups; and, for the first one of the processors, proceeding with computation or data exchange without waiting at the first barrier synchronization while an execution unit of each of the processors belonging to the first one of the configurable synchronization groups is paused and waiting at the first synchronization barrier.

[0032] In some embodiments, the synchronization of a second one of the configurable synchronization groups is a second barrier synchronization, and the method includes, at a first one of the processors, proceeding beyond the second barrier synchronization in response to receiving an acknowledgment to a request to join the synchronization of the second one of the configurable synchronization groups.

[0033] In some embodiments, the method includes, in a first one of the processors, progressing beyond the second barrier synchronization by entering an exchange phase in which the first one of the processors at least one of transmits or receives data.

[0034] In some embodiments, the method includes, at a first one of the processors, executing an update instruction that updates an indication for a first one of the configurable synchronization groups to specify that the first one of the processors belongs to the first one of the configurable synchronization groups after asserting a request to join synchronization of the first one of the configurable synchronization groups.

[0035] In some embodiments, the method includes, at a first one of the processors, executing an update instruction that updates an instruction for a second one of the configurable synchronization groups to specify that the first one of the processors does not belong to the second one of the configurable synchronization groups after asserting a request to join synchronization of the second one of the configurable synchronization groups.

[0036] In some embodiments, the method includes providing a first aggregate synchronization request to the synchronization controller in response to all of the processors providing respective requests to participate in synchronization of a first one of the configurable synchronization groups; providing a second aggregate synchronization request to the synchronization controller in response to all of the processors providing requests to participate in synchronization of a second one of the configurable synchronization groups; and in response to each of the first aggregate synchronization request and the second aggregate synchronization request, the synchronization controller returning an acknowledgement to the processor.

[0037] In some embodiments, the data processing device includes a first synchronization request wire connected to a first one of the processors and a second synchronization request wire connected to the first one of the processors, and the method includes the first one of the processors asserting a request to join synchronization of a first one of the configurable synchronization groups by asserting a signal on the first synchronization request wire, and the second one of the processors asserting a request to join synchronization of a second one of the configurable synchronization groups by asserting a signal on the second synchronization request wire.

[0038] In some embodiments, the method includes, in a first multiplexer belonging to a first one of the processors, selecting a first input and outputting a first signal representing a request to join synchronization of the first one of the configurable synchronization groups in response to an indication that the first one of the processors does not belong to a first one of the configurable synchronization groups.

[0039] In some embodiments, the method includes, at a second multiplexer belonging to a second one of the processors, selecting a second input to output a second signal controlled by the execution unit in response to an indication that a first one of the processors belongs to a second one of the configurable synchronous groups.

[0040] In some embodiments, the method includes causing a first one of the processors, upon reaching a synchronization point, to execute a synchronization instruction to set a second signal to a state representing a request to join synchronization of a second one of the configurable synchronization groups.

[0041] In some embodiments, a second one of the configurable synchronization groups includes further processors belonging to one or more further data processing devices, and the method includes a first one of the processors participating in synchronization of the second one of the configurable synchronization groups by exchanging data with one or more of the further processors.

[0042] In some embodiments, the method includes, when a first one of the processors reaches a synchronization point of a second one of the configurable synchronization groups, executing a synchronization instruction to cause assertion of a request to join the synchronization of the second one of the configurable synchronization groups.

[0043] In some embodiments, the method includes receiving, at a first one of the processors, a control signal from an execution unit of the first one of the processors indicating that the execution unit has reached a synchronization point, and translating the control signal into a request to join synchronization of a second one of the configurable synchronization groups.

[0044] In some embodiments, a first one of the processors includes a first interface, and the method includes receiving, at the first interface, a first acknowledgment signal of a first one of the configurable groups of synchronizations, and inverting, at the first one of the processors, the first acknowledgment signal to generate a request to join synchronization of the first one of the configurable groups of synchronizations; and a first one of the processors includes a second interface, and the method includes receiving, at the second interface, a second acknowledgment signal of a second one of the configurable groups of synchronizations, and inverting, at the second one of the processors, the second acknowledgment signal to generate a request to join synchronization of the second one of the configurable groups of synchronizations.

[0045] In some embodiments, the data processing device is an integrated circuit.

[0046] For a better understanding of the present invention, and to show how the same may be carried into effect, reference will now be made, by way of example, to the accompanying drawings in which: FIG. [Brief explanation of the drawings]

[0047] [Figure 1] 1 illustrates an example of a multi-tile processing unit. [Figure 2] FIG. 1 is a schematic diagram illustrating the calculation and exchange phases within a multi-tile processing unit. [Figure 3] 1 illustrates the exchange of data in a bulk synchronous parallel system. [Figure 4] 1 is a schematic diagram of an internal synchronization barrier and an external synchronization barrier; [Figure 5] 1 is a schematic diagram of an integrated circuit including a multi-tile processing unit and a synchronization controller circuit. [Figure 6] FIG. 2 is a schematic diagram of a processor tile. [Figure 7] 1 shows the state of a tile's synchronization request wire and the timeline of the corresponding synchronization acknowledge wire for that tile. [Figure 8A] FIG. 10 is a schematic diagram of a synchronization output interface of a tile that outputs synchronization requests towards a synchronization controller of a processing unit. [Figure 8B] FIG. 1 is a schematic diagram of a synchronous input interface including circuitry for generating a synchronous acknowledge pulse in response to an edge in a synchronous acknowledge signal. [Figure 9] FIG. 10 is a schematic diagram of a synchronization aggregation circuit that aggregates the synchronization request states of all tiles of a processing unit for sending to a synchronization controller. [Figure 10] FIG. 10 is a schematic diagram of the synchronization distribution wiring that sends synchronization acknowledgement signals to all tiles of a processing unit. [Figure 11] FIG. 10 is a schematic diagram of a circuit that aggregates the synchronization request states output by two pairs of tiles with the upstream synchronization request state. [Figure 12] FIG. 10 is a schematic diagram of a circuit for aggregating synchronization request states for a column of tiles. [Figure 13] FIG. 10 is a schematic diagram of a circuit that aggregates synchronization request states from multiple columns of tiles. [Figure 14A] FIG. 10 is a schematic diagram of a circuit within a synchronization controller that provides a synchronization acknowledgement to a processing unit in response to receiving a synchronization request. [Figure 14B] 1 is a schematic diagram of a synchronization signaling scheme used for signaling external synchronization requests and acknowledgments; [Figure 15]FIG. 1 is a schematic diagram of the division of tiles of processing units between different synchronization zones and the transmission of synchronization requests by tiles of those zones. [Figure 16] FIG. 10 is a schematic diagram of the division of tiles of processing units between different synchronization zones and the sending of synchronization acknowledgements to tiles of those zones. [Figure 17] 1 is a schematic diagram of the exchange of synchronization requests and acknowledgments between GSPs in the system. [Figure 18] 1 is a schematic diagram of a system in which an outer synchronization zone is implemented. [Figure 19] FIG. 1 is a diagram of an example synchronization network. [Figure 20] 1 illustrates an example of how synchronization can be performed between different configurable synchronization groups of processors. [Figure 21] 1 illustrates an example of a method for signaling a synchronization request. DETAILED DESCRIPTION OF THE INVENTION

[0048] Referring to Figure 1, Figure 1 illustrates an example processing unit 2 including multiple processors 4. In the described embodiment, the processors 4 are presented as being tiles 4. However, the tiles 4 may be more generally described as being processors 4. Each such processing unit 2 is formed on an integrated circuit. The illustrated multi-tile processing unit 2 is described in U.S. Patent Application Publication No. 15 / 886,065, which is incorporated by reference.

[0049] The processing unit 2 includes an array 6 of multiple processing tiles 4 and an interconnect 34 connecting the tiles 4. The processing unit 2 may be implemented singly or as one of multiple dies packaged in the same IC package. The interconnect 34 is sometimes referred to herein as a "switching fabric" 34 because it allows the tiles 4 to exchange data with each other. Each tile 4 includes respective instances of execution units and memory. For example, by way of illustration, the processing unit 2 may be composed of as many as hundreds of tiles 4 or even more than a thousand tiles 4. For completeness, it should also be noted that the "array" referenced herein does not necessarily imply any particular number of dimensions or physical layout of the tiles 4.

[0050] In an embodiment, each processing unit 2 also includes one or more external links that allow the processing unit 2 to connect to one or more other processing units (e.g., one or more other instances of the same processing unit 2). These external links may allow the processing unit 2 to connect to a host system and to one or more other instances of processing unit 2 on the same IC package or card or on different cards. The processing unit 2 receives work from the host in the form of application data to process.

[0051] The interconnect 34 is configured to allow different tiles 4 within the array 6 to communicate with each other. However, just as there can potentially be dependencies between threads on the same tile 4, there can also be dependencies between portions of a program running on different tiles 4 within the array 6. Therefore, a technique is needed to prevent code on one tile 4 from executing before data on which it depends that is provided by other code on another tile 4. This is achieved using a data consistency model.

[0052] Parallel programming models for AI and data science typically follow a three-phase iterative execution model: compute, barrier, and exchange. Data exchange with processors typically relies on barriers to provide data consistency between processors and between each processor and external storage. Typically used data consistency models are bulk synchronous parallel (BSP), stall synchronous parallel (SSP), and asynchronous. The processing unit 2 described herein uses the BSP model, but it will be understood that other synchronization models may alternatively be utilized.

[0053] 2 and 3, which illustrate an implementation of a BSP exchange scheme in which each tile 4 executes a computation phase 33 and an exchange phase 32 in alternating cycles, separated by a barrier synchronization 30 between tiles 4. In the example shown in FIGS. 2 and 3, a barrier synchronization is placed between each computation phase 33 and the following exchange phase 32. During the computation phase 33, each tile 4 executes one or more computational tasks locally on the tile, but does not communicate any results of these computations to any other tiles in the tile 4. In the exchange phase 32, each tile 4 is allowed to exchange one or more results of a computation from a previous computation phase with and / or from one or more other tiles in the tile 4, but does not execute any new computation until it receives any data on which the task depends from the other tiles 4. It is not excluded that other operations, such as internal control-related operations, may be performed in the exchange phase 32. Communication outside the tile group may optionally utilize a BSP mechanism, but may alternatively not utilize a BSP and instead use some other synchronization mechanism of its own.

[0054] According to BSP principles, the barrier synchronization 30 is located at the junction of the transition from a computation phase 33 to an exchange phase 32, or the junction of the transition from an exchange phase 32 to a computation phase 33, or both. That is, (a) all tiles 4 in a group must first complete each computation phase 33 before any tile in the group is allowed to proceed to the next exchange phase 32, or (b) all tiles 4 in a group must first complete each exchange phase 32 before any tile in the group is allowed to proceed to the next computation phase 33, or (c) both of these conditions are enforced. In all three variants, it is the individual tiles 4 and the synchronizing assemblies that change between phases. The sequence of exchange and computation phases can then be repeated for multiple iterations. In BSP terminology, each iteration of an exchange phase and a computation phase is sometimes called a "superstep" (however, note that in the literature this term is not always used consistently, and each individual exchange phase and computation phase is sometimes called a superstep, while in other cases the exchange phase and computation phase together are called a superstep, as is the terminology adopted herein).

[0055] 3 shows (a) the BSP principle implemented between some or all groups 4i, 4ii, 4iii of tiles in an array 6 when imposing a barrier synchronization (see above) from the computation phase 33 to the exchange phase 32. Note that in this configuration some tiles 4 are allowed to start computation 33 while some other tiles 4 are still exchanging.

[0056] The BSP model can be used for data exchange between tiles 4 in a processing unit 2. Communication between tiles 4 of a processing unit 2 occurs time-deterministically, and data packets are transmitted without headers as in our earlier application, U.S. Patent Application Publication No. 15 / 886,065. Additionally, the BSP model can also be used for data exchange between processing units 2.

[0057] Referring to Figure 4, Figure 4 shows an example BSP program flow involving both internal synchronization (i.e., between tiles 4 of a single processing unit 2) and external synchronization (i.e., between processing units 2). As shown, the flow includes an internal exchange 50 (of data between tiles 4 of the same processing unit 2) and an external exchange 50' (of data between tiles 4 of different processing units 2). The program flow in Figure 4 shows the program flow for a first processing unit 2i and a second processing unit 2ii.

[0058] As shown in FIG. 4, internal BSP supersteps (including internal exchanges 50 of data between tiles 4 of the same processing unit 2) are kept separate from external synchronization and exchanges (including external exchanges 50' of data between tiles 4 of different processing units 2).

[0059] The program may be configured to execute a sequence of synchronization, exchange, and computation phases including, in the following order: (i) first a computation phase, then (ii) an internal barrier synchronization 30, then (iii) an internal exchange phase 50, then (iv) an external barrier synchronization 80, then (v) an external exchange phase 50'. The external barrier 80 is imposed after the internal exchange phase 50, so that the program proceeds to the external exchange 50' only after the internal exchange 50. Note also that, optionally, a computation phase may be included between the (iii) internal exchange and the (iv) external barrier, as shown with respect to processing unit 2I in FIG. 4.

[0060] This overall sequence is implemented by a program (e.g., itself generated by a compiler). In an embodiment, the program is programmed to operate in this manner by synchronization instructions executed by tile 4. The internal synchronization and exchange does not extend to any tiles or other entities on another processing unit 2. The sequence (i)-(v) (together with any computation phases described above included between (iii) and (iv)) can be repeated in a series of overall iterations. Per iteration, there can be multiple instances of internal computation, synchronization, and exchange (i)-(iii) before external synchronization and exchange. That is, multiple instances of (i)-(iii) (maintaining their order), i.e., multiple internal BSP supersteps, can be performed before (iv)-(v), i.e., external synchronization and exchange. Note also that any tile 4 can execute its own instance of internal synchronization and exchange (ii)-(iii), respectively, in parallel with other tiles 4.

[0061] Thus, for every overall BSP cycle (i)-(v), there is at least one portion in cycle (ii)-(iii) where synchronization is constrained to be performed internally only.

[0062] Note that within the external exchange 50, communication is not limited to only external: some tiles 4 may perform only internal exchanges, some may perform only external exchanges, and some may perform a mix of both.

[0063] 4, some tiles 4 may also perform local input / output during the computation phase, for example, they may exchange data with a host or other type of external storage.

[0064] Note also that it is generally possible for any or all tiles to have a null computation phase 52 or a null exchange phase 50 in any given BSP superstep, as shown in FIG.

[0065] For both internal and external barrier synchronization, tiles 4 participating in the barrier synchronization are referred to as a synchronization group. A configurable set of synchronization groups is supported by a processing unit 2, and each of these configurable synchronization groups is referred to herein as a synchronization zone. Each tile 4 can join a specific synchronization zone, thereby synchronizing any group of tiles together. Each synchronization zone can be individually configured to include a different synchronization group of tiles 4 depending on the settings of each synchronization zone. By changing these settings, tiles 4 can be associated or disassociated from a synchronization zone. A synchronization zone supported by a particular processing unit 2 can be configured as internal, in which case tiles 4 of that processing unit 2 that join the zone synchronize only with each other. On the other hand, a synchronization zone supported by a particular processing unit 2 can be configured as external, in which case the zone spans multiple processing units 2, and tiles 4 of one processing unit 2 participating in the zone synchronize with tiles 4 of another processing unit 2 participating in the zone.

[0066] For each tile 4 of a processing unit 2 that belongs to a synchronization zone, that tile 4 issues a synchronization request when it reaches the barrier synchronization. The synchronization requests issued by the tiles 4 are aggregated to form an internal synchronization request, which is provided to the synchronization logic of the processing unit 2. In an embodiment, the synchronization logic includes an internal synchronization controller 55 and an external synchronization controller 58, which are described in more detail below. In response to receiving an internal synchronization request, the synchronization logic may propagate an external synchronization request to a further entity in the synchronization zone before acknowledging the request. The further entity may be a proxy for exchanging data with synchronization logic associated with a host system or another processing unit 2.

[0067] When an external synchronization request propagates to synchronization logic associated with another processing unit 2, the action taken by the synchronization logic associated with that other processing unit 2 in response to the external synchronization request depends on whether that logic is defined as a master of the synchronization zone or a propagation node of the synchronization zone. The propagation node of the synchronization zone propagates the received external synchronization request toward the master defined in the synchronization zone. When the synchronization master receives an external synchronization request for each of the processing units 2 containing tiles 4 belonging to its synchronization zone, it returns an external synchronization acknowledgment to the synchronization logic associated with each of those other processing units 2 (other than its own processing unit 2) containing tiles 4 belonging to its synchronization zone. The synchronization master also causes each of the tiles 4 within its own processing unit 2 to return a synchronization acknowledgment. When the synchronization logic (including the propagation node) associated with each processing unit 2 receives the external synchronization acknowledgment originating from the synchronization master, it returns a synchronization acknowledgment to those tiles 4 of that processing unit 2. In response to receiving the synchronization acknowledgment, the tiles 4 of the synchronization zone pass through the barrier synchronization and exchange data with each other during the exchange phase. This exchange of data between the different processing units 2 is performed in a non-time-determined manner as described in our earlier patent application, US Patent Application Publication No. 15 / 886,065.

[0068] In this description, the term synchronization network refers to the connected synchronization propagation nodes / circuits of a synchronization zone that are used to exchange synchronization requests / acknowledgements to coordinate barrier synchronization between tiles 4 that belong to the synchronization zone. A synchronization request sent towards a master node defined in the synchronization network is said to be sent "upstream" in the synchronization network. A synchronization acknowledgment sent towards a slave node defined in the synchronization network is said to be sent "downstream" in the synchronization network. The concept of a synchronization network is further detailed with respect to FIG. 19.

[0069] Referring to FIG. 5, FIG. 5 illustrates an example integrated circuit 500a (i.e., chip 500a). Multiple similar such chips may be connected together. For the remainder of this description, the connected chips, of which example chip 500a is one, will be referred to as chips 500. Each chip 500 includes a processing unit 2 that includes tiles 4. Because the processing unit 2 of each chip 500 functions as an accelerator for processing workloads provided by a host system, each chip 500 may also be referred to as a processing device 500 or an accelerator subsystem 500. Although processing device 500 will be described as a chip 500 throughout this description, this does not exclude that, in some embodiments, such processing devices 500 may be implemented on the same integrated circuit.

[0070] 5 will be referred to as chip 500a in order to distinguish it from the multiple chips 500. Similarly, the specific processing unit shown in FIG. 5 will be referred to as processing unit 2a in order to distinguish it from the multiple processing units 2. The features of chip 500a and processing unit 2a described below are also the features of chip 500 and processing unit 2, respectively.

[0071] Each of the tiles 4 in the processing unit 2a may participate in different types of barrier synchronization: The first type of barrier synchronization is internal synchronization, in which only tiles 4 in the same processing unit 2a participate.

[0072] A second type of synchronization is external wired synchronization, where the synchronization zone of the synchronization, in addition to including the tile 4 of the processing unit 2a, also includes tiles 4 in one or more chips 500 accessible via the local wired connection. In external wired synchronization, synchronization messages are exchanged between chips 500 via dedicated wires used to transmit different types of synchronization messages. Application data exchanged between chips 500 during the exchange phase in external wired synchronization is sent via PCIe connections between the chips 500 participating in the synchronization.

[0073] The third type of synchronization is external synchronization involving the host, in which a host synchronization proxy (HSP) participates in the barrier synchronization by exchanging synchronization messages with processing unit 2a before the exchange phase in which data is exchanged between the host and tile 4 of processing unit 2a.

[0074] A fourth type of synchronization is external packet-based synchronization, in which the synchronization group, in addition to including tiles 4 of processing unit 2a, also includes tiles 4 in one or more chips 500 accessed via a packet-switched network (e.g., an Ethernet network). For external packet-based synchronization, in addition to sending application data between chips 500 via the packet-switched network, synchronization messages are also sent via the same packet-switched network.

[0075] To allow each tile 4 in the processing unit 2a to participate in different types of synchronization, multiple synchronization zones are provided in the processing unit 2a. In an embodiment, 30 different synchronization zones are provided in the processing unit 2a, each of which is configurable to include one or more of the tiles 4 of the processing unit 2a. Each synchronization zone is individually configurable to include a different synchronization group of tiles 4.

[0076] Each synchronization zone can be configured as an external synchronization zone of external barrier synchronization (in which case the corresponding synchronization group includes tiles 4 of other processing units 2) or as an internal synchronization zone of internal barrier synchronization (in which case the synchronization group of that synchronization zone is limited to tiles 4 of processing unit 2a).

[0077] The synchronization zones may be classified into different sets depending on the hardware provided for that synchronization zone and therefore the type of synchronization that may be implemented using that synchronization zone. The first set of synchronization zones are synchronization zones that may be configured to be used for either the first type of synchronization described above (i.e., internal synchronization) or the second type of synchronization described above (i.e., external wired synchronization). In an embodiment in which 30 synchronization zones are defined in processing unit 2a, the first 22 of these zones (denoted synchronization zones 1 to 22) belong to the first set of synchronization zones.

[0078] Of the first set of synchronization zones, a subset of these synchronization zones may also be used for communication with a host, i.e., for the third type of synchronization described above. In an embodiment in which the first set of synchronization zones includes 22 synchronization zones, two of these synchronization zones (denoted synchronization zones 1 and 2) may be used for barrier synchronization, following which data exchange is performed between the host and tile 4 of processing unit 2a.

[0079] The second set of synchronization zones are synchronization zones that can be used for the first type of synchronization described above (i.e., internal synchronization) or the fourth type of synchronization described above (i.e., external packet-based synchronization). In an embodiment in which 30 synchronization zones are defined in processing unit 2a, the last eight of these zones (denoted synchronization zones 23-30) belong to the second set of synchronization zones.

[0080] As mentioned above, to coordinate synchronization between different tiles 4 that are part of a synchronization zone, an exchange of synchronization requests and synchronization acknowledgments is performed between tiles 4. The path through the synchronization network along which synchronization requests are propagated and acknowledged will depend on the configuration settings of the synchronization zone in use to coordinate synchronization between tiles 4 that belong to a synchronization group.

[0081] In the following description, several types of synchronization request and acknowledgement signals are used in different parts of the synchronization network. To distinguish between these signals, the following terminology is adopted:

[0082] Each tile 4 has a synchronization request wire for each synchronization zone. The state of this wire is referred to herein as a tile synchronization request state. If the state of the wire is set to indicate that a synchronization request is asserted by the tile 4, the generated synchronization request is referred to as a tile synchronization request. Each tile 4 includes an execution unit 52, which may control the state of the synchronization request wire. For any such wire, the signal output by the execution unit 52 and used to assert the tile synchronization request on that wire is referred to as a synchronization control signal.

[0083] Each tile 4 also has a synchronization acknowledge wire for each synchronization zone. The state of this wire is referred to herein as the internal synchronization acknowledge state. If the state of the wire indicates that a synchronization acknowledge is asserted by the tile 4, the synchronization acknowledge generated is referred to as an internal synchronization acknowledge. The execution unit 52 responds with a pulse generated in response to an edge in the internal synchronization acknowledge state. Such a pulse is referred to herein as a synchronization acknowledge pulse.

[0084] An aggregation circuit is provided in processing unit 2a to aggregate the synchronization request states of all tiles 4 in processing unit 2a. The state of the signal output by each such unit of the aggregation circuit is referred to herein as an aggregate synchronization request state, and a synchronization request signaled by an aggregate synchronization request state is referred to as an aggregate synchronization request. The aggregate synchronization request states of all tiles 4 in processing unit 2a are referred to as internal aggregate synchronization request states, and a synchronization request signaled by such states is referred to as an internal synchronization request. Such internal synchronization requests are provided as inputs to internal synchronization controller 55, which in response outputs a corresponding internal synchronization acknowledgment. This internal synchronization acknowledgment propagates to all tiles 4 in processing unit 2a.

[0085] The internal synchronization controller 55 of the configured specific synchronization zone outputs a synchronization request to the external synchronization controller (GSP 58) in response to the internal synchronization request. This synchronization request is called an external synchronization request. In response, the GSP 58 returns a synchronization acknowledgement to the internal synchronization controller 55. This returned acknowledgement is called an external synchronization acknowledgement.

[0086] FIG. 5 shows that processing unit 2a includes a synchronization controller circuit 55 (shown as IPU synchronization controller 55) between tile 4 and GSP 58. IPU synchronization controller 55 is sometimes referred to as internal synchronization controller 55 because it acknowledges internal synchronization requests for internal barrier synchronization without requiring input from GSP 58. IPU synchronization controller 55 receives internal synchronization requests represented by aggregate synchronization request states output by tile 4 and performs operations according to settings in register 501 of GSP 58. The settings in register 501 define, for each synchronization zone, whether the synchronization zone is defined as internal or external. An indication of the setting in register 501 is provided to IPU synchronization controller 55 via interface 502 between GSP 58 and IPU synchronization controller 55. Any of the 30 synchronization zones can be defined as either external or internal.

[0087] When an internal synchronization request is received by the IPU synchronization controller 55 and the synchronization zone of the synchronization request is defined as an external synchronization zone in register 501, the IPU synchronization controller 55 responds by providing an external synchronization request to the GSP 58 on the interface of the GSP 58 associated with the synchronization zone specified for the synchronization request. As shown in FIG. 5, the GSP 58 has several interfaces (labeled IS0-IS29), each associated with one of the synchronization zones provided to the processing unit 2a. The synchronization controller 55 provides the external synchronization request via one of the interfaces (IS0-IS29) associated with the same synchronization zone as the internal synchronization request. In response, the GSP 58 returns an external synchronization acknowledgement, which is sent via one of the interfaces IS0-IS29 via the same interface via which the external synchronization request was provided. In response to the external synchronization acknowledgement returned by the GSP 58, the synchronization controller 55 outputs an internal synchronization acknowledgement to each tile 4 in the processing unit 2a.

[0088] When an internal synchronization request associated with a particular synchronization zone is received by IPU synchronization controller 55, if that synchronization zone is defined as an internal synchronization zone in register 501, IPU synchronization controller 55 causes an internal synchronization acknowledgement to be sent to tile 4 of processing unit 2a. IPU synchronization controller 55 performs this operation without waiting for an external synchronization acknowledgement from GSP 55. However, IPU synchronization controller 55 may also pass an external synchronization request signal to GSP 58, which will then assert the signal at the interface of GSP 58 associated with the synchronization zone, allowing GSP 58 to record trace data for that synchronization.

[0089] To send and receive internal synchronization requests and internal synchronization acknowledgments, IPU synchronization controller 55 includes multiple wire sets, each wire set associated with a different synchronization zone, including at least a synchronization request wire on which the internal synchronization requests for each synchronization zone are received and a synchronization acknowledgment wire on which the internal synchronization acknowledgments for each synchronization zone are sent to tile 4.

[0090] To send and receive external synchronization requests and external synchronization acknowledgments, IPU synchronization controller 55 includes multiple wire sets, each wire set associated with a different synchronization zone. Each wire set is also associated with a different one of GSP 58 interfaces IS0-IS29 and is used to pass external synchronization requests to GSP 58 and receive external synchronization acknowledgments from GSP 58 for its respective synchronization zone.

[0091] To ensure that each tile 4 indicates which synchronization zone it should participate in, each tile 4 also has multiple dedicated synchronization request wires, each associated with one of the synchronization zones defined in the processing unit 2a. When each tile 4 is to participate in barrier synchronization associated with a particular synchronization zone, it issues a tile synchronization request on the synchronization request wire associated with that synchronization zone. Each tile 4 also has multiple dedicated synchronization acknowledge wires, each associated with one of the synchronization zones defined in the processing unit 2a. After each tile 4 issues a high-gain synchronization request on the synchronization request wire of a synchronization zone, it receives an internal synchronization acknowledge from the synchronization controller 55 on the synchronization acknowledge wire associated with that synchronization zone. In response, the tile 4 proceeds to the exchange phase following barrier synchronization.

[0092] Referring to FIG. 6, FIG. 6 illustrates an example of a tile 4 according to an embodiment of an example application. The tile 4 includes a memory 51 that stores both instructions to be executed by the execution units 52 and data that configures the execution units 52 to perform operations when executing the instructions. The memory 51 includes a local program for each tile 4, which includes instructions to be executed by the execution units 52. The instruction (or code) set for each tile 4 includes an indication of a synchronization point at which the tile 4 should participate in barrier synchronization. The indication includes a synchronization instruction, which is to be executed by each execution unit 52 when the tile 4 reaches a synchronization point in its local program. Each such synchronization instruction, described below, is executed when the execution units 52 of each tile 4 reach a given synchronization point, causing the synchronization controller to issue a synchronization request in connection with the barrier synchronization to be performed at that synchronization point.

[0093] Tile 4 includes a sync zone register 53 that stores, for each of a plurality of sync zones defined in processing unit 2, an indication as to whether tile 4 belongs to the respective sync zone. Sync zone register 53 includes a bitmap, with each bit indicating whether tile 4 belongs to a different one of the plurality of sync zones.

[0094] To change the synchronization zone to which it belongs, the execution unit 52 of a tile 4 may execute an instruction that changes the indication held in the synchronization zone register 53. In some embodiments, the synchronization zone to which each tile 4 of a processing unit 2a belongs is fixed for a particular application. In other embodiments, during execution of an application, the execution unit 52 of one or more tiles 4 of a processing unit 2a executes an instruction that changes the synchronization zone indication held in the register 53 to change the synchronization zone to which it belongs.

[0095] Tile 4 includes a data output interface 54, which is used to send data to other tiles 4 belonging to the same processing unit 2a during the internal switching phase, and to send data to destinations external to device 500a during the external switching phase. Tile 4 includes a data input interface 59, which is used to receive data from other tiles 4 belonging to the same processing unit 2a during the internal switching phase, and to receive data from sources external to device 500a during the external switching phase.

[0096] Tile 4 includes multiple synchronization output interfaces 60, which are used to output tile synchronization request states from tile 4 toward internal synchronization controller 55. Tile 4 also includes multiple synchronization input interfaces 61 for receiving internal synchronization acknowledgments from synchronization controller 55 and notifying execution unit 52. Each synchronization output interface 60 is associated with a different synchronization zone and is used to send the tile synchronization request for the associated synchronization zone on a corresponding synchronization request wire. Each synchronization input interface 61 is associated with a different synchronization zone and is used to receive the internal synchronization acknowledgment for the associated synchronization zone on a corresponding synchronization acknowledgment wire. For simplicity, tile 4 is shown including only two synchronization output interfaces 60 and two synchronization input interfaces 61, but in practice tile 4 would include three or more (e.g., 30) of each type of interface 60, 61.

[0097] A tile 4 is configured to output a tile synchronization request to a synchronization zone by setting the state of an associated synchronization request wire to the inverse of the state of the corresponding synchronization acknowledge wire for that synchronization zone. For example, if a particular synchronization zone's internal synchronization acknowledge signal is set low, then the signal on the corresponding synchronization request wire is set high to assert a tile synchronization request. Conversely, if a particular synchronization zone's internal synchronization acknowledge signal is set low, then the signal on the corresponding synchronization request wire is set high to assert a tile synchronization request.

[0098] Referring to Figure 7, Figure 7 shows a timeline in which multiple tile synchronization requests are issued by tile 4. The synchronization request and acknowledgement signals shown are for the particular synchronization zone, marked "z," in which tile 4 is configured to participate. Thus, the synchronization request wire, output interface 60, synchronization acknowledgement wire, and input interface 61 considered are for synchronization zone z.

[0099] Initially, the state of the synchronization acknowledge wire for tile 4 is held low. Execution unit 52 for tile 4 executes a synchronization instruction, causing a tile synchronization request (i.e., req1) to be asserted on the synchronization request wire. Circuitry in output interface 60 causes the tile synchronization request (i.e., req1) to be issued by setting the state of the synchronization request wire to the inverse of the state of the synchronization acknowledge wire. Because the acknowledge wire is held in a low state, a tile synchronization request is issued by setting the state of the synchronization request wire to high. A tile synchronization request is represented by a transition of the synchronization request signal from low to high.

[0100] At some point after the assertion of the tile synchronization request, an internal synchronization acknowledge (i.e., ack1) is received at the input interface 61 of tile 4. The internal synchronization acknowledge is detected when the synchronization acknowledge wire changes state, i.e., when an edge in the received synchronization acknowledge is detected. Following the issuance of the tile synchronization request "req1," the synchronization request wire of tile 4 is held in a high state. Once the synchronization acknowledge wire is also set to a high state, the internal synchronization acknowledge "ack1" is received at tile 4. When a transition occurs reflecting the receipt of "ack1," the synchronization request wire and the synchronization acknowledge wire are both held in a high state. Because both wires are held in the same state, the tile synchronization request (i.e., req1) is no longer asserted. The transition point at which ack1 is received also reflects the point at which req1 is deasserted.

[0101] After ack1 is received, execution unit 52 may move to the exchange phase, in which it executes one or more send instructions to cause data to be sent from memory 51 via data output interface 54 .

[0102] To participate in further barrier synchronization, execution unit 52 executes a further synchronization instruction to issue a further tile synchronization request (i.e., req2). In this case, the corresponding synchronization acknowledge wire is set to a high state, so the further tile synchronization request (i.e., req2) is issued by setting the synchronization request wire to a low state. The tile synchronization request remains asserted until the corresponding acknowledge (i.e., ack2) is received, which is when the synchronization acknowledge wire is set low.

[0103] Following receipt of ack2, execution unit 52 proceeds to execute a further synchronization instruction that causes the next tile synchronization request (i.e., req3) issued by setting the synchronization request wire to a high state. This synchronization request for tile 4 remains asserted until the synchronization acknowledge wire is also set to a high state, indicating receipt of ack3.

[0104] As shown in FIG. 6 , in addition to the execution units 52, the value of the synchronization zone register 53 can also be used to control the state of the synchronization request wire. Controlling the state of the synchronization request wire according to the value in the synchronization zone register 53 allows barrier synchronization for a particular synchronization zone to proceed even if tile 4 does not belong to and is not configured to participate in that particular synchronization zone. Consider that, for a particular synchronization zone, the synchronization zone register 53 includes an indication that tile 4 does not belong to that synchronization zone. In that case, the synchronization output interface 60 for that particular synchronization zone is configured to output a tile synchronization request in response to the indication in register 53 that tile 4 does not belong to that synchronization zone. As long as the indication in register 53 is such that tile 4 does not belong to that synchronization zone, the synchronization output interface 60 continues to output tile synchronization requests regardless of the activity of the execution units 52. During the computation phase, the execution units 52 may continue to execute additional instructions to perform computations or execute synchronization instructions to participate in barrier synchronization for other synchronization zones. In this way, tile 4 may operate asynchronously with the computation-exchange cycles of other tiles 4 that belong to the synchronization zones in which tile 4 of FIG. 6 does not participate.

[0105] 8A, which illustrates an example synchronization output interface 60. Synchronization output interface 60 includes an inverter 82 configured to invert the internal synchronization acknowledge signal to provide a signal that tile 4 may output to provide a tile synchronization request when it is not participating in the synchronization zone associated with interface 60. Synchronization output interface 60 includes a multiplexer 81 controlled in response to an instruction in register 53 to select either to output an inverted version of the internal synchronization acknowledge provided by inverter 82 or to output a synchronization signal controlled by execution of a synchronization instruction by execution unit 52.

[0106] If tile 4 does not participate in the synchronization zone associated with interface 60 shown in FIG. 8A, a control signal reflecting the indication in register 53 that tile 4 does not participate is received by multiplexer 81, which controls multiplexer 81 to select the inverted form of the internal synchronization acknowledge. Interface 60 outputs this inverted form of the internal synchronization acknowledge on the synchronization request wire. Because the inverted form of the internal synchronization acknowledge reflects an asserted tile synchronization request, in this way, if tile 4 does not participate in that synchronization zone, the tile synchronization request for that synchronization zone continues to be asserted regardless of activity of execution unit 52.

[0107] When tile 4 joins the synchronization zone associated with interface 60 shown in FIG. 8A , a control signal reflecting the indication in register 53 that tile 4 is joining is received by multiplexer 81, which controls multiplexer 81 so that the state of the synchronization request wire is controlled by execution unit 52. This execution unit 52 provides a synchronization signal, setting it high to assert a tile synchronization request or low to deassert the synchronization request. An XOR gate 83 is used to provide the tile synchronization request according to a signaling scheme for signaling the synchronization request to synchronization controller 55. XOR gate 83 receives the internal synchronization acknowledge signal and outputs this synchronization acknowledge if execution unit 52 is not asserting the synchronization control signal, or outputs an inverted version of the internal synchronization acknowledge if the execution unit is asserting the synchronization control signal. In this manner, when multiplexer 81 is controlled to select the output from XOR gate 83, multiplexer 81, and therefore output interface 60, outputs a tile synchronization request if controlled by execution unit 52 to do so. To issue a tile synchronization request, execution unit 52 executes a synchronization instruction, which causes the state of the synchronization request wire output from interface 60 to be set to the inverse of the current state of the synchronization acknowledge wire.

[0108] 8B, an example synchronization input interface 61 receives an internal synchronization acknowledge signal and outputs a synchronization acknowledge pulse to execution unit 52 depending on the state of this signal. The example synchronization input interface 61 may be associated with any synchronization zone, and each synchronization input interface 61 for a different synchronization zone includes the same circuitry. The example synchronization input interface 61 may be associated with the same synchronization zone as the example synchronization output interface 60 shown in FIG. 8A.

[0109] Interface 61 includes flip-flop 85, which stores the state of the internal synchronization acknowledge signal received by interface 61. Flip-flop 85 outputs this latched state. Interface 61 also includes XOR gate 86, which receives the state of the synchronization acknowledge wire as one input and the output of flip-flop 85 as another input. XOR gate 86 outputs a high signal if these two inputs are different. If the state of the synchronization acknowledge wire of interface 61 changes, the state of this synchronization acknowledge wire will temporarily not match the output of flip-flop 85. As a result, XOR gate 86 receives one high input and one low input, and therefore outputs a high signal. One clock cycle later, if the state of flip-flop 85 is updated to match the updated state of the synchronization acknowledge wire, the output of flip-flop 85 will match the state of the synchronization acknowledge wire. As a result, XOR gate 86 outputs a low signal. In this manner, interface 61 provides a pulse (synchronization acknowledge pulse) in response to an edge of the received synchronization acknowledge signal. The synchronization acknowledge pulse is output from interface 61 to execution unit 52. If execution unit 52 has executed a synchronization instruction for the synchronization zone corresponding to the synchronization acknowledge pulse, it stalls while waiting for this synchronization acknowledge pulse. In response to receiving the synchronization acknowledge pulse, execution unit 52 passes through the barrier and proceeds to the exchange phase, in which data is exchanged between that tile 4 and other tiles 4. If execution unit 52 is not executing such a synchronization instruction and is part of a tile 4 indicated in synchronization zone register 53 as not participating in the synchronization zone associated with the synchronization acknowledge pulse, execution unit 52 ignores the synchronization acknowledge pulse.

[0110] Also shown in Figure 8B, the internal synchronization acknowledge signal received at interface 61 is also provided to a corresponding synchronization output interface 60 associated with synchronization input interface 61. As shown in Figure 8A, this internal synchronization acknowledge signal is provided as an input to both XOR gate 83 and inverter 82, and is thus used to provide a tile synchronization request signal.

[0111] Each tile 4 in the processing unit 2a operates similarly to assert a tile synchronization request in response to the state of the synchronization acknowledge wire. A summation circuit is provided in the processing unit 2a to aggregate the tile synchronization requests output by the tiles 4 to provide an internal synchronization request that is provided to the synchronization controller 55. The summation circuit performs such aggregation for each synchronization zone to provide an aggregate synchronization request state for each synchronization zone. The summation circuit is configured to aggregate the states of the tile synchronization request outputs such that the summation signal changes state in response to each tile 4 changing state of the tile synchronization request state. For example, if the state of the synchronization request wire for each tile 4 in a particular synchronization zone in the processing unit 2 is set to a low state, the summation signal also goes low. The summation circuit causes the summation signal to change state to a high state in response to all of the tile synchronization request wires for that synchronization zone being set to a high state.

[0112] Referring to Figure 9, Figure 9 shows an example of how tile synchronization requests are aggregated across processing units 2a. As shown, tiles 4 are arranged in pairs (referred to as "paired tiles"), and the pairs are arranged in columns. The aggregation circuit includes a set of circuits 910 and 920. Each paired tile is associated with a set of circuits 910 configured to aggregate synchronization requests from the associated tile 4. Each set of circuits 910 receives synchronization request status from upstream in the synchronization network and aggregates this status with the synchronization request status output by the associated paired tile 4. Between columns, circuits 920 are configured to receive aggregated synchronization request status from different columns.

[0113] As explained, the summation circuits 910, 920 are configured to perform the summation of the synchronization request state depending on the state of the internal synchronization acknowledge signal (output by the synchronization controller 55) for that synchronization zone.

[0114] 10, which illustrates how the internal synchronization acknowledgements may be distributed to different tiles 4 and summing circuits 910, 920 of a processing unit 2a. As will be explained in more detail, the synchronization controller 55 changes the state of the internal synchronization acknowledgement signal in response to receiving an internal synchronization request.

[0115] As shown, the internal synchronization acknowledge signal is provided to all of tile 4, summing circuit 910, and summing circuit 920. The internal synchronization acknowledge signal output by synchronization controller 55 is provided in the same state on all synchronization acknowledge wires used to distribute signals to tile 4 and circuits 910, 920.

[0116] 11, which illustrates an example summation circuit 910 according to an example embodiment. The summation circuit 910 includes an OR gate 1100 and an AND gate 1110. Each gate 1100, 1110 receives a synchronization request state from each of two pairs of wires 4 associated with the circuit 910. Each gate 1100, 1110 also receives an upstream synchronization request signal that reflects the aggregated synchronization request state of tiles 4 further up in the associated column.

[0117] Circuit 910 includes a multiplexer 1120, which is controlled in response to the internal synchronization acknowledge signal to select either the output of OR gate 1100 or the output of AND gate 1110. When the internal synchronization acknowledge signal is high, OR gate 1100 is selected, while when the internal synchronization acknowledge signal is low, AND gate 1110 is selected. As a result of this selection, circuit 910 only outputs a signal that is the opposite of the internal synchronization acknowledge signal if all tile synchronization requests received by circuit 910 (i.e., signals from both tile 4 and upstream synchronization request signals) have the opposite state of the internal synchronization acknowledge signal. For example, when the internal synchronization acknowledge signal is high, OR gate 1100 is selected, and therefore the synchronization request state output by circuit 910 is also high until all inputs to OR gate 1100 are low. On the other hand, when the internal synchronization acknowledge signal is low, AND gate 1110 is selected, and therefore the tile synchronization request state output by circuit 910 is also low until all inputs to AND gate 1110 are high.

[0118] Multiple instances of circuit 910 are chained together to provide aggregate synchronization request state for a column.

[0119] Referring to FIG. 12, FIG. 12 illustrates how aggregate synchronization request states may be provided to a column 1200 of tiles 4. Each set of circuits 910b-e is configured to receive the tile synchronization request state output by its associated tile 4 and further receive an aggregate synchronization request state (also referred to as an upstream synchronization request state) provided by a neighboring set of circuits 910, as shown in FIG. 11. For example, circuit 910b receives the upstream synchronization request state output by circuit 910a, circuit 910c receives the upstream synchronization request state output by circuit 910b, and so on. Because circuit 910a is located at the top of column 1200 and no more aggregation circuits 910 are located above it, circuit 910a receives the output of inverter 1210 as the upstream synchronization request state. Inverter 1210 inverts the synchronization acknowledge signal and, in doing so, provides an asserted synchronization request signal as an output. Each set of circuits 910a-e receives an asserted tile synchronization request from its associated tile 4, and will output an asserted synchronization request if it receives an asserted synchronization request from higher up in column 1200. In this way, if all tiles 4 also provide asserted synchronization request signals, the aggregate output by circuit 910e will be the same asserted synchronization request signal.

[0120] Referring to FIG. 13, FIG. 13 illustrates an example summation circuit 920 and how the circuit 920 sums the synchronization request states of different columns. Similar to circuit 910, circuit 920 includes an OR gate 1300 and an AND gate 1310. Both AND gate 1310 and OR gate 1300 receive as inputs the aggregate synchronization request states from two of the columns, as well as a further aggregate synchronization request state. The further aggregate synchronization request state is shown in FIG. 13 as a "swap aggregate synchronization request state" because the states are summed in the data exchange wiring direction that runs perpendicular to the columns. The swap aggregate synchronization request state may be output by a further instance of circuit 920 upstream in the synchronization network, or may be provided by an inverted version of the internal synchronization acknowledge signal if there is no further upstream instance of circuit 920.

[0121] Circuit 920 includes a multiplexer 1320, which is controlled in response to the internal synchronization acknowledge signal to select between the output of OR gate 1300 and the output of AND gate 1310. When the internal synchronization acknowledge signal is high, OR gate 1300 is selected, while when the internal synchronization acknowledge signal is low, AND gate 1310 is selected. As a result of this selection, circuit 920 only outputs a signal that is the inverse of the internal synchronization acknowledge signal if all of the aggregate synchronization request states received by circuit 920 (i.e., the aggregate synchronization request states from both associated columns and the exchange aggregate synchronization request states) have the inverse state of the internal synchronization acknowledge signal. For example, when the internal synchronization acknowledge signal is high, OR gate 1300 is selected, and therefore the aggregate synchronization request state output by circuit 920 is also high unless all inputs to OR gate 1300 are low. On the other hand, if the internal synchronization acknowledge signal is low, then AND gate 1310 is selected and therefore the aggregate synchronization request state output by circuit 920 is also low unless all inputs to AND gate 1310 are high.

[0122] Multiple instances of circuit 920 are chained together to provide an aggregate synchronization request state to processing unit 2 a. The aggregate synchronization request state provided as a result of the aggregation performed by instances of circuit 910 and instances of circuit 920 represents an internal synchronization request if all of the tiles 4 have set their synchronization request output signals appropriately.

[0123] Referring to FIG. 14A, FIG. 14A illustrates circuitry within synchronization controller 55. FIG. 14A illustrates circuitry provided in synchronization controller 55 associated with a single synchronization zone. However, it will be appreciated that synchronization controller 55 includes a separate instance of such circuitry for each synchronization zone. Circuitry within synchronization controller 55 provides internal synchronization acknowledgements in response to receiving internal synchronization requests. Circuitry in synchronization controller 55 also communicates with GSP 58 to send and receive external synchronization request and acknowledgement signals. External synchronization request and acknowledgement signals are provided in accordance with an alternative synchronization signaling scheme that will be described in more detail with reference to FIG. 14B.

[0124] A register 501 is provided in GSP58 to indicate, for each synchronization zone supported by processing unit 2a, which of these zones are configured as internal (including only tiles 4 of processing unit 2a) and which are configured as external (including tiles 4 of other processing units 2).

[0125] When the synchronization zone shown in FIG. 14A is configured as internal, a signal indicating such (shown as the "internal synchronization enable" signal) is provided by the GSP 58 circuitry to OR gate 1440. OR gate 1440 therefore outputs a high signal to multiplexer 1450. The signal is used to control multiplexer 1450 to output the internal synchronization request state (i.e., the aggregate synchronization request state received by controller 55). Thus, the internal synchronization acknowledge state is set to be the same state as the internal synchronization request state. In this way, when the synchronization zone is configured to be internal, synchronization controller 55 immediately acknowledges the internal synchronization request by setting the internal synchronization acknowledge state to be the same as the internal synchronization request state.

[0126] 14A is configured as external, the internal synchronization enable signal will be set low, and therefore the output of OR gate 1440 will be set low until GSP 58 provides an external synchronization acknowledge signal. Before an external synchronization acknowledge is received from GSP 58, synchronization controller 55 provides an external synchronization request to GSP 58.

[0127] External synchronization requests and synchronization acknowledgments are expressed according to a different synchronization scheme than the scheme used for tile synchronization requests and acknowledgments and internal synchronization requests and acknowledgments (illustrated in FIG. 7). Referring to FIG. 14B, FIG. 14B illustrates a scheme for signaling external synchronization requests and acknowledgments. This figure illustrates an example of a synchronization handshake between a downstream propagation node and an upstream propagation node in a synchronization network. The downstream propagation node may be, for example, a GSP 58 on one chip 500, while the upstream propagation node is a GSP 58 on another chip 500.

[0128] FIG. 14B shows the state of the external synchronization request signal and the state of the external synchronization acknowledge signal. These are each provided on separate wires, so the state of the signals reflects the state of the wires. At the start of a synchronization handshake, the downstream propagation node provides an external synchronization request (shown as 1. Synchronization Request Asserted) by setting the external synchronization request signal high. This causes the external synchronization request to be detected by the upstream propagation node. The downstream propagation node will keep the external synchronization request asserted until it receives the external synchronization acknowledge. In effect, the downstream propagation node stalls until it receives the external synchronization acknowledge. In response to the external synchronization request, the upstream propagation node provides an external synchronization acknowledge (shown as 2. Synchronization Acknowledge Asserted) by setting the external synchronization acknowledge signal high. This causes the external synchronization acknowledge to be detected by the downstream propagation node. In response to the external synchronization acknowledge, the downstream propagation node deasserts the external synchronization request (shown as 3. Synchronization Request Deasserted) by setting the external synchronization request signal low. The upstream propagating node detects that the synchronization request signal has been deasserted and, in response to the deassertion of the synchronization request signal, deasserts the external synchronization acknowledge (shown as 4. Deasserted Synchronization Acknowledge) by setting the state of the external synchronization acknowledge signal to low. With the external synchronization acknowledge signal deasserted, the synchronization handshake between the two nodes of the synchronization network is complete.

[0129] The synchronization controller 55 includes circuitry for converting internal synchronization requests to external synchronization requests and external synchronization acknowledgements to internal synchronization acknowledgements. The circuitry includes an XOR gate 1410, which is part of a circuit for generating external synchronization requests from internal synchronization requests. The XOR gate 1410 receives the aggregate synchronization request state of the processing unit 2a as one input and the current state of the internal synchronization acknowledge signal as another input. The XOR gate 1410 outputs a high signal (indicating a synchronization request) if there is a mismatch between the aggregate synchronization request state and the synchronization acknowledge state. Such a mismatch indicates that a tile synchronization request has been asserted by all tiles 4 of the processing unit 2a. When a high signal is output from the XOR gate 1410, it is provided to an OR gate 1420, which in response also issues a high signal, which is provided to an AND gate 1430. The AND gate 1430 receives the output of the OR gate 1420 as one input and the inverted state of the external synchronization acknowledge signal as another input. Thus, AND gate 1430 only outputs a high signal if the external synchronization acknowledge signal is currently low (indicating that the external synchronization acknowledge signal is not currently asserted). The output of AND gate 1430 provides the external synchronization request signal to GSP 58.

[0130] As described in more detail below in connection with FIG. 17 , GSP58 is configured to exchange external synchronization requests and acknowledgements with additional GSP58s within system 550. Following this process, GSP58 provides an external synchronization acknowledge signal, which is provided to inverter 1460 and OR gate 1440. Inverter 1460 inverts the external synchronization acknowledge signal (which is currently set high) to generate a low signal, which is provided to AND gate 1430. In response, AND gate 1430 outputs a low signal, deasserting the external synchronization request. OR gate 1440 provides a high output to multiplexer 1450. This signal is used to control multiplexer 1450 so that the internal synchronization request state is output from multiplexer 1450. Thus, in response to the external synchronization acknowledge signal, multiplexer 1450 is configured to set the internal synchronization acknowledge state to the internal synchronization request state, thereby causing the internal synchronization acknowledge to be sent to tile 4.

[0131] Referring to FIG. 15, FIG. 15 shows how different groups of tiles 4 can join three different synchronization zones. FIG. 15 shows the registers 53 of each of these three synchronization zones. As shown, the registers 53 of tiles 4a and 4b belong to the first synchronization zone (labeled "Z1") and the third synchronization zone (labeled "Z3"), but do not belong to the second synchronization zone (labeled "Z2"). Similarly, as shown, the registers 53 of tiles 4c and 4d belong to the second synchronization zone (labeled "Z2") and the third synchronization zone (labeled "Z3"), but do not belong to the first synchronization zone (labeled "Z1"). The synchronization summation circuit 1500 shown in FIG. 15 may include the summation circuits 910 and 920 described above.

[0132] In practice, more than two tiles 4 of a processing unit 2 are likely to participate in each synchronization zone, but in Figure 15, Z1 is shown as including a group 1510a of two tiles 4a, 4b, and Z2 is shown as including a group 1510b of two tiles 4c, 4d.

[0133] Because tiles 4c and 4d do not belong to Z1, these tiles 4c and 4d continuously assert tile synchronization requests on the synchronization request wires associated with Z1. These synchronization requests are shown as SRZ1 in Figure 15. As mentioned above, in an embodiment, the assertion of a tile synchronization request is represented by setting the state of the associated synchronization request wire to the state of the associated synchronization acknowledge wire.

[0134] Because tiles 4a and 4b belong to Z1, these tiles 4a, 4b assert a tile synchronization request on the synchronization request wire of Z1 only when they reach the barrier synchronization associated with Z1 in their compiled code set. At this point, the execution unit 52 of each tile 4a, 4b executes the synchronization instruction, taking the instruction of Z1 as the operator, thereby asserting a tile synchronization request on the synchronization request wire of the tile 4 associated with Z1.

[0135] When the tiles 4a and 4b belonging to Z1 reach barrier synchronization, all tiles 4 (including those not belonging to Z1) in the processing unit 2a assert a tile synchronization request for Z1. The synchronization aggregation circuit 1500 aggregates the tile synchronization requests and provides an internal synchronization request for Z1 (denoted as AZ1) to the synchronization controller 55.

[0136] Referring to FIG. 16, FIG. 16 shows how the internal synchronization acknowledgment is returned to tile 4. The synchronization distribution wiring shown in FIG. 16 may include the wiring shown in FIG. 10 for providing the internal synchronization acknowledgment to tile 4. In response to receiving AZ1, synchronization controller 55 asserts an internal synchronization acknowledgment signal (shown as SAZ1) associated with Z1. SAZ1 is provided to each of tiles 4a-d by synchronization distribution wiring 1600. In response to receiving this internal synchronization acknowledgment signal, execution units 52 of tiles 4a, 4b pass barrier synchronization and proceed to the exchange phase. If the exchange phase is an internal exchange phase, one or more of execution units 52 of tiles 4a, 4b execute instructions to exchange data between tiles 4a and 4b. If the exchange phase is an external exchange phase, one or more of execution units 52 of tiles 4a, 4b execute instructions to exchange data with devices external to device 500a.

[0137] When SRZ1 is received by tiles 4c, 4d belonging to Z2, the execution units 52 of these tiles 4c, 4d take no action in response to SRZ1 and continue to operate asynchronously with respect to the computation and exchange cycle of Z1.

[0138] Referring again to Figure 15, consider the situation with Z2. Because tiles 4a and 4b do not belong to Z2, these tiles 4a, 4b continuously assert a tile synchronization request on the synchronization request wire associated with Z2. This synchronization request is shown as SRZ2 in Figure 15.

[0139] Because tiles 4c and 4d belong to Z2, these tiles 4c and 4d assert a tile synchronization request on the synchronization request wire of Z2 only when they reach the barrier synchronization associated with Z2 in their compiled code set. At this point, execution units 52 of tiles 4c and 4d each execute a synchronization instruction, taking the instruction of Z2 as the operator. Each such synchronization instruction causes logic in each tile 4c to assert an internal synchronization request on the synchronization request wire of Z2.

[0140] When tiles 4c and 4d belonging to Z2 reach barrier synchronization, all tiles 4 (including those not belonging to Z2) in processing unit 2a assert a tile synchronization request for Z2. The synchronization aggregation circuit 1500 provides the aggregate synchronization request for Z2 (denoted as AZ2) to the synchronization controller 55.

[0141] Referring again to FIG. 16, which shows how a synchronization acknowledgement to Z2 synchronization is sent back to tile 4. In response to receiving AZ2, synchronization controller 55 asserts an internal synchronization acknowledgement signal (denoted as SAZ2) associated with Z2. SAZ2 is provided to each tile 4a-d by synchronization distribution wiring 1600. In response to receiving this synchronization acknowledgement signal, execution units 52 in tiles 4c and 4d pass through barrier synchronization and proceed to the exchange phase. If the exchange phase is an internal exchange phase, one or more of execution units 52 in tiles 4c and 4d execute instructions to exchange data between tiles 4c and 4d. If the exchange phase is an external exchange phase, one or more of execution units 52 in tiles 4c and 4d execute instructions to exchange data with devices external to device 500a.

[0142] Referring again to FIG. 15, consider a situation with Z3. As shown, in each of tiles 4a-d of processing unit 2a, each register 53 indicates that tile 4 belongs to Z3. As a result, each of tiles 4a-d issues a tile synchronization request when it reaches the barrier synchronization associated with Z3. When this barrier is reached, execution unit 52 of each tile 4 executes the synchronization instruction, taking the instruction of Z3 as the operator, thereby issuing a tile synchronization request on the synchronization request wire of each tile 4. Synchronization summary circuit 1500 summarizes the state of the synchronization request wires and provides an internal synchronization request (AZ3) to synchronization controller 55. As shown in FIG. 16, in response to receiving AZ3, synchronization controller 55 transmits an internal synchronization acknowledgement back to each tile 4a-d in processing unit 2a. Synchronization distribution wiring 1600 asserts an internal synchronization acknowledgement signal on the synchronization acknowledgement wire associated with Z3. Since these tiles 4a to 4d all belong to Z3, in response to the internal synchronization acknowledgement, the execution units 52 of each tile 4a to 4d pass through barrier synchronization and enter the exchange phase (which may be the internal exchange phase or the external exchange phase).

[0143] As described above, if the synchronization zone in which the internal synchronization request is received is configured as an internal synchronization zone, synchronization controller 55 acknowledges the synchronization request without providing an external synchronization request to GSP 58. However, if the synchronization zone is configured as an external synchronization zone, synchronization controller 55 forwards the external synchronization request to GSP 58 and waits to receive an external synchronization acknowledgement from GSP 58 before forwarding the internal synchronization acknowledgement to tile 4. GSP 58 itself contains different configuration settings that indicate how external synchronization requests should be propagated in different external synchronization zones.

[0144] To illustrate the concept of a synchronization network including a synchronization master node, synchronization propagation nodes, and slave nodes, reference is made to FIG. 19, which illustrates an example synchronization network 700 of a synchronization zone. The synchronization network 700 includes a synchronization master 710 and multiple synchronization propagation nodes 720a, 720b, and 720c. In one example embodiment, the synchronization master 710 and each of the synchronization propagation nodes 720a, 720b, and 720c are GSPs 58. The synchronization network 700 also includes multiple sets of slave nodes 730a, 730b, 730c, and 730d that issue synchronization requests. The slave nodes 730a, 730b, 730c, and 730d together form a synchronization group defined in the synchronization network 700. In the embodiment described herein, tiles 4 function as slave nodes for barrier synchronization, and synchronization requests originating from these slave nodes are the tile synchronization requests described above. The slave nodes are divided into different sets. For example, there is a first set of slave nodes 730a, a second set of slave nodes 730b, a third set of slave nodes 730c, and a fourth set of slave nodes 730d. In these embodiments, each set of slave nodes 730a, 730b, 730c, and 730d is a tile 4 of a different processing unit 2a. Each slave node issues a synchronization request upstream in the synchronization network 700. The synchronization requests from the slave nodes are aggregated and provided to a node higher in the synchronization network.

[0145] 19, each of the synchronization master 710 and synchronization propagation nodes 720a, 720b, and 720c has an associated downstream set of slave nodes from which it receives each barrier synchronization aggregate synchronization request. For example, synchronization propagation node 720a is associated with a first set of slave nodes 730a. In an embodiment in which the synchronization master 710 and synchronization propagation nodes 720a, 720b, and 720c are GSP58s, each set of slave nodes is tile 4 on the same chip 500 in the associated GSP58.

[0146] The synchronization propagation nodes 720b, 720c receive aggregate synchronization requests originating from their associated slave nodes 730b, 730c, but do not receive synchronization requests from other synchronization propagation nodes. In response to receiving a synchronization request originating from its associated slave node 730b, 730c, each synchronization propagation node 720b, 720c propagates an upstream synchronization request in the synchronization network 700 to the synchronization propagation node 720a.

[0147] The synchronization propagation node 720a waits until it receives a synchronization request from each downstream node. These downstream nodes include synchronization propagation nodes 720b, 720c, and a set of slave nodes 730a associated with synchronization propagation node 720a. When synchronization propagation node 720a has received all synchronization requests from each downstream node, it issues a synchronization request to the synchronization master 710.

[0148] The synchronization master 710 waits until it receives a synchronization request from each downstream node, including the synchronization propagation node 720a and a set of slave nodes 730d associated with the synchronization master 710. When the synchronization master 710 has received all synchronization requests from each downstream node, it issues a synchronization acknowledgement to the synchronization propagation node 720a and the slave node 730d.

[0149] When the synchronization propagation node 720a receives a synchronization acknowledgment from the synchronization master 710, it issues a synchronization acknowledgment to each downstream synchronization propagation node 720b, 720c and associated slave node 730a. Similarly, in response to receiving these synchronization acknowledgments, the synchronization propagation nodes 720b, 720c issue synchronization acknowledgments to their associated slave nodes 730b, 730c, respectively. In response to receiving the synchronization acknowledgments, all slave nodes 730a-d in the synchronization network 700 pass the barrier synchronization and exchange data during the exchange phase.

[0150] 19 shows a particular configuration of the synchronization network 700 in which the synchronization master 710 receives synchronization requests from only one downstream synchronization propagation node 720a. However, in other examples, the synchronization master 710 may receive synchronization requests from two or more downstream synchronization propagation nodes. Similarly, while the synchronization propagation node 720a in the example receives synchronization requests from two downstream synchronization propagation nodes 720b, 720c, it may alternatively receive synchronization requests from a different number of downstream synchronization propagation nodes.

[0151] Referring to Figure 17, Figure 17 shows a system 550 including multiple devices 500a-c, and illustrates how the GSPs 58 of these devices 500a-c exchange external synchronization requests and external synchronization acknowledgments in external barrier synchronization.

[0152] Each processing unit 2a-c issues an external synchronization request to its associated GSP 58. Such an external synchronization request is issued by the synchronization controller 55 of a processing unit 2 when the synchronization controller 55 of the processing unit 2 receives an aggregate synchronization request status (i.e., an internal synchronization request) indicating that each tile 4 of the processing unit 2 has issued a tile synchronization request. Each of the internal synchronization requests shown in Figure 17 as being sent by the synchronization controller 55 to the GSP 58 is associated with the same synchronization zone.

[0153] Each GSP 58 stores configuration settings for different synchronization zones that indicate how it responds to receiving external synchronization requests from those zones. These configuration settings indicate which interfaces of the GSP 58 are enabled in a particular synchronization zone and the directionality of the enabled interfaces (i.e., whether synchronization requests are sent or received on those interfaces). In the example of FIG. 17, the GSP 58 of device 500b is configured to propagate an external synchronization request upstream to the GSP 58 of device 500a in response to receiving an external synchronization request from the synchronization controller 55 of processing unit 2b. The GSP 58 of device 500a is configured to propagate an external synchronization request upstream to the GSP 58 of device 500c in response to receiving both an external synchronization request from the synchronization controller 55 of processing unit 2a and an external synchronization request from the GSP 58 of device 500b. Thus, the GSPs 58 of devices 500a and 500b both function as intermediate nodes (i.e., propagation nodes) in the synchronization network. The GSP 58 of the device 500c is configured to receive external synchronization requests from the GSP 58 of the device 500a and to receive external synchronization requests from the synchronization controller 55 of the device 500c. In response to receiving these external synchronization requests, the GSP 58 of the device 500c issues an external synchronization acknowledgement to the synchronization controller 55 of the device 500c and an external synchronization acknowledgement to the GSP 58 of the device 500a. Thus, the GSP 58 of the device 500c functions as a master node of the synchronization network. In response to receiving the external synchronization acknowledgement, the GSP 58 of the device 500a issues an external synchronization acknowledgement to the synchronization controller 55 of the device 500a and the GSP 58 of the device 500b. In response to receiving the external synchronization acknowledgement from the GSP 58 of the device 500a, the GSP 58 of the device 500b issues an external synchronization acknowledgement to the synchronization controller 55 of the device 500b. In response to receiving each synchronization acknowledgement, each synchronization controller 55 of devices 500a-c issues a synchronization acknowledgement to all tiles 4 of each processing unit 2-c, as described above with respect to Figure 14A. The tiles 4 belonging to that synchronization zone (as indicated in their synchronization zone registers 53) proceed to the external exchange phase in response to receiving such an external synchronization acknowledgement.

[0154] The synchronization zone in which the external synchronization requests and external synchronization acknowledgments shown in FIG. 17 are exchanged may be synchronization zone Z3 described above with reference to FIGS.

[0155] Referring to FIG. 18, FIG. 18 illustrates how data exchange may be performed between different processing units 2 during the external exchange phase of synchronization zones Z1 and Z2. In this example, synchronization zone Z1, described above with reference to FIGS. 15 and 16, includes tiles 4a and 4b as well as tiles 4e and 4f belonging to processing unit 2b. Registers 53 in tiles 4e and 4f include an indication that these tiles belong to synchronization zone Z1. When tiles 4a and 4b reach barrier synchronization of Z1, they each issue an external synchronization request to synchronization controller 55 of device 500a, as described above with reference to FIG. 15. Similarly, when tiles 4e and 4f reach barrier synchronization of Z1, they each issue a tile synchronization request to synchronization controller 55 of device 500b. In response, these synchronization controllers 55 forward the external synchronization requests to the GSPs 58 of their respective devices 500a and 500b. As described above with reference to Figure 17, the GSPs 58 of devices 500a, 500b exchange external synchronization requests and external synchronization acknowledgements, and then cause the tiles 4a, 4b, 4e, 4f of the devices 500a, 500b to which they belong to send internal synchronization acknowledgements (via the synchronization controllers 55 of the devices 500a, 500b). Data exchange then occurs between one or more of the tiles 4a, 4b and one or more of the tiles 4e, 4f via the interface 580 between the devices 500a, 500b. In Figure 18, an example of data exchange is shown with tile 4b sending data 590a to tile 4f.

[0156] Also shown in this example of FIG. 18, it is tile 4 that belongs to synchronization zone Z2, described above with reference to FIGS. 15 and 16. As shown, synchronization zone Z2 includes tiles 4h and 4g, which belong to processing unit 2c, in addition to tiles 4c and 4d, described above with reference to FIG. 15. Registers 53 in tiles 4h and 4g include an indication that these tiles belong to synchronization zone Z2. When tiles 4c and 4d reach barrier synchronization of Z2, they each issue a tile synchronization request to synchronization controller 55 of device 500a, as described above with reference to FIG. 15. Similarly, when tiles 4h and 4g reach barrier synchronization of Z2, they each issue a tile synchronization request to synchronization controller 55 of device 500c. In response, these synchronization controllers 55 forward external synchronization requests to GSPs 58 of those devices 500a and 500c, respectively. As described above with reference to Figure 17, the GSPs 58 of devices 500a, 500c exchange external synchronization requests and external synchronization acknowledgements, and then cause tiles 4c, 4d, 4h, 4g of the devices 500a, 500c to which they belong (via the synchronization controllers 55 of those devices 500a, 500b) to send internal synchronization acknowledgements. Data exchange then takes place between one or more of the tiles 4c, 4d and one or more of the tiles 4h, 4g via an interface 580 between devices 500a and 500c. In the example of Figure 18, tile 4d is shown sending data 590b to tile 4g.

[0157] 20, which illustrates a method 2000 for coordinating synchronization between processors 4 based on configurable synchronization groups (i.e., synchronization zones as described above). The method is performed on a single device 500.

[0158] In S2010, for each processor 4, an indication of which synchronization zone each processor 4 belongs to is stored in the processor's register 53. Each processor 4 stores in its register 53 an indication of whether or not the processor 4 belongs to each synchronization zone.

[0159] In S2020, each processor 4 executes instructions stored in memory 51. S2020 may be performed at the same time that other steps of method 2000 are being performed.

[0160] In S2030, the synchronization controller 55 of the device 500 receives a synchronization request from the processor 4 and returns a synchronization acknowledgement in response. The synchronization request is received by the synchronization controller 55 in the form of an aggregate (or internal) synchronization request resulting from individual tile synchronization requests. These tile synchronization requests include the requests issued in S2040 and S2050. Therefore, when S2040 and S2050 are executed, S2030 has not completed.

[0161] In S2040, a first processor of the plurality of processors 4 that does not belong to the first synchronization zone issues a synchronization request for the first synchronization zone. The first processor of the processors 4 asserts the request in response to an indication in register 53 of the first processor of the processors 4 that the first processor of the processors 4 does not belong to the first synchronization zone. The first processor of the processors 4 on which S2040 is executed may be, for example, tile 4c shown in FIG. 15, and the first synchronization zone is Z1.

[0162] After S2040, the synchronization controller 55 returns an acknowledgement to all processors 4 in the device 500 once all processors have asserted a synchronization request for Z1.

[0163] In S2050, a first one of processors 4 that belongs to the second synchronization zone issues a synchronization request to the second synchronization zone. The first one of processors 4 asserts the request in response to execution unit 52 reaching a synchronization point (e.g., a barrier) of the second synchronization zone in code in memory 51. When this synchronization point is reached, execution unit 52 executes a synchronization instruction to assert the synchronization request. The first one of processors 4 on which S2040 executes may be, for example, tile 4c shown in FIG. 15, and the second synchronization zone is Z2.

[0164] In S2050, the synchronization controller 55 sends an acknowledgement back to all processors 4 in the device 500 once all processors have asserted a synchronization request for the second synchronization zone.

[0165] 21, which illustrates a method 2100 for coordinating synchronization using a new scheme for signaling synchronization requests and acknowledgments. The method 2100 is performed by a single device 500.

[0166] At S2110, each processor 4 receives a signal representing the state of the synchronization acknowledge wire of each processor 4. Each such synchronization acknowledge wire for which a signal is received at S2110 is associated with the same synchronization zone. Each such signal received at each processor represents the same state (i.e., either high or low).

[0167] In S2120, each processor 4 asserts a synchronization request by setting the state of its synchronization request wire to the opposite of the state of its synchronization acknowledge wire in response to the received signal.

[0168] In S2130, in response to detecting that each synchronization request wire has been set to the opposite state of the synchronization acknowledge wire, the summation circuits 920, 910 output an aggregate synchronization request (i.e., an internal synchronization request) for the first barrier synchronization of the barrier synchronizations to the synchronization controller 55.

[0169] In S2140, in response to the aggregate synchronization request, the synchronization controller 55 sends a synchronization acknowledgement back to each processor 4. This is achieved by each processor 4 setting the state of its synchronization acknowledgement wire to the same as the state of its synchronization request wire.

[0170] It will be appreciated that the above embodiments have been described by way of example only, and in particular, although an embodiment has been described in which the synchronization point is a BSP barrier synchronization, it will be appreciated that in other embodiments the synchronization point may be a different type of synchronization point.

Claims

1. 1. A data processing device, comprising: a plurality of processors; a synchronization controller including circuitry configured to receive a request from the processor to participate in a synchronization and, in response to receiving the request, to return an acknowledgement to the processor; each of the processors comprising: an execution unit configured to execute a set of computer readable instructions retained in a memory of the respective processor; a register storing, for each of a set of configurable synchronization groups, an indication as to whether a respective processor belongs to a respective configurable synchronization group; and for a first one of the processors: the indication for a first one of the configurable synchrony groups indicates that the first one of the processors does not belong to the first one of the configurable synchrony groups; the indication for a second one of the configurable synchrony groups indicates that the first one of the processors belongs to the second one of the configurable synchrony groups; the first one of the processors includes circuitry configured to, in response to the indication for the first one of the configurable synchronization groups indicating that the first one of the processors does not belong to the first one of the configurable synchronization groups, assert a request to the synchronization controller to join synchronization of the first one of the configurable synchronization groups; a data processing device, wherein the circuitry of the first one of the processors is configured to assert to the synchronization controller a request to join synchronization of the second one of the configurable synchronization groups in response to the execution unit of the first one of the processors reaching a synchronization point of the second one of the configurable synchronization groups indicated in the set of computer-readable instructions for the first one of the processors.

2. 2. The data processing device of claim 1, wherein the circuitry of the synchronization controller is configured to issue a corresponding acknowledgement to each of the processors in response to all of the processors of the data processing device issuing a request to join the synchronization of the first one of the configurable synchronization groups.

3. The circuitry of the synchronous controller comprises: in response to the request to join the synchronization of the first one of the configurable synchronization groups, the processor issuing a further request to an external synchronization controller to synchronize with a further processor belonging to a further device; subsequently, in response to receiving a further acknowledgement of the further request from the external synchronization controller, transmitting the corresponding acknowledgement to each of the processors; 3. The data processing device of claim 2, configured to:

4. The external synchronization controller includes: a storage for storing a set of configuration settings for the first one of the configurable synchronization groups; circuitry configured to, in response to the further request received from the synchronization controller, exchange one or more further requests and one or more further acknowledgements with further devices according to the configuration of the first one of the configurable synchronization groups; 4. The data processing device of claim 3, comprising:

5. for each of the processors belonging to the first synchronization group of the configurable synchronization group, the execution unit of the respective processor is configured to issue a request to join the synchronization of the first synchronization group of the configurable synchronization group upon reaching a first barrier synchronization implemented between processors belonging to the first synchronization group of the configurable synchronization group; 2. The data processing device of claim 1, wherein for the first one of the processors, a respective execution unit is configured to proceed with computation or data exchange without waiting at the first barrier synchronization while the execution units of each of the processors belonging to the first one of the configurable synchronization groups are paused and waiting at the first barrier synchronization.

6. 2. The data processing device of claim 1, wherein the synchronization of the second one of the configurable synchronization groups is a second barrier synchronization, and the execution unit of the first one of the processors is configured to proceed across the second barrier synchronization in response to receiving an acknowledgment to the request to join the synchronization of the second one of the configurable synchronization groups.

7. 7. The data processing device of claim 6, wherein the execution unit of the first one of the processors is configured to proceed beyond the second barrier synchronization by entering an exchange phase in which the first one of the processors at least one of sends or receives data.

8. 2. The data processing device of claim 1, wherein the execution unit of the first one of the processors is configured to, after asserting the request to join the synchronizations of the first one of the configurable synchronization groups, execute an update instruction to update the indication for the first one of the configurable synchronization groups to specify that the first one of the processors belongs to the first one of the configurable synchronization groups.

9. 2. The data processing device of claim 1, wherein the execution unit of the first one of the processors is configured to execute an update instruction to update the indication for the second one of the configurable groups of synchronizations to specify that the first one of the processors does not belong to the second one of the configurable groups of synchronizations after asserting the request to join the synchronizations of the second one of the configurable groups of synchronizations.

10. a summing circuit, the summing circuit comprising: providing a first aggregate synchronization request to the synchronization controller in response to all of the processors providing respective requests to participate in the synchronization of the first one of the configurable synchronization groups; providing a second aggregate synchronization request to the synchronization controller in response to all of the processors providing the request to participate in the synchronization of the second one of the configurable synchronization groups; 2. The data processing device of claim 1, wherein the circuitry of the synchronization controller is configured to return an acknowledgement to the processor in response to each of the first aggregate synchronization request and the second aggregate synchronization request.

11. a first synchronization request wire connected to the first one of the processors; a second synchronization request wire connected to the first one of the processors; Including, the circuitry of the first one of the processors is configured to assert the request to join the synchronization of the first one of the configurable synchronization groups by asserting a signal on the first synchronization request wire; 2. The data processing device of claim 1, wherein circuitry of a second one of the processors is configured to assert the request to join the synchronization of the second one of the configurable synchronization groups by asserting a signal on the second synchronization request wire.

12. 2. The data processing device of claim 1, wherein the circuitry of the first one of the processors includes a first multiplexer configured to select a first input to output a first signal representing the request to join the synchronization of the first one of the configurable synchronization groups in response to the indication that the first one of the processors does not belong to the first one of the configurable synchronization groups.

13. 2. The data processing device of claim 1, wherein the circuitry of the first one of the processors includes a second multiplexer configured to select a second input to output a second signal controlled by the execution unit in response to the indication that the first one of the processors belongs to the second one of the configurable synchronous groups.

14. 14. The data processing device of claim 13, wherein the execution unit of the first one of the processors is configured, upon reaching the synchronization point, to execute a synchronization instruction to cause the second signal to be set to a state representing the request to participate in synchronization of a second processor of the configurable synchronization group.

15. the second one of the configurable synchronous groups includes a further processor belonging to one or more further data processing devices; 2. The data processing device of claim 1, wherein the first one of the processors is configured to participate in the synchronization of the second one of the configurable synchronization groups by exchanging data with one or more further processors.

16. 2. The data processing device of claim 1, wherein the execution unit of the first one of the processors is configured to execute a synchronization instruction upon reaching the synchronization point of the second one of the configurable synchronization groups to cause the assertion of the request to participate in the synchronization of the second one of the configurable synchronization groups.

17. The circuitry of the first one of the processors comprises: receiving a control signal from the execution unit of the first one of the processors indicating that the execution unit has reached the synchronization point; converting the control signal to the request to join the synchronization of the second one of the configurable synchronization groups; The data processing device of claim 1 configured to:

18. the first one of the processors includes a first interface configured to receive a first acknowledgement signal of the first one of the configurable synchronized groups; the circuitry of the first one of the processors is configured to invert the first acknowledgement signal to generate the request to join the synchronizations of the first one of the configurable synchronization groups; the first one of the processors includes a second interface configured to receive a second acknowledgement signal of the second one of the configurable synchronized groups; 2. The data processing device of claim 1, wherein circuitry of a second one of the processors is configured to invert the second acknowledgement signal to generate the request to join the synchronization of the second one of the configurable synchronization groups.

19. A data processing device according to any preceding claim which is an integrated circuit.

20. 1. A method implemented in a data processing device including a plurality of processors, comprising, in each of said processors: storing, for each of a set of configurable synchronization groups, an indication as to whether a respective processor belongs to the respective configurable synchronization group; executing a set of computer readable instructions contained in a memory of each processor; and for a first one of the processors: the indication for a first one of the configurable synchrony groups indicates that the first one of the processors does not belong to the first one of the configurable synchrony groups; and the indication for a second one of the configurable synchrony groups indicates that the first one of the processors belongs to the second one of the configurable synchrony groups; The method comprises: receiving, at a synchronization controller, a request from the processor to participate in synchronization and, in response to receiving the request, returning an acknowledgement to the processor; in a first one of the processors, in response to the indication for the first one of the configurable synchronization groups indicating that the first one of the processors does not belong to the first one of the configurable synchronization groups, asserting to the synchronization controller a request to join synchronization of the first one of the configurable synchronization groups; asserting a request to the synchronization controller to join synchronization of the second one of the configurable synchronization groups in response to an execution unit of the first one of the processors reaching a synchronization point of the second one of the configurable synchronization groups as indicated in the set of computer readable instructions for the first one of the processors; A method comprising:

Images