Hierarchical network design for dynamic dataflow architectures

Abstract

The processing unit for the dynamic dataflow architecture comprises multiple processing entity clusters (PCs), each having multiple processing entities. Each PC is associated with a switch that receives packets from other PCs and stores the received packets in a FIFO buffer associated with the PC. Each switch includes an arbiter that selects packets from one of the FIFO buffers and provides them as input to the PC associated with that switch.

Description

Technical Field

[0001] The present disclosure relates to computer architecture, and more particularly to computers using a dynamic data flow architecture.

Background Art

[0002] In a data flow architecture, the execution of an instruction is determined based on the availability of the inputs of the instruction. This data flow architecture is composed of a plurality of processing entities, the processing entities are mutually connectable, and data can be communicated between the processing entities.

[0003] Since the data flow architecture has a spatial nature, it has become difficult to scale up the design to support more complex workloads while maintaining high throughput. High interconnect complexity can result in increased power consumption, impact on performance, and chip area overhead. As an alternative, there is a method of improving performance using a faster and more efficient network within a cluster of processing elements (PEs) and connecting a large number of clusters to each other using a more flexible interconnect. For example, the processing clusters of individual processing entities can be interconnected with a mesh-type network-on-chip architecture.

[0004] Additional, alternative, and / or improved data flow architectures are desired.

Summary of the Invention

[0005] According to the present disclosure, there is provided a processing apparatus including a plurality of processing entity clusters (PCs) and a plurality of switches, each of the switches being associated with a respective PC.

[0006] Each switch comprises multiple FIFO (First-In, First-Out) packet buffers, each associated with each of the multiple PCs, and an arbiter that selects packets from the multiple FIFO packet buffers and provides the packets as input to the PCs associated with the switch.

[0007] In a further embodiment of the processing device, each of the multiple PCs comprises multiple processing entities and an external processing entity that controls input packets to the PCs and output packets from the PCs.

[0008] In a further embodiment of the processing unit, the external processing entity includes an arbiter that selects packets received from multiple processing entities and provides them as output from the PC, making selections based on ready signals from other PCs.

[0009] In a further embodiment of the processing unit, the external processing entity comprises a plurality of FIFO buffers, each associated with each processing entity of the PC, where received packets are stored in the FIFO buffers and read by the arbiter.

[0010] In a further embodiment of the processing unit, the arbiter of the external processing entity outputs the PC identifier along with the output from the PC.

[0011] The present disclosure further provides a method for providing interconnection between multiple processing clusters. The method includes the steps of: receiving packets from multiple processing clusters in multiple switches, each switch being associated with a respective processing cluster; storing the received packets in one of a plurality of FIFO buffers associated with the processing cluster that received the packets in each of the multiple switches; and selecting a packet from one of the switch's FIFO buffers and providing the packet as input to the processing cluster associated with the switch.

[0012] In a further embodiment of the present method, the method further includes the steps of receiving an input packet in each processing cluster, storing it in an input FIFO buffer, and broadcasting the packet output from the FIFO buffer to all processing entities in the processing cluster.

[0013] In a further embodiment of the present method, the method further includes the steps of: in the arbiter of each processing cluster, receiving ready signals from multiple processing clusters; selecting packets from multiple FIFO output buffers associated with each processing entity of the processing clusters; selecting one of the multiple FIFO output buffers; and outputting packets from the selected FIFO output buffer to multiple FIFO buffers of the switch.

[0014] This summary does not necessarily describe the entire scope of all embodiments of the time series forecasting system and method. Other embodiments, configurations, and advantages will become apparent to those skilled in the art by referring to the description of the specific embodiments below. [Brief explanation of the drawing]

[0015] [Figure 1] This is a diagram of a hierarchical dynamic data flow architecture. [Figure 2] This is a detailed diagram of the processing cluster. [Figure 3] This is a detailed diagram of an external processing entity. [Figure 4] This diagram shows the communication method between processing clusters. [Figure 5] This diagram shows how to provide external input to the processing cluster. [Figure 6] This diagram shows how output is provided from one processing cluster to another. [Modes for carrying out the invention]

[0016] The attached drawings show one or more exemplary embodiments.

[0017] This disclosure describes a hierarchical network design of processing entities suitable for a dynamic dataflow architecture. The network of processing entities is constructed to ensure packet ordering and enable connected clusters of processing elements to connect to each other using faster and more efficient interconnections, thereby achieving high throughput.

[0018] The processing entity network design described above offers higher performance than conventional mesh network-on-chip designs. However, this design may not be as scalable to a large number of processing clusters as conventional mesh network-on-chip designs. When using the processing entity network design described herein, the number of processing clusters that can be deployed on the chip may be limited by the achievable operating frequency targeted by the design. Nevertheless, although the current design may not always reach the level of scalability achievable by conventional mesh network-on-chip designs, extremely high scalability is not required for many applications, and the current network design can still provide sufficiently high performance.

[0019] Figure 1 shows a hierarchical dynamic dataflow architecture. The processing entity cluster (PC) interconnect, i.e., the network design 100, can be implemented on one or more chips, including, for example, a field-programmable gate array (FPGA), an application-specific integrated circuit, or a general-purpose dynamic dataflow processor. The PC interconnect 100 comprises several PCs 102a to 102m (collectively referred to as PC102). Each PC102 groups together several individual processing entities capable of processing data. When processing entities are hierarchically grouped into multiple clusters, some form of interconnection between PCs may be required to provide data from one PC to another.

[0020] In this architecture, connections between PCs are provided by multiple switches 104a to 104m (collectively referred to as switch 104). Each switch 104 is associated with one of the PCs 102. Switch 104 receives packets from all PCs 102 and provides input to the PC to which it is associated. For example, switch 104a is shown as being associated with PC 104a and provides input to PC 104a. Similarly, switch 104b is shown as being associated with PC 102b, switch 104c is associated with PC 102c, and switch 104m is shown as being associated with PC 104m.

[0021] Each of the switches 104 has an architecture similar to that shown for switch 104a. As shown in the figure, switch 104a has multiple FIFO (First-In, First-Out) input buffers 106a to 106m (collectively referred to as input buffer 106). Each input buffer is associated with one of the PCs 102. As shown in the figure, input buffer 106a is associated with PC 102a, input buffer 106b is associated with PC 102b, and so on. Each input buffer 106 receives data packets output from the PC to which it is associated. The arbiter 108a of switch 104a selects a packet from one of the input buffers 106 and provides it to the PC to which the switch is associated. Each switch has multiple input buffers, each associated with its respective PC. A PC can broadcast packets to its associated buffers within each switch. That is, PC 102a may output packets not only to input buffer 106a of switch 104a, but also to the input buffers of the other switches 102b to 102m.

[0022] The processing entity network shown in FIG. 1 processes data transfer between PCs (PC-to-PC data transfer) and includes a number of switches. Packets from each PC are multicast simultaneously to all switches to reduce latency. Each switch includes a number of FIFO buffers and an arbiter. The FIFO is used to buffer packets from each PC, and the arbiter is used to select one of the packets from the FIFO and transfer the data to the target PC associated with the switch.

[0023] By using FIFO buffers in the interconnection, the packet delivery order of each source-destination PC pair is surely maintained during PC-to-PC data transfer. Although described as PC-to-PC transfer, data transfer may also occur between processing entities belonging to different PCs. In a conventional mesh network-on-chip, a routing algorithm for PC-to-PC data transfer that does not maintain packet order has been used. By maintaining the packet delivery order, there is no need for the compiler to find an optimal solution and no need to consider timing information to ensure communication accuracy, significantly simplifying the program compilation process.

[0024] FIG. 2 is a detailed view of a processing cluster. The processing cluster architecture shown in FIG. 2 can be used as any of the PCs 102 shown in FIG. 1. As shown, PC 200 includes a plurality of individual processing entities 202a to 202n (collectively referred to as processing entity 202). Each individual processing 202 entity may have a structure and function substantially equivalent to other processing entities. In addition to the processing entity 202 that processes data, PC 200 also includes an external processing entity 204 responsible for communication with the outside of the PC. Both the output packet 206 and the input packet 208 are processed by the external processing entity 204.

[0025] The processing entity 202 can communicate data with other processing entities. Note that not all possible communication paths between processing entities are shown in FIG. 2. Each of the processing entities 202 can also communicate with an external processing entity and receive data from the external processing entity. When any of the processing entities 202 has a data packet that needs to be transferred outside the PC 200, the data packet is sent to the external processing entity 204, and the external processing entity 204 is controlled to send the data packet to the destination or multiple destinations. Similarly, when data is destined for one or more processing entities 202 within the PC 200, the data is received at the external processing entity and distributed to the processing entities.

[0026] The external processing entity can be added to the PC to handle external packet communication. When the external processing entity is included, the processing entity first sends the packet that needs to be transferred outside the PC to the external processing entity to process the output packet. The external processing entity can select one of the packets received from the processing entities within the cluster that are required by the destination. The destination can be, for example, one or more processing clusters within the architecture. Similarly, the external processing entity is an entity that receives input packets and broadcasts data to other internal PEs via an internal data channel.

[0027] Figure 3 is a detailed diagram of the external processing entity. The processing entity cluster (PC) 300 can be any of the PCs 102 shown above in Figure 1. PC 300 comprises multiple processing entities 302a to 302n (collectively referred to as processing entities 302). Each processing entity 302 communicates with an external processing entity 304 that processes input packets to PC 300 and output packets from PC 300. The external processing entity 304 comprises multiple FIFO buffers 306a to 306n (collectively referred to as FIFO buffers 306) each associated with one of the processing entities 302. The FIFO buffers receive data packets from processing entities 302 destined for other PCs (not shown).

[0028] The external processing entity 304 includes an arbiter 308 that selects data packets from the FIFO buffer 306 and forwards them to appropriate destinations. When selecting data packets, the arbiter 308 may receive ready signals 310 from other PCs. When all destination PCs corresponding to the data packet are ready to receive, the arbiter 308 can select and forward a data packet from any of the buffers 306. The selected packet may be forwarded to all destination PCs simultaneously. The arbiter sends out an output packet 312, which may include an identifier 314 of PC 300 indicating which PC sent the data packet. When a data packet from another PC is destined for PC 300, an input packet 316 is received, stored in the FIFO buffer 318, and then broadcast to all processing entities 320.

[0029] As shown in Figure 3, the components for processing outgoing packets are similar to those in a PC interconnect and include a PE buffer 306, a PC identifier (PC id) 314, and an arbiter 308. The arbiter is responsible for forwarding the packet in the FIFO to all destination PCs when all destination PCs are ready to receive the packet. When a packet is sent, the PC identifier is added as payload, so that the processing entities in the processing cluster can recognize the source address. The FIFO buffer is used for processing incoming packets. The incoming packet is then broadcast to all internal processing entities. It is assumed that the internal processing entities know the data necessary to perform the calculations based on their programming information.

[0030] Figure 4 shows a method of communication between processing clusters. Method 400 can be implemented by the dynamic data flow architecture described above. This method receives packets from processing clusters (402). Packets received from processing clusters may be received at switches associated with each PC. Received packets may be destined for one or more PCs and sent to switches associated with the destination PCs. The received packets may then be stored in a FIFO buffer associated with the processing cluster that sent the packets at each switch that received them (404). The FIFO buffer, and therefore the packets stored in the FIFO buffer, are selected and provided to the processing cluster associated with the switch of the FIFO buffer (406). Each switch may select packets from its FIFO buffer and send them to its associated PC.

[0031] Figure 5 shows a method for providing external input to a processing cluster. Method 500 can be performed by an external processing entity as described above. Method 500 receives input packets from a switch (502) of the interconnect network as described above. The received packets are stored in an input FIFO buffer (504). The packets are output from the input FIFO buffer and broadcast to all processing entities on the PC (506).

[0032] Figure 6 shows a method for providing output from one processing cluster to another. Method 600 can be performed by an external processing entity as described above. Method 600 receives an output packet from the processing entity (602). The packet may be stored in a FIFO buffer associated with each processing entity. The output packet may specify one or more destination PCs, or may be associated with one or more destination PCs. Ready signals are received from multiple PCs (604). Ready signals indicate that the PCs are ready to receive data. Based on the ready signals, a packet is selected from the FIFO buffer (606). A packet may be selected if all destination PCs in the packet are ready, as indicated by the ready signals. The selected packet is output to the switch associated with the destination PCs (608).

[0033] The hierarchical dataflow architecture described above enables communication between clusters of processing entities. When multiple packets are destined for the same destination, this design stores these packets and, based on a ready signal from the receiver, selects and forwards one of them when the target receiver is ready to receive. In this way, the design operates dynamically without requiring timing information from the compiler. This improves the number of packets processed per unit time, i.e., throughput. When a packet is destined for multiple destinations, it is multicast simultaneously to all switches associated with the destination PC. This reduces packet delivery time compared to mesh networks where latency increases due to multi-hop data transfer. The single-cycle multicast functionality ensures that packets are delivered in the order they were sent, avoiding potential errors caused by mixed packet arrival order, and also contributes to simplifying the design complexity and compilation process.

[0034] Any part of any aspect or embodiment described herein is intended to be implemented or combined with any part of any other aspect or embodiment described herein.

[0035] The claims should not be limited to the embodiments described above, but should be interpreted in the broadest way that is consistent with the overall description.

[0036] It should be recognized that the configurations and aspects of the various embodiments shown above can be combined with further embodiments included within the scope of this disclosure. In addition, the figures are not to scale and may be exaggerated in size and shape for illustrative purposes.

Claims

1. Processing apparatus, Multiple processing entity clusters (PCs), Multiple switches, each of which is associated with a PC, and Equipped with, Each of the switches is Multiple FIFO (First-In, First-Out) packet buffers associated with each of the aforementioned multiple PCs, An arbiter selects a packet from the plurality of FIFO packet buffers and provides the packet as input to the PC associated with the switch. A processing device equipped with the following features.

2. The aforementioned multiple PCs Multiple processing entities, An external processing entity that controls input packets to the PC and output packets from the PC. The apparatus according to claim 1, comprising each of the following:

3. The processing apparatus according to claim 2, wherein the external processing entity includes an arbiter that selects packets received from the plurality of processing entities and provides them as output from the PC, and performs the selection based on a ready signal from another PC.

4. The processing apparatus according to claim 3, wherein the external processing entity comprises a plurality of FIFO buffers, each associated with each processing entity of the PC, and the received packets are stored in the FIFO buffers and read by the arbiter.

5. The processing apparatus according to claim 4, wherein the arbiter of the external processing entity outputs the identifier of the PC along with the output from the PC.

6. A method for providing interconnection between multiple processing clusters, A step of receiving packets from multiple processing clusters in multiple switches, wherein each of the switches is associated with its respective processing cluster, The steps include: in each of the plurality of switches, storing the received packet in one of the plurality of FIFO buffers associated with the processing cluster that received the packet, first-in, first-out buffer; The steps include: selecting a packet from one of the FIFO buffers of the switch in each of the switches, and providing the packet as input to the processing cluster associated with the switch. Methods that include...

7. The method according to claim 6, further comprising the steps of receiving the input packets in each processing cluster, storing them in an input FIFO buffer, and broadcasting the packets output from the FIFO buffer to all processing entities in the processing cluster.

8. The arbiter of each processing cluster receives ready signals from multiple processing clusters and selects packets from multiple FIFO output buffers associated with each processing entity of the processing cluster. The steps include selecting one of the aforementioned plurality of FIFO output buffers, The steps include outputting the packet from the selected FIFO output buffer to multiple FIFO buffers of the switch, and The method according to claim 7, further comprising:

Images