Fair arbitration among multiple sources targeting a destination

Abstract

The hardware module includes at least a first input buffer and a second input buffer, the second input buffer holding data packets from multiple source components. The module first determines that a packet at the head of the first input buffer is targeted to a first destination. To ensure fairness between one or more sources providing data to the first input buffer and multiple sources providing data to the second input buffer, the processing circuitry examines source identifiers in packets held in the second input buffer and selects between the buffers to arbitrate between the sources. In some embodiments, examining the source identifiers provides weighted round robin statistics between the input buffers. In other embodiments, the source identifier of which packet is currently at the head of the second input buffer is used to perform a simple round robin between the sources.

Description

[Technical Field]

[0001] This application relates to a hardware module, and in particular to a hardware module for receiving data originating from multiple sources and forwarding at least a portion of that data to a first destination. [Background technology]

[0002] When processing data for complex or large volumes of applications, a work accelerator may be a subsystem where certain data processing is offloaded from the host system. Such a work accelerator may have specialized hardware to perform a particular type of processing.

[0003] Specifically, a work accelerator specialized for machine learning applications may have an architecture that supports a high degree of parallelism. One form of parallelism can be achieved by a processor including an arrangement of multiple tiles on the same chip (i.e., the same die), with each tile including an individual processing unit and memory (including program memory and data memory). Thus, separate portions of program code can be executed in parallel on different tiles. The tiles are connected via an on-chip interconnect that enables data exchange between the tiles. Such an accelerator can function as a subsystem for a host system to perform parallel processing of a dataset provided to it.

[0004] A work accelerator may be provided on an integrated circuit (i.e., chip), which is a collection of electronic circuits fabricated on a single semiconductor material (e.g., silicon). In addition to providing multiple tiles of work accelerators on a single chip, additional system-on-chip (SoC) components may be provided to support the operation of the work accelerator. Furthermore, various interfaces may be provided on the chip to support communication between the work accelerator and other work accelerators or off-chip devices, such as a host device.

[0005] When multiple different components exchanging data are provided, for example on a single chip, various challenges arise in enabling communication between those components. Summary of the Invention [Means for solving the problem]

[0006] We propose a hardware module for use as a switch that forwards data originating from multiple source components and destined for multiple destination components. For example, the switch may have multiple input ports for receiving data from different source components. When the network configuration for forwarding data packets between components results in at least some of the input ports receiving data from multiple data sources, a challenge arises in ensuring fair data forwarding to a particular destination. In this case, using a simple round-robin scheme for arbitration between input ports may unfairly prioritize traffic from certain sources targeting a particular destination, i.e., from sources that share an input port with fewer other source components.

[0007] According to a first aspect, there is provided a hardware module for receiving data from a plurality of source components and transmitting at least a portion of the data to a first destination, the hardware module comprising: a first input port configured to receive a first set of data packets originating from a first subset of source components and provide them to a first input buffer, the first subset consisting of one or more of a plurality of source components; a second input port configured to receive a second set of data packets originating from a second subset of source components and provide them to a second input buffer, the second subset of source components including two or more source components, the first subset consisting of a different number of source components for the second subset; and an output port for outputting the first set of data packets and the second set of data packets, the output port configured to output the first set of data packets and the second set of data packets to the first destination. and a processing circuit configured to: examine a destination identifier in one of the first set of data packets at the head of the first input buffer; examine one or more source identifiers in the one or more second sets of data packets; determine which of a second subset of source components each of the one or more second sets of data packets originated from; and, in response to determining that the destination identifier identifies the first destination, select between the first input buffer and the second input buffer as a source of data to the first destination according to the one or more second source identifiers; and arbitrate among the multiple source components for transmitting data to the first destination.

[0008] The circuitry is configured to first determine whether a packet at the head of the first input buffer is targeted to the first destination. If so, it arbitrates between the sources to ensure fairness among sources targeted to the first destination. By analyzing the origin of the packet in the second input buffer (i.e., which source component the data packet originated from), the processing circuitry can arbitrate between multiple sources for sending data to the first destination on the output port. In some embodiments, arbitration between sources is achieved by using inspection of a source identifier in the second input buffer to provide statistics for a weighted round robin involving the first input buffer and the second input buffer. In other embodiments, the source identifier of the packet at the head of the second input buffer is used to perform a simple round robin among sources.

[0009] In some embodiments, the processing circuit is configured, upon selecting the second input buffer, to transmit a first data packet of the second set of data packets located at the beginning of the second input buffer, and the one or more second source identifiers are source identifiers of packets of the second set of data packets transmitted from the second input buffer prior to transmission of the first data packet of the second set of data packets.

[0010] In some embodiments, the hardware module includes storage configured to store an indication of a number of source components from which data was last transmitted from the second input buffer, and selecting between the first input buffer and the second input buffer includes performing a weighted round robin between the first input buffer and the second input buffer, wherein the second input buffer is weighted by the number of source components from which data was last transmitted from the second input buffer.

[0011] In some embodiments, the hardware module includes a set of input buffers configured to receive data originating from a second subset of source components, the set of input buffers including a second input buffer.

[0012] In some embodiments, the storage includes, for each of the second subset of source components, an indication of to which of the set of input buffers data originating from the individual source component was last sent, and the indication of the number of source components is given by the number of the second subset of source components, for which the indication of to which of the set of input buffers data originating from the individual source component was last sent specifies a second input buffer.

[0013] In some embodiments, a first data packet of the first set of data packets located at the head of the first input buffer originates from a first source component of a first subset of source components identified by the first source identifier, and examining one or more second source identifiers includes examining a second source identifier in the first data packet of the second set of data packets located at the head of the second input buffer, and the processing circuitry is configured to select the first data packet of the second set of data packets for transmission at the output port according to the second source identifier and the first source identifier in the first data packet of the second set of data packets.

[0014] In some embodiments, the processing circuitry is configured to determine the first source identifier by examining the first source identifier in a first data packet of the first set of data packets.

[0015] In some embodiments, the hardware module further includes a register that stores an identifier of one of the plurality of source components from which data was most recently transmitted to the first destination, and selecting between the first input buffer and the second input buffer includes sequentially selecting the next of the plurality of source components following the one of the plurality of source components identified in the register.

[0016] In some embodiments, selecting between the first input buffer and the second input buffer includes applying a round-robin scheme among the multiple source components.

[0017] In some embodiments, the first subset of source components consists of a single source component, and the first input buffer is dedicated to holding data packets originating from the single source component.

[0018] In some embodiments, the hardware module includes a set of input buffers for storing data originating from a second subset of source components, each of the set of input buffers being devoted to storing data for transmission to a different one of the multiple destinations, and the second input buffer being devoted to storing data for transmission to the first destination.

[0019] In some embodiments, each set of input buffers is associated with a different virtual channel of the interconnect in which the hardware module serves as a node.

[0020] In some embodiments, the processing circuitry is configured to select the first destination as the next destination to send the data to as part of an arbitration scheme for arbitrating among multiple destinations for sending data on the output port before selecting between the first input buffer and the second input buffer.

[0021] In some embodiments, a first bandwidth available for transmitting on the output port exceeds a second bandwidth for receiving data at the first input port.

[0022] In some embodiments, the hardware module is a node on an interconnect, the first input port is a local input port for receiving a first set of data packets from a first subset of a plurality of source components connected to the node, and the second input port is configured to receive a second set of data packets from a further node on the interconnect.

[0023] In some embodiments, the second input port is configured to receive data for transmission to at least one of the first subset of source components, and the processing circuitry is configured to provide this data for transmission to at least one of the first subset of source components on a further output port of the hardware module.

[0024] According to a second aspect, there is provided an apparatus including a hardware module according to the first aspect or any embodiment thereof, the apparatus including a plurality of source components and a first destination.

[0025] In some embodiments, the apparatus includes multiple instances of a hardware module connected together as nodes forming part of an interconnect, each of the instances configured to receive data from a different one or more of the source components.

[0026] In some embodiments, the apparatus includes a further hardware module connected to the first destination, the further hardware module configured to output data from different ones of the source components connected to each of the multiple instances of the hardware module to the first destination, wherein the aggregate bandwidth of the different ones of the source components exceeds the bandwidth of the further hardware module for supplying data to the first destination.

[0027] In some embodiments, the device is an integrated circuit.

[0028] According to a third aspect, there is provided a method for receiving data from a plurality of source components and forwarding at least a portion of the data to a first destination of a plurality of destinations, the method comprising the steps of receiving a first set of data packets originating from a first subset of the source components and providing them to a first input buffer, receiving a second set of data packets originating from a second subset of the source components and providing them to a second input buffer, examining a destination identifier in one of the first set of data packets at the top of the first input buffer, examining one or more source identifiers in one or more second sets of data packets to determine which of the second subset of source components each of the one or more second sets of data packets originated from, and, in response to determining that the destination identifier identifies the first destination, selecting between the first input buffer and the second input buffer as a source of data to the first destination in response to the one or more second source identifiers and arbitrating among the plurality of source components to transmit the data to the first destination. The first subset comprises one or more of the plurality of source components, some of the first set of data packets are for transmission to different ones of the plurality of destinations, and the second subset of source components includes two or more source components, the first subset comprising a different number of source components relative to the second subset.

[0029] In some embodiments, the method includes, upon selecting the second input buffer, transmitting a first data packet of the second set of data packets located at the beginning of the second input buffer, wherein the one or more second source identifiers are source identifiers of packets of the second set of data packets transmitted from the second input buffer prior to transmission of the first data packet of the second set of data packets.

[0030] In some embodiments, the method includes storing an indication of the number of source components from which data was last transmitted from the second input buffer, and wherein selecting between the first input buffer and the second input buffer includes performing a weighted round robin between the first input buffer and the second input buffer, wherein the second input buffer is weighted by the number of source components from which data was last transmitted from the second input buffer.

[0031] In some embodiments, the second input buffer belongs to a set of input buffers, each of which is configured to receive data originating from a second subset of source components.

[0032] In some embodiments, the method includes storing, for each of the second subset of source components, an indication of to which of a set of input buffers data originating from the individual source component was last sent, wherein the indication of the number of source components is given by the number of the second subset of source components for which the individual indication of to which of the input buffers data originating from the individual source component was last sent specifies a second input buffer.

[0033] In some embodiments, a first data packet of the first set of data packets located at the head of the first input buffer originates from a first source component of a first subset of source components identified by a first source identifier, and the step of examining one or more second source identifiers includes examining a second source identifier in the first data packet of the second set of data packets located at the head of the second input buffer, and the method includes selecting the first data packet of the second set of data packets for transmission in response to the second source identifier and the first source identifier in the first data packet of the second set of data packets.

[0034] In some embodiments, the method includes determining the first source identifier by examining a first data packet of the first set of data packets.

[0035] In some embodiments, the method includes storing an identifier of one of a plurality of source components from which data was most recently transmitted to the first destination, and selecting between the first input buffer and the second input buffer includes sequentially selecting the next of the plurality of source components following the identified one of the plurality of source components.

[0036] In some embodiments, the step of selecting between the first input buffer and the second input buffer includes applying a round robin scheme among the multiple source components.

[0037] In some embodiments, the first subset of source components consists of a single source component, and the first input buffer is dedicated to holding data packets originating from the single source component.

[0038] In some embodiments, the second input buffer is one of a set of input buffers each for storing data originating from a second subset of source components, each of the set of input buffers being devoted to storing data for transmission to a different one of the multiple destinations, and the second input buffer being devoted to storing data for transmission to the first destination.

[0039] In some embodiments, each set of input buffers is associated with a different virtual channel of the interconnect in which the hardware module serves as a node.

[0040] In some embodiments, the method includes selecting the first destination as the next destination to send the data to as part of an arbitration scheme for arbitrating among multiple destinations for sending the data before selecting between the first input buffer and the second input buffer.

[0041] In some embodiments, a first bandwidth available for transmitting data to a first destination exceeds a second bandwidth for receiving data from a first subset of source components.

[0042] In some embodiments, the method is implemented within a node of an interconnect, wherein a first set of data packets is received at a local input port from a first subset of a plurality of source components connected to the node, and a second set of data packets is received from a further node of the interconnect.

[0043] In some embodiments, the method includes receiving data for transmission to at least one of the first subset of source components, and providing the data for transmission to at least one of the first subset of source components.

[0044] In some embodiments, the method is implemented within a hardware module of an apparatus, the apparatus including a plurality of source components and a first destination.

[0045] In some embodiments, the apparatus includes multiple instances of a hardware module connected together as nodes forming part of an interconnect, each of the instances configured to independently perform the steps of the method.

[0046] In some embodiments, the apparatus includes a further hardware module connected to the first destination, the further hardware module configured to output data from different ones of the source components connected to each of the multiple instances of the hardware module to the first destination, wherein the aggregate bandwidth of the different ones of the source components exceeds the bandwidth of the further hardware module for supplying data to the first destination.

[0047] In some embodiments, the device is an integrated circuit.

[0048] According to a fourth aspect, there is provided a computer program product comprising a set of computer-readable instructions for causing at least one processor to perform a method, the method comprising: examining a destination identifier in one of a first set of data packets at the head of a first input buffer; examining one or more source identifiers in one or more second sets of data packets to determine which of a second subset of source components each of the one or more second sets of data packets originated from; and, in response to determining that the destination identifier identifies a first destination of a plurality of destinations, selecting between the first input buffer and the second input buffer as a source of data to the first destination according to the one or more second source identifiers and arbitrating among the plurality of source components for transmitting data to the first destination, wherein the first set of data packets originates from the first subset of a plurality of source components, the first subset consisting of one or more of the plurality of source components, and some of the first set of data packets are intended for transmission to different ones of the plurality of destinations. Additionally, the second subset of the plurality of source components includes two or more source components, and the first subset of source components is comprised of a different number of source components relative to the second subset.

[0049] In some embodiments, the method comprises any of the steps included in the method embodiments according to the third aspect.

[0050] According to a fifth aspect, there is provided a non-transitory computer readable medium storing a computer program according to the fourth aspect.

[0051] For a better understanding of the present invention, and to show how the same may be carried into effect, reference will now be made, by way of example, to the accompanying drawings in which: [Brief explanation of the drawings]

[0052] [Figure 1A] 1 illustrates an exemplary hardware module having a single input buffer for receiving data from a locally connected source component. [Figure 1B] 1 illustrates an exemplary embodiment in which an arbitration circuit selects between a single input buffer containing data from multiple sources and a single virtual channel buffer. [Figure 1C] 1 illustrates an exemplary embodiment in which an arbitration circuit selects between a single input buffer containing data from a single source and a single virtual channel buffer. [Figure 1D] 1 illustrates an exemplary embodiment in which an arbitration circuit selects between a single input buffer containing data from a single source and multiple virtual channel buffers. [Figure 1E] 10 illustrates a further exemplary embodiment in which an arbitration circuit selects between a single input buffer containing data from a single source and multiple virtual channel buffers. [Figure 2A] 1 illustrates an exemplary hardware module having multiple input buffers for receiving data from locally connected source components. [Figure 2B] 1 illustrates an exemplary embodiment in which an arbitration circuit selects between two input buffers and a single virtual channel buffer. [Figure 2C] 1 illustrates an exemplary embodiment in which an arbitration circuit selects between two input buffers and two virtual channel buffers. [Figure 2D]10 illustrates a further exemplary embodiment in which an arbitration circuit selects between two input buffers and two virtual channel buffers. [Figure 3] 1 illustrates an exemplary hardware module that supports the ingress of data from and the output of data to multiple other hardware modules. [Figure 4] 1 illustrates an exemplary interconnect in which multiple hardware modules function as nodes / switches of the interconnect. [Figure 5] 1 shows an example of how interconnects are used to deliver data packets to components on a chip. [Figure 6] 1 illustrates an example of a method according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0053] Embodiments of the present application relate to a hardware module for receiving data packets from multiple sources and forwarding those packets to a destination. As described in more detail below, multiple such hardware modules may be implemented as a switch in a network on a chip for providing data packets between components of the chip. The components of the chip may include processing units. An example of a chip including processing units (in the form of a multi-tile processing unit) is described in more detail in our prior U.S. patent application Ser. No. 16 / 276,834, which is incorporated by reference.

[0054] Reference is made to FIG. 1A , which illustrates an example of a hardware module 100 according to an embodiment of the present application. The hardware module 100 includes a first input port 101 (labeled as “Xi0” in FIG. 1A ) for receiving data packets from one or more source components 103. The one or more source components 103 may be locally connected source components 103. The hardware module 100 also includes a second input port 102 (labeled as “tia” in FIG. 1A ) for receiving data packets originating from multiple additional source components 104. While the multiple additional source components 104 are shown as being directly connected to the hardware module 100, in at least some embodiments, multiple source components are connected to the hardware module 100 via additional intermediate modules (as shown in FIG. 4 ).

[0055] A first input buffer 105 is provided for storing data packets received at hardware module 100 at a first input port 101. The data packets stored in this buffer 105 are for transmission from hardware module 100 to one or more destinations at output port 106. Hardware module 100 includes additional buffers 107 for storing data packets received at hardware module 100 at a second port 102. Each of these additional buffers 107 can be thought of as operating as a second input buffer for storing data packets from multiple destinations.

[0056] Each of these additional buffers 107 is associated with a different destination component to which the data is to be sent. Each data packet received at port 102 includes a destination identifier in its header. Circuitry in hardware module 100 is configured to examine the destination identifier in the packet's header when a data packet is received at port 102 and store the data packet in one of the buffers 107 associated with the identified destination. Each of these buffers 107 is labeled vc0, vc1, vc2, etc., to indicate that each buffer 107 is associated with a different destination. Each of the source components 103 connected to module 100 constitutes one of these different destinations, so that when a data packet arrives from source component 104 for transmission to one of components 103, the data is sent to one of the buffers 107 associated with one of components 103. Circuitry 111 sends the data from one of the buffers to output buffer 109, from which it is sent at a further output port 110 to an associated one of the source components 103.

[0057] Each of the destinations to which module 100 is configured to send data is associated with a virtual channel. Thus, each of buffers 107 is associated with a virtual channel, and the virtual channels are labeled vc0, vc1, etc. As will be described in more detail with respect to FIG. 5, using virtual channels to send data to different destinations prevents head-of-line blocking.

[0058] Hardware module 100 includes arbitration circuit 111, which is a processing circuit for determining from which of buffers 105, 107 data should be sent next. Arbitration circuit 111 first selects one of the destinations for which data should be sent. In other words, the arbitration scheme first selects one of the destination components as the next component to send data to. The selected destination can be one of components 103 or it can be a remote destination accessible at port 106. The destination selection is performed using a round-robin among the destinations. For example, if the last data sent was from buffer vc0, the next data to send is from buffer vc1.

[0059] Once a destination to which data should be sent is selected, arbitration circuit 111 selects whether to send data from first input buffer 105 or from one of the further buffers 107 associated with the selected destination. The selection between first input buffer 105 and one of the further buffers 107 is performed depending on the source components from which the data packets in those buffers originated. In some embodiments, a source identifier of a packet at the head of one of the further buffers 107 is used to perform a simple round robin between the sources. Such an exemplary embodiment is described below with respect to FIGS. 1B-1D. In other embodiments, arbitration between the sources is achieved by using an examination of a source identifier in one of the further buffers 107 to provide weighted round robin statistics involving first input buffer 105 and one of the further buffers 107. Such an exemplary embodiment is described below with respect to FIG. 1E.

[0060] 1B, which shows an example of how the selection between the first input buffer 105 and the second input buffer 112 can be made. The second input buffer 112 can be one of the further buffers 107 discussed above with respect to FIG. 1A. In that case, the second input buffer 112 is associated with a destination (Ds=0) that is selected by applying a round robin among the destinations. In this example, the example data packet shown in the first input buffer 105 is intended for transmission to the same destination (Ds=0) as the data packet in the second input buffer 112.

[0061] The first input buffer 105 contains multiple data packets 113a-d. The second input buffer 112 contains multiple data packets 114a-d. Each of the data packets 113a-d, 114a-d includes a header and a payload. Each header includes a source identifier (identifying which of the source components 103, 104 the packet originated from) and a destination identifier (identifying which of the destination components the data packet is being sent to). In FIG. 1B, the source identifier is shown as "Sc=_", while the destination identifier is shown as "Ds=_". Each of the buffers 105, 112 may be a first-in, first-out (FIFO) buffer, and the data packet at the head of each of the buffers 105, 112 is the next packet to be transmitted from that buffer 105, 112.

[0062] Circuit 111 selects one of buffers 105, 112 by arbitration among the source components. This may be achieved by applying a round-robin scheme among the source components to select one of the source components from which data originating from that source component should next be transmitted. Arbitration circuit 111 may access register 115, which stores an indication of the source component from which the last data packet transmitted from one of buffers 105, 112 originated. This identified source component may be referred to as the last source component, and the indication in register 115 may be referred to as the last source identifier. This indication is used by circuit 111 to select one of buffers 105, 112 from which to transmit the next data packet. Circuit 111 applies an arbitration scheme so that the next of the two packets at the top of buffer 105, 112 to be transmitted is the packet with the next highest source identifier, or, if no such packet with a higher source identifier can be determined, the packet with the lowest source identifier. Thus, the buffer selected as a result of this arbitration scheme is one of the buffers 105, 112 that contains the packet. After transmitting the data packet from that selected buffer 105, 112, circuit 111 updates the indication in register 115 with the source identifier of the data packet just transmitted.

[0063] 1B, one or more source components 103 include at least two source components (labeled as sources Sc=3 and Sc=7), and a further source component 104 includes at least three source components (labeled as sources Sc=1, Sc=5, and Sc=9). A first input buffer 105 stores data packets originating from sources Sc=3 and Sc=7, while a second input buffer 112 stores data packets originating from sources Sc=1, Sc=5, and Sc=9.

[0064] Suppose the register identifies the source component of the data packet last transmitted from one of the buffers 105, 112 as source Sc=1. Circuit 111 examines the source identifiers of packets 113a, 114 at the head of each of buffers 105, 112 (i.e., the next to be transmitted from that buffer). Packet 113a has a source identifier of source Sc=3, while packet 114a has a source identifier of source Sc=1. Because the source identifier of packet 113a is higher than the identifier from register 115 (while the source identifier of packet 114a is equal to the identifier from register 115), arbitration circuit 111 causes packet 113a to be transmitted on output port 106. Arbitration circuit 111 then updates the identifier in register 115 to the identifier of the source from which packet 113a originated (i.e., source Sc=3). Arbitration circuit 111 then selects packet 113b as the next packet to transmit (because this packet has the next highest source identifier, i.e., Sc=7). After transmitting packet 113b, circuit 111 updates the identifier in register 115 to the identifier of the source from which packet 113b originated (i.e., source Sc=7). Since none of packets 113c and 114a currently at the head of buffers 105 and 112 have a source number higher than the source number in register 115 (Sc=7), circuit 111 selects one of the packets with the lowest source number, i.e., packet 114a, which has source number Sc=1. After transmitting packet 114a, the packet transmission order continues as follows: packet 114b, packet 113c, packet 114c, packet 113d, and packet 114d.

[0065] 1B illustrates an embodiment in which first input buffer 105 contains packets originating from multiple sources. In this case, circuit 111 examines the source identifiers of packets held in buffer 105 in addition to the source identifiers of packets held in buffer 112. However, in some embodiments, buffer 105 may be dedicated to holding packets originating from a single source. In such embodiments, it may be unnecessary for circuit 111 to individually examine the source identifiers of packets sent from buffer 105 for arbitration purposes.

[0066] 1C, which illustrates an embodiment in which buffer 105 contains data packets 116a-d originating from a single source component rather than from multiple source components. In this case, because buffer 105 is dedicated to holding packets from a single source component, circuit 111 does not need to analyze the source identifiers in packets 116a-d when selecting between buffers 105, 112. Circuit 111 has access to the identifier associated with this single source component to use for arbitration.

[0067] 1C, circuit 111 examines the source identifier in packet 117a at the head of buffer 112. Circuit 111 compares this examined source identifier with the last source identifier from register 115 and the source identifier associated with buffer 105. Circuit 111 determines which of the examined source identifier and the source identifier associated with buffer 105 is next highest after the last source identifier (if any) and causes one of packets 116a, 117a having this next highest source identifier to be transmitted next on output port 106. If neither the examined source identifier nor the source identifier associated with buffer 105 is higher than the last source identifier, circuit 111 causes one of packets 116a, 117a having the lowest source identifier to be transmitted next on output port 106.

[0068] See FIG. 1D, which illustrates how a scheme for selecting a buffer from which to send data based on the source of the outgoing data (i.e., source arbitration) can be combined with a scheme for selecting a buffer based on the destination to which the data is sent (i.e., destination arbitration).

[0069] 1A and 1B, the second input buffer 112 can be one of a plurality of buffers 107 for storing data received on the second input port 102, each of which stores data for transmission to a different destination. In this case, the circuit 111 arbitrates data transmission from different sources as well as data transmission to different destinations. The step of selecting a destination to which a data packet should be sent occurs before the step of selecting a source from which the data is sent. Once the circuit 111 has determined the destination to which the data should next be sent, the circuit 111 selects between the buffers 105, 107 that have at their heads a data packet for transmission to that destination. The selection between the buffers 105, 107 is performed based on a source identifier in the packet at the head of each of the buffers 105, 107.

[0070] To perform destination-based arbitration, hardware module 100 includes a last destination register 118 that indicates the destination last selected for transmitting a data packet on output interface 106. Circuit 111 performs a round-robin among the destinations by selecting the next highest valid destination identifier following the destination identifier stored in register 118, if one exists. If the destination identifier stored in register 118 is higher than any of the destinations to which data can be transmitted, circuit 111 selects the lowest destination identifier.

[0071] When circuit 111 selects a destination identifier, if there is no data to send to that destination (either at the head of the first input buffer 105 or at the head of the further input buffer 107 associated with that destination), circuit 111 selects the next destination without causing any data to be sent at output port 106 to that destination. If there is data to send to that destination at the head of only one of buffers 105, 107, the circuit sends data from that buffer that has data to send to that destination without selecting between buffers 105, 107 based on the source from which that data originated. If there is data to send to the selected destination at the head of two of buffers 105, 107, circuit 111 selects between the two buffers based on the source component from which the data in the two buffers originated.

[0072] Last source register 115 stores, for each destination to which data is sent on output port 106, the source identifier of the originating source component that last sent data to that destination. Arbitration between sources is therefore performed on a destination-by-destination basis. In this manner, hardware module 100 supplies data from various sources to a given destination with fairness among each of the sources.

[0073] The example of Figure 1D shows a second input buffer 112 and a third input buffer 119, both of which are buffers 107 for receiving data via the second input port 102. While Figure 1D shows data for dispatching to only two destinations, it will be understood that in other embodiments there may be data packets for more than two destinations.

[0074] As an example, assume that circuit 111 initially transmits a data packet to destination Ds=0. Circuit 111 examines packet 120a at the head of first input buffer 105. Because first packet 120a includes an identifier for destination Ds=1, circuit 111 does not select between buffers 105, 112 based on a source identifier. In response to determining that the destination identifier (Ds=0) in packet 120a does not match the selected destination identifier (Ds=0), circuit 111 causes packet 121a to be transmitted on output port 106.

[0075] After transmitting packet 121a, circuit 111 updates the last source identifier for Ds=0 in register 115 to the source identifier of packet 121a (i.e., Sc=1). Circuit 111 also updates last destination register 118 to Ds=0. Circuit 111 then selects the next destination to send data to, Ds=1. Circuit 111 examines packet 120a at the head of first input buffer 105. Because this packet 120a has a destination identifier of Ds=1, circuit 111 selects whether to send packet 120a from buffer 105 or packet 122a from buffer 119. This selection is made depending on the source component from which these packets originated. Circuit 111 obtains from register 115 the identifier of the last source component that sent a data packet to Ds=1. 1B and 1C, circuit 111 applies a round-robin scheme among the sources and transmits packets 120a, 122a having the next highest source number after the source identifier obtained from register 115 (or the lowest, if the source identifier from register 115 is higher than the source identifier of packets 120a, 122a). Assume that the last source identifier for Ds=1 obtained from register 115 is Sc=1. In this case, the next highest source identifier of packets 120a, 122a is Sc=3, and therefore packet 120a from buffer 105 is selected for transmission on output port 106.

[0076] After transmitting packet 120a, circuit 111 updates the last source identifier for Ds=1 in register 115 to Sc=3. Circuit 111 also updates the last destination identifier in register 118 to Ds=1. Circuit 111 then selects Ds=0 as the next destination to which data should be transmitted by circuit 111. Circuit 111 then selects whether to transmit packet 120b from buffer 105 or packet 121b from buffer 112. Because the last source component for Ds=0 is Sc=1, circuit 111 selects packet 120b, 121b, which has the next highest source identifier (i.e., Sc=3), of the two packets 120b, 121b, for transmission on port 106.

[0077] Circuit 111 continues to transmit packets 120a-d, 121a-d, 122a-d according to the described scheme for arbitrating between sources and destinations.

[0078] 1B-1D illustrate an example in which circuit 111 selects the next source for transmitting data based on a stored indication of the last source that transmitted data on port 106. In some embodiments, circuit 111 can rely on different types of stored information to select a buffer from which to transmit data. In particular, in some embodiments, hardware module 100 can store, for each source (i.e., remote source) from which data is transmitted from buffer 107, an indication of the last destination to which data originating from that particular source was transmitted. These indications are used to weight each of buffers 107 so that a weighted selection can be performed between one of buffers 107 containing data for a particular destination and first input buffer 105.

[0079] 1E, which illustrates an embodiment in which a weighted selection is performed between buffer 105 and a selected one of buffers 112, 119. In this example, register 130 is part of hardware module 100 and stores, for each remote source 104 (i.e., the source from which data is received at port 102), an indication of the destination to which data originating from that source was last transmitted at port 106. Because data from remote source 104 is stored in buffer 107, each such indication corresponds to an indication of from which of buffers 107 data originating from the respective source was last transmitted. Circuit 111 selects a destination for transmitting data according to a round-robin arbitration scheme. This can be done using register 118, as described above with respect to FIG. 1D. Once a destination is selected, circuit 111 selects among the buffers (including one of buffers 107, which may include buffer 105) that have data to transmit to that destination. Selection between these buffers is performed by applying a weighted round robin between buffer 105 and one of the buffers 107 associated with the destination, with buffer 105 being assigned a weight of 1 and one of the buffers 107 associated with the destination being assigned a variable weight. The variable weight assigned to one of the buffers 107 is given by the number of remote sources 104 that last sent data to the destination associated with that buffer. This number of remote sources 104 can be derived from an indication held in register 130. Specifically, for each destination, the number of remote sources 104 is given by the number of last destination indications in buffer 130 that point to that destination. It is noted that in an embodiment, the minimum weight for each of buffers 107 is 1, so if the source that last sent outgoing data from a particular one of buffers 107 is not indicated in register 130, circuit 111 applies a weight of 1 to that buffer.

[0080] To apply a weighted round robin among buffers having data for transmission to a particular destination, registers 131 containing state information are provided within module 100. Register 131 stores state information that identifies, for each destination, which of the buffers (i.e., first input buffer 105 or one of buffers 107) will next provide data to that destination on port 106. When circuit 111 transmits a first data packet to the first destination on port 106, circuit 111 selects a further destination to transmit the data to. When circuit 111 again selects the first destination to transmit data to, circuit 111 uses the stored state information for the first destination to select from which buffer the next data packet should be transmitted to the first destination on port 106.

[0081] For a particular destination, if the buffer indicated by the associated state information as the next to send data is one of buffers 107, circuit 111 also stores as part of the destination's state information a count of the number of data packets being successively sent from the currently selected one of buffers 107 to the respective destination. After sending a data packet from one of buffers 107 associated with a destination and updating its associated count in register 131, circuit 111 compares the associated count for one of buffers 107 with the weight for that one of buffers 107. If circuit 111 determines that the count for one of buffers 107 matches its associated weight, circuit 111 updates the destination's state information to indicate buffer 105 as the next buffer from which data should be sent to the destination.

[0082] 1E , assume that data from source Sc=1 was last transmitted from buffer 112, while data from sources Sc=5 and Sc=9 was last transmitted from buffer 119. These indications are held in register 130. Because data originating from two sources (i.e., Sc=5 and Sc=9) was last transmitted from buffer 119, buffer 119 (and its associated destination) has a weight of 2 (implied by the indication in register 130). On the other hand, because data originating from one source (i.e., Sc=1) was last transmitted from buffer 112, buffer 112 has a weight of 1 (implied by the indication in register 130). Assume further that for destination Ds=0, the associated status information for Ds=0 in register 131 indicates that a data packet should be transmitted next from buffer 112, and for destination Ds=1, the associated status information for Ds=1 in register 131 indicates that a data packet should be transmitted next from buffer 119.

[0083] Circuit 111 examines packet 120a at the head of buffer 105 and determines that packet 120a is for transmission to Ds=1. If circuit 111 is next configured to transmit a data packet to Ds=1, circuit 111 selects between buffer 105 and buffer 119 for transmitting the data packet on port 106. Because the status information in register 131 indicates that the next data packet to be transmitted to destination Ds=1 on port 106 is from buffer 119, circuit 111 causes packet 122a at the head of buffer 119 to be transmitted on port 106. Circuit 111 then updates the status information in register 131 to indicate that one packet has been continuously transmitted from buffer 119. Circuit 111 also updates the last destination of source Sc=1, held in register 130, to indicate that data originating from Sc=1 was last transmitted to destination Ds=1 (i.e., from buffer 119).

[0084] Updating the last destination indication for source Sc=1 has the effect of changing the weight for buffer 119. Buffer 119 now has a weight of 3 because data originating from three sources (i.e., Sc=1, Sc=5, and Sc=9) is currently the last to be sent from buffer 119. For buffer 112, the last source from which data was sent from this buffer 112 is currently 0. As noted above, if a source of zero is the last from a buffer, it is implied that the buffer has a weight of 1. Buffer 112 is therefore indicated to have a weight of 1 by the indication in register 130.

[0085] After circuit 111 transmits packet 122a, circuit 111 selects Ds=0 as the next destination to which to send data. Because packet 120a at the head of buffer 105 is intended for transmission to Ds=1 rather than Ds=0, buffer 105 is excluded from the weighted selection performed by circuit 111. As a result, circuit 111 selects buffer 112 and transmits packet 121a at the head of this buffer 112 on port 106. Circuit 111 then updates register 130 to indicate that data originating from Sc=1 was last transmitted to destination Ds=0 (i.e., from buffer 112). This update has the effect of again adjusting the weightings so that buffer 112 has a weight of 1, while buffer 119 has a weight of 2.

[0086] After transmitting packet 121a, one packet has been continuously transmitted from buffer 112 without the count being reset, so the count value is equal to the current weight for buffer 112. Because the count value is equal to the weight, circuit 111 updates the state information in register 131 to indicate that the next buffer with destination Ds=0 is buffer 105. Circuit 111 also resets the count value for buffer 112 to 0.

[0087] After circuit 111 transmits packet 121a, circuit 111 selects Ds=1 as the next destination to which to transmit data. Circuit 111 examines the status information for Ds=1 and determines that a packet was last transmitted from buffer 119 to this destination and that one packet (i.e., packet 122a) is being transmitted consecutively from buffer 119. Circuit 111 compares the number of consecutively transmitted packets (i.e., 1 in this case) with the weight implied by the indication in register 130 (i.e., 2 in this case). Because the number of consecutively transmitted packets is less than the weight, the result of the weighted selection by circuit 111 is to again select buffer 119 as the next buffer from which to transmit a data packet to Ds=1. Therefore, circuit 111 causes buffer 119 to transmit packet 122b on port 106. Circuit 111 updates the status information in register 131 to indicate that two packets are being transmitted consecutively from buffer 119. Since the count of packets transmitted from buffer 119 now equals the weight for buffer 119 (i.e., 2), circuit 111 resets the count and updates the status information in register 131 to indicate that the next buffer for Ds=1 is buffer 105.

[0088] When circuit 111 transmits packet 122b from buffer 119, circuit 111 selects Ds=0 as the next destination to transmit to. Buffer 105 is indicated in the state information as being the next buffer for destination Ds=0. Circuit 111 examines packet 120a at the head of buffer 105, and in response to determining that this packet is not for transmission to destination Ds=0, selects buffer 112 instead and transmits packet 121b from buffer 112 on port 106. Because the weight of buffer 112 is simply 1, in response to transmitting the single packet from buffer 112, circuit 111 causes the state information for Ds=0 to indicate that buffer 105 is the next buffer for transmission to Ds=0.

[0089] After transmitting packet 121b, circuit 111 selects Ds=1 as the next destination to send data to. In response to determining that the status information for Ds=1 indicates that the next buffer to send data to for Ds=1 is buffer 105, circuit 111 causes packet 120a, which is at the head of buffer 105, to be transmitted on port 106. After transmitting packet 120a, circuit 111 updates the status information to indicate buffer 119 as the next buffer for Ds=1.

[0090] After transmitting packet 120a, circuit 111 selects Ds=0 as the next destination to send data to. In response to determining that the status information for Ds=0 indicates that the next buffer to send data to for Ds=0 is buffer 105, circuit 111 causes packet 120b, which is at the head of buffer 105, to be transmitted on port 106. After transmitting packet 120b, circuit 111 updates the status information to indicate buffer 112 as the next buffer for Ds=0.

[0091] Circuit 111 continues to transmit packets from buffers 105, 112, 119 in the manner described.

[0092] 1A shows an example in which hardware module 100 has a single input buffer 105 for receiving data packets from one or more locally connected source components 103 via a single input port 101. In some embodiments, there may be multiple input buffers for receiving data from locally connected source components. In this case, the selection process performed to select the buffer to which data should be sent at output port 106 also includes selecting between those input buffers that hold data received from the locally connected source components.

[0093] 2A, which illustrates one embodiment of a hardware module 200 including a plurality of input buffers 201A-D for receiving data from source components 202A-D via input ports xi0-3. Each of source components 202A-D is locally connected to hardware module 200 and configured to provide data packets to an associated one of input buffers 201A-D at an associated one of input ports xi0-3. Each input buffer 201A-D is dedicated to holding data originating from its associated source component 202A-D. Input buffer 201A may be labeled as a first input buffer in this embodiment.

[0094] If there are four input buffers 202A-D, selecting between buffers from which to send a data packet based on the source of the data packet at the head of the buffer may involve selecting between up to five buffers, where the selection is between one of buffers 107 holding data for delivery to a given destination and any of buffers 201A-D having at its head a data packet also for delivery to this same destination. Circuit 111 arbitrates based on the originating source component to determine from which of these buffers data should be sent on port 104.

[0095] To illustrate how this process is performed when selecting between three buffers, reference is made to Figure 2B, which shows the use of circuit 111 to select between first input buffer 201A, second input buffer 112, and further input buffer 201B. As with the binary selection between buffers described above with respect to Figures 1B-1D, this ternary selection is performed based on the source from which the data at the head of each buffer 201A, 201B, 112 originated. Circuit 111 applies a round robin to select between the three buffers 201A, 201B, 112 depending on the source from which the data packet at the head of buffer 201A, 201B, 112 originated and depending on the last source identifier in register 115.

[0096] In the example of Figure 2B, a first input buffer 201A stores data packets originating from source Sc=1 and a further input buffer 201B stores data packets originating from source Sc=3, while a second input buffer 112 stores data packets originating from sources Sc=2 and Sc=5.

[0097] Assume that register 115 identifies the source component of the data packet last transmitted from one of buffers 201A, 201B, 112 as having source Sc=1. Circuit 111 examines the source identifier of packet 125a at the head of buffer 112 and compares it with the source identifiers of packets 123a and 124a. Packet 123a has a source identifier of source Sc=1, packet 124a has a source identifier of source Sc=3, and packet 125a has a source identifier of source Sc=2. Because packet 125a's source identifier (Sc=2) is the next highest source identifier, arbitration circuit 111 causes packet 125a to be transmitted on output port 106. Arbitration circuit 111 then updates the identifier in register 115 to the identifier of the source from which packet 125a originated (i.e., Sc=2). Arbitration circuit 111 then selects packet 124a as the next packet to transmit (because this packet has the next highest source identifier, i.e., Sc=3). After transmitting this packet 124a, arbitration circuit 111 updates the identifier in register 115 to the identifier of the source from which packet 124a originated (i.e., Sc=3). Arbitration circuit 111 then selects packet 125b as the next packet to transmit (because this packet has the next highest source identifier, i.e., Sc=5). After transmitting this packet 125b, none of the three packets 123a, 124b, 125c currently at the head of buffers 201A, 201B, 112 have a higher source identifier, so circuit 111 selects one of the packets, i.e., packet 123a with the lowest source number, i.e., Sc=1, as the next packet to transmit.

[0098] 2B shows an embodiment in which input buffers 201A, 201B contain only data packets 123a-d, 124a-d for transmission to a single destination, i.e., Ds=0. However, in some embodiments, buffers 201A, 201B may contain data packets for delivery to multiple destinations, and thus selection based on the originating source component may include only buffers 201A, 201B that have the next data packet for delivery to that particular destination.

[0099] 2C, which illustrates an embodiment in which circuit 111 arbitrates between buffers 201A, 201B, 112, 119 based on the destination to which packets in the buffers should be sent and based on the source component from which the packets originated. In this case, circuit 111 selects a subset of buffers 201A, 201B, 112, 119 that have data packets at their heads for transmission to a given destination. Having selected a subset of buffers 201A, 201B, 112, 119, circuit 111 selects one of this subset of buffers based on the source component from which the next packet to be transmitted from each of those buffers originates.

[0100] Similar to that described above with respect to Figure 1D, in the embodiment of Figure 2C, to allow for round robin between destinations, there is a last destination register 118 that indicates the last destination to which a data packet was sent. Additionally, for each destination, there is also a last source register 115 that identifies which originating source component last sent data to that destination.

[0101] As an example, circuit 111 may be configured to first send data packets to destination Ds=0. Circuit 111 examines packet 126a at the head of first input buffer 201A. Because packet 126a at the head of buffer 201A contains an identifier for destination Ds=0, circuit 111 includes first input buffer 201A in the subset of buffers for which selection is based on originating source. Because packet 127a at the head of buffer 201B contains an identifier for destination Ds=0, circuit 111 also includes buffer 201B in the subset of buffers for which selection is based on originating source. Buffer 112 is dedicated to holding packets for transmission to destination Ds=0, and therefore buffer 112 is included in the subset of buffers for which selection is based on originating source.

[0102] Once circuit 111 has determined subset 201A, 201B, 112, it selects one of the buffers in this subset according to the source identifiers (Sc=1, Sc=3, Sc=5) in packets 126a, 127a, 128a at the head of these buffers 201A, 201B, 112. Suppose last source register 115 indicates Sc=1 as the last originating source associated with data sent on port 106 to destination Ds=0. In this case, when choosing between buffers 201A, 201B, 112, circuit 111 selects buffer 201B because packet 127a has the next highest source identifier (Sc=3) at its head. Circuit 111 then causes packet 127a to be transmitted on port 106.

[0103] After transmitting packet 127a, circuit 111 updates the last source identifier of Ds=0 in register 115 to the source identifier of packet 127a (i.e., Sc=3). Circuit 111 also updates last destination register 118 to Ds=0. Circuit 111 then selects a subset of buffers whose head packets are for transmission to Ds=1. Circuit 111 examines packet 126a at the head of first input buffer 201A. Because packet 126a has a destination identifier of Ds=0, circuit 111 does not include buffer 201A in the subset of buffers. Circuit 111 examines packet 127b at the head of second input buffer 201B. Because packet 127b has a destination identifier of Ds=0, circuit 111 does not include buffer 201B in the subset of buffers. Buffer 119 is dedicated to holding packets for transmission to destination Ds=1, and therefore buffer 119 is included in the subset of buffers selected based on the originating source. Because only one buffer (i.e., buffer 119) has data at its head to transmit to Ds=1, circuit 111 transmits data packet 129a from buffer 119 without arbitrating based on the source component. After transmitting packet 129a, circuit 111 updates the last source identifier for Ds=1 in register 115 to the source identifier of packet 129a (i.e., Sc=2). Circuit 111 also updates last destination register 118 to Ds=1.

[0104] Circuit 111 is then configured to send packets to Ds=0 again. Circuit 111 includes buffers 201A, 201B, and 112 in the subset of buffers because those buffers have packets 126a, 127b, and 128a, respectively, at their beginning for sending to Ds=0. This selection between buffers 201A, 201B, and 112 is made depending on the source component from which those packets 126a, 127b, and 128a originate. Circuit 111 obtains from register 115 the identifier of the last source component that sent a data packet to Ds=0. Circuit 111 applies a round-robin scheme between sources as described and transmits the packet 126a, 127b, and 128a with the next highest source number after the source identifier obtained from register 115 for destination Ds=1 (or the lowest, if the source identifier from register 115 is higher than the source identifier of packets 120a and 122a). In this case, the next highest source identifier of packets 126 a , 127 b , and 128 a is Sc=5, and therefore packet 128 a from buffer 112 is selected for dispatch at output port 106 .

[0105] Circuit 111 continues to transmit packets 120a-d, 121a-d, 122a-d according to the described scheme for arbitration between sources and destinations.

[0106] As discussed above with respect to FIG. 1E, selection between buffers can alternatively be performed using weights associated with each of buffers 107, where the weights are based on the number of sources that last transmitted from each of buffers 107. These weights are used to perform a weighted round-robin for buffer selection. See FIG. 2D, which illustrates an example in which such a scheme can be applied to perform weighted round-robin arbitration between a selected one of buffers 107 and multiple buffers 201A, 201B. In this example, similar to the example of FIG. 1E, register 131 stores state information for each destination, where the state information indicates the next buffer selected for transmission to that destination. In the example of FIG. 2D, the next buffer indicated in the state information for Ds=0 would be one of buffer 112, buffer 201A, or buffer 201B, whereas the next buffer indicated in the state information for Ds=1 would be one of buffer 119, buffer 201A, or buffer 201B. The state information further specifies, for each of buffers 112, 119, a count of the number of packets transmitted consecutively from that buffer before the weight of that buffer was reached. The count for buffer 112 is updated by circuit 111 when a packet is sent from buffer 112, and the count for buffer 119 is updated by circuit 111 when a packet is sent from buffer 119. When the count for either buffer 112, 119 reaches its weight, the count for that buffer is reset and the next buffer indication in the state is updated to identify buffer 201A.

[0107] 2D , assume that data from source Sc=5 was last transmitted from buffer 112, while data from sources Sc=2 and Sc=9 was last transmitted from buffer 119. These indications are held in register 130. Because data originating from two sources (i.e., Sc=2 and Sc=9) was last transmitted from buffer 119, buffer 119 (and its associated destination) has a weight of 2 (implied by the indication in register 130). On the other hand, because data originating from one source (i.e., Sc=5) was last transmitted from buffer 112, buffer 112 (and its associated destination) has a weight of 1 (implied by the indication in register 130). Assume further that for Ds=0, the associated status information in register 131 indicates that a data packet should be transmitted next from buffer 112, whereas for Ds=1, the associated status information in register 131 indicates that a data packet should be transmitted next from buffer 119.

[0108] If circuit 111 determines that the next destination for sending data on port 106 is Ds=0, circuit 111 transmits data packet 128a from buffer 112. After transmitting this packet 128a, circuit 111 determines that the count of consecutively transmitted packets from buffer 112 is equal to the weight of buffer 112 (i.e., 1). In response to this determination, circuit 111 updates the Ds=0 status information in register 131 to indicate buffer 201A as the next buffer for Ds=0. Circuit 111 also resets the count for buffer 112 to zero.

[0109] Following the transmission of packet 128a, circuit 111 selects Ds=1 as the next destination to send data to on port 106. Circuit 111 transmits packet 129a from buffer 119 on port 106. After transmitting this packet 129a, circuit 111 sets the count of consecutively transmitted packets from buffer 119 equal to 1.

[0110] After transmitting packet 129a, circuit 111 selects Ds=0 as the next destination to send data to on port 106. In response to determining that packet 126a at the head of buffer 201A is for transmission to Ds=0, circuit 111 causes packet 126a to be transmitted on port 106. Circuit 111 then updates the state information associated with Ds=0 to indicate buffer 201B as the next buffer after Ds=0.

[0111] After transmitting packet 126a, circuit 111 selects Ds=1 as the next destination to send data to on port 106. Circuit 111 transmits packet 129b from buffer 119 on port 106. After transmitting packet 129b, circuit 111 determines that the count of packets transmitted from buffer 119 is equal to the weight for buffer 119 (i.e., 2), and responsively resets the count for buffer 119 to zero and updates the state information for Ds=1 to indicate buffer 201A as the next buffer to transmit to Ds=1.

[0112] After transmitting packet 129b, circuit 111 selects Ds=0 as the next destination to send data to on port 106. In response to determining that buffer 201B is indicated in Ds=0's state information as being the next packet to send to Ds=0 on port 106, circuit 111 transmits packet 127a on port 106. Circuit 111 then updates Ds=0's state information to indicate buffer 112 as the next buffer to send data to Ds=0.

[0113] After transmitting packet 127a, circuit 111 selects Ds=1 as the next destination to send data to on port 106. The status information for Ds=1 indicates buffer 201A as the next buffer to send data to on port 106. In response to determining that packet 126b at the head of buffer 201A is for transmission to Ds=1, circuit 111 causes packet 126b to be transmitted on port 106 toward Ds=1.

[0114] In this manner, circuit 111 continues arbitrating between sources for transmitting data by applying separate weighted round robins to the first set of buffers (i.e., buffers 201A, 201B, 112) and the second set of buffers (i.e., buffers 201A, 201B, 119).

[0115] In the examples described above with respect to Figures 1B-1E and 2B-2D, several registers 115, 118, 130, 131 are described for storing instructions used to arbitrate between different destinations and / or different sources. In some embodiments, one or more of these registers 115, 118, 130, 131 may be combined into a single unified storage. Alternatively, one or more of the described registers 115, 118, 130, 131 may be subdivided into multiple registers. What is important is the information stored by the hardware modules 100, 200 that enables the selection of the appropriate buffer from which to send data.

[0116] 2B, 2C, and 2D, selection between only two local input buffers 201A, 201B is described, but the same selection scheme may be applied for use with three or more such buffers for receiving data from locally connected source components. Furthermore, while arbitration between only two destinations is described with respect to FIGS. 1D, 1E, 2C, and 2D, arbitration between three or more destinations may be performed.

[0117] 1A-2D, the selection between buffers (e.g., first input buffer 105 / 201A and second input buffer 112) was described as being based on the source component of the data packet at the head of those buffers. If one of those buffers is empty, the empty buffer is excluded from the selection process.

[0118] In some embodiments, a hardware module may have multiple output ports through which locally connected source components can send data to destinations, in which case the hardware (i.e., input buffers 105, 107 and arbitration circuit 111) is replicated for each additional output port.

[0119] 3, which illustrates an example of a hardware module 300 with multiple output ports 106a, 106b to which data packets received from one or more source components 103 may be transmitted from the hardware module 300. One set of destinations may be accessible on port 106a, while a different set of destinations may be accessible on port 106b.

[0120] This exemplary hardware module 300 includes some of the same components as those described above with respect to FIG. 1A. Specifically, hardware module 300 includes a first input port 101, an output port 110, and an output buffer 109. This hardware module also includes two input buffers 105a and 105b, which both function like the first input buffer 105 described above. Buffer 105a is associated with output port 106a, while buffer 105b is associated with output port 106b. Circuitry in hardware module 300 sorts data packets received on port 101 from one or more source components 103 into buffer 105a or 105b according to a destination identifier in the packet's header. In particular, packets for delivery to a destination accessible on output port 106b are stored in buffer 105b, while packets accessible on output port 106a are stored in buffer 105a.

[0121] The hardware module includes a pair of buffers 107a for receiving data packets via input port 102a and a pair of buffers 107b for receiving data packets via input port 102b. Each of these pairs of buffers 107a, 107b has the same features and functionality as buffer 107 described above with respect to Figures 1A-2D. Buffer 107a holds data packets for transmission on output port 106b or for delivery to locally connected source component 103 via output port 110. Buffer 107b holds data packets for delivery on output port 106a or for transmission to locally connected source component 103 via output port 110.

[0122] Hardware module 300 includes arbitration circuit 111a and arbitration circuit 111b, each having the same features and functionality as arbitration circuit 111 described above. Circuits 111a and 111b can arbitrate between buffers based on source and destination in the manner described with respect to any of the embodiments described above with respect to Figures 1B-1E and 2B-2D. Circuit 111a selects between buffers 105a and 107a for transmitting data over interface 106b, while circuit 111a selects between buffers 105b and 107b for transmitting data over interface 106a.

[0123] 3 shows only one pair of input buffers 105, 105b for receiving data packets from source component 103 via a single input port 101, in some embodiments, hardware module 300 may include multiple input ports and input buffers, as shown in FIG. 2A. In this case, each input port may be associated with a pair of input buffers, and for each pair of input buffers, one buffer is associated with circuit 111a and the other buffer is associated with circuit 111b. In this manner, each circuit 111a, 111b selects among transmitting data packets from three or more buffers in the manner described with respect to FIGS. 2A-2D.

[0124] Each of the exemplary hardware modules 100, 200, and 300 described above is configured to transmit a single data packet on its output port once per clock cycle. Within each single clock cycle, circuit 111 of hardware modules 100 and 200 selects a buffer (based on the destination or source identifier of the data packet at the head of that buffer) and transmits the packet at the head of that buffer. In module 300, within each clock cycle, circuit 111a selects a buffer and transmits a packet from that buffer, and circuit 111b selects a further buffer and transmits a further packet from that buffer.

[0125] When data is received from multiple different sources on at least one port, the hardware modules 100, 200, 300 described above have the advantage of providing fairness among sources sending data packets to a particular destination. In some embodiments, a series of hardware modules are provided in an interconnect, with each hardware module functioning as a node in the interconnect.

[0126] Refer to FIG. 4, which illustrates an interconnect 410 including multiple hardware modules 400a-f. Each of the hardware modules 400a-f may be one of the exemplary hardware modules 100, 200, 300 described above. Each of the hardware modules 400a-f is a node within the interconnect 410 and is operable to forward packets to its neighboring nodes 400a-f of the interconnect 410. Packets forwarded by each node 400a-f are received from the node's locally connected components or from its neighboring nodes 400a-f of the interconnect 410. In FIG. 4, the locally connected components are represented as b_(n-1), b_(n-2),...b_1, and b_0. Each of these locally connected components is connected to one of the nodes 400a-e.

[0127] It will be appreciated that by employing the above-described scheme of arbitrating based on the source of the data, fairness is achieved among the various sources (b_(n-1) through b_0) when providing data to particular destinations accessible on interconnect 410.

[0128] In FIG. 4, an exemplary destination is labeled "B(VC X)." B(VC X) is a component connected to node 400f. Assume that each of sources b_(n-1) to b_0 targets destination B(VC X) by sending a data packet to B(VC X). It is clear that each of nodes 400b-400e receives data packets originating from its locally connected source component and from a source component connected to at least one of nodes 400a-d. A scheme can be applied in which each node 400b-e arbitrates by applying round-robin between a single local input buffer and additional input buffers for receiving data from the other nodes 400a-e. This provides fairness between buffers but not between sources. If a simple round-robin between buffers is applied, the source closest to destination B(VC X) will be prioritized. Node 400e will divide its available bandwidth for transmitting to B(VC X) between b_0 (from which data is received on node 400e's local input port and stored in node 400e's local input buffer) and the combination of other sources b_(n-1) to b_1 (from which data is received on node 400e's second input port and stored in the same one of node 400e's further buffers 107). Because remote sources b_(n-1) to b_1 share the same buffer, providing fairness between the buffers will result in a bias in favor of source b_0, with more bandwidth being allocated to this source. Similarly, node 400d will divide its available bandwidth for transmitting data to B(VC X) between b_1 and the combination of sources b_(n-1) to b_2, thereby allocating more bandwidth to b_1 than to the other sources b_(n-1) to b_2.

[0129] Thus, it is understood that the bandwidth for delivering data from a particular source to B(VC X) decreases with the distance from B(VC X) (i.e., the number of nodes 400). Furthermore, in some situations, the bandwidth available for nodes 400a-f to output data may exceed the bandwidth available for sources b_(n-1) to b_0 to output data to their own nodes 400a-e. In this case, if fairness between sources is not provided, the total time it takes for each of sources b_(n-1) to b_0 to transmit a set of data to destination B(VC X) may be long. Suppose each node 400a-e can output data to destination B(VC X) at 96 Gbps, but each of sources b_(n-1) to b_0 can only output data to its own individual node 400a-e at 32 Gbps. Given the number of sources (four or more), if each of sources b_(n-1) to b_0 attempts to transmit data at its maximum rate, the 96 Gbps link to B(VC X) will saturate. Sources farther from B(VC X) (e.g., b_n-1) may stall, while sources closer to B(VC X) (e.g., b_0) may transmit at a higher rate and use their full 32 Gbps. If sources b_(n-1) through b_0 each have an equal set of data to send, the sources closer to B(VC X) (e.g., b_0) will finish first, and the sources farther from B(VC X) (e.g., b_(n-1)) will finish last. In this case, the 96 Gbps link to B(VC X) will no longer saturate once the closer sources finish sending their data, so that only the farthest sources are still sending data, implying wasted bandwidth and an increase in the total time required for transmission from all of sources b_(n-1) through b_0.

[0130] Thus, according to an embodiment, each of nodes 400a-f selects between its local input buffer and an additional buffer (for receiving from the other nodes 400a-f) by arbitrating among the sources as described. In this way, fairness is ensured among each of sources b_(n-1) to b_0 for transmission to B(VC X). For example, assuming each of sources b_(n-1) to b_0 is transmitting at the same rate, node 400f transmits data to B(VC X) at equal rates from each of b_(n-1) to b_0. Similarly, node 400e transmits data to node 400f at equal rates from each of b_(n-1) to b_1. In an embodiment, the bandwidth (e.g., 96 Gbps) for node 400f to transmit data to B(VC X) (e.g., via output buffer 109) is greater than the bandwidth (e.g., 32 Gbps) at which each of sources b_(n-1) to b_0 can output data. If sources b_(n-1) through b_0 each have an equal set of data to send to B(VC X), the fairness provided among the sources ensures that the link bandwidth between node 400f and B(VC X) remains saturated at 96 Gbps throughout the transmission of the data, thus reducing the total time to transmit the data.

[0131] 4 shows that embodiments are particularly advantageous when multiple instances of hardware modules 100, 200, 300 are provided together as nodes 400 of an interconnect and used in a configuration where the aggregate transmit bandwidth of the sources exceeds the receive bandwidth of the destinations. In this case, by providing fairness between sources at each node 400 in the interconnect, the receive bandwidth is constantly saturated throughout the transfer of data, thus minimizing the total transfer time.

[0132] Reference is now made to FIG. 5, which illustrates a further exemplary embodiment of a system-on-chip 500 including an interconnect. In this example, the nodes 400 are shown as trunk nodes 400. Each of the trunk nodes 400 has two adjacent trunk nodes 400 that can send and receive data packets. The trunk nodes 400 may take the form of hardware modules 300 shown in FIG. 3. The interconnect also includes a trunk extension unit 510 that functions as a node that routes traffic from one side of the chip to the other. The trunk extension unit 510 functions as a simplified version of the trunk node 400 that does not connect to any SoC components (other than the trunk nodes 400).

[0133] The interconnect is configured as a ring around which packets can circulate. Packets can travel in a clockwise or counterclockwise direction on the interconnect. Packets have headers that contain information such as destination and source identifiers, allowing nodes 400 to store them in the appropriate virtual channel buffers 107 and select and transmit them by arbitrating based on the destination and source identifiers.

[0134] The chip includes processing units 2, each containing multiple processor tiles 4. The interconnect forms a ring path around the processing units 2 and is used to transmit data packets to and from the tiles 4 of the processing units 2. The interconnect also transmits data packets to and from other SoC components of the chip. The interconnect is used to transmit data plane traffic (e.g., application instructions and application data for processing by tiles 4). The interconnect carries this data plane traffic between tiles 4 and a host system or between tiles 4 and directly connected external memory. Each tile 4 can exchange data packets with a trunk node via an associated switching block 520. The switching block 520 converts packets between the ELink packet format, suitable for transmission around the interconnect between nodes 400, and the TLink packet format used for packet transmission between tiles 4 and the switching block 520.

[0135] Various components are shown connected to interconnect node 400. One example of such a component is a PCI complex for interfacing with a host or with another chip. The PCIe complex receives packets from its associated node 400 and provides them to the host or another chip. Such packets may be read or write request packets sent by a tile 4 on a chip to read or write data from or to a host, or write packets sent by a tile 4 on another chip to write data to memory in a tile 4 on an attached chip. The PCIe complex receives packets from a host or another chip 500 and converts them into Elink packet format for circulation on the interconnect to provide to another component accessible on the interconnect. A PCIe complex can thus act as both a connected source and a connected destination for its associated trunk node 400.

[0136] A further component accessible over the interconnect is a DDR memory controller for reading and writing data to on-chip DDR memory, which is accessible over the interconnect to tile 4.

[0137] A further component accessible on the interconnect is an Ethernet port controller for transmitting and receiving data over an Ethernet network.

[0138] To transfer data between trunk nodes 400, multiple virtual channels are defined bidirectionally around the ring interconnect. Each of these virtual channels is associated with one of the buffers 107a, 107b. The virtual channels are defined to prevent HOL blocking. A virtual channel on the interconnect serves exactly one of the components accessible on the ring. Each virtual channel has its own dedicated buffer at each node 400. A node forwards received packets based on the occupancy of the virtual channel's buffer at the upstream node 400. Each node 400 provides credit signals for each virtual channel to its neighbors, and these credit signals indicate the occupancy of each virtual channel's buffer. In this way, the node 400 can backpressure the interconnect on a per-virtual-channel basis.

[0139] In the above description, operations have been described as being performed by hardware modules 100 / 200 / 300 / 400. These operations described as being performed by hardware modules may be performed by any suitable circuitry (including circuits 111, 111a, 111b, or other circuitry) of modules 100 / 200 / 300 / 400. This circuitry may take the form of special-purpose processing circuitry such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), or may take the form of a processor configured to execute computer-readable instructions to perform the described operations.

[0140] Please refer to FIG. 6, which illustrates a method 600 according to an embodiment of the present application.

[0141] At S610, a first input port receives a first set of data packets originating from a first subset of source components and provides them to a first input buffer, the first subset consisting of one or more of the plurality of source components.

[0142] At S620, a second input port receives a second set of data packets originating from a second subset of source components and provides them to a second input buffer. The second subset of source components includes two or more source components. The first subset is configured with a different number of source components than the second subset.

[0143] At S630, the processing circuit examines one or more second source identifiers in the one or more sets of second data packets to determine which of a plurality of second source components each of the one or more sets of second data packets originated from.

[0144] At S640, the processing circuit selects between the first input buffer and the second input buffer as a source of data to the first destination according to one or more second source identifiers, and arbitrates among multiple source components for transmitting data to the first destination.

[0145] It will be understood that the embodiments have been described by way of example only.

Claims

1. a hardware module for receiving data from a plurality of source components and forwarding at least a portion of the data to a first destination, the hardware module comprising: a first input port configured to receive a first set of data packets comprising data packets originating from a first subset of the source components and to provide the first set of data packets to a first input buffer, the first subset consisting of one or more of the plurality of source components; a second input port configured to receive a second set of data packets comprising data packets originating from a second subset of the source components and to provide the second set of data packets to a second input buffer, the second subset of source components comprising two or more source components, and the first subset consisting of a different number of source components relative to the second subset; an output port for outputting the first set of data packets and the second set of data packets, the output port being configured to output data to a plurality of destinations including the first destination, some of the data packets included in the first set of data packets being intended for transmission to different ones of the plurality of destinations; examining a destination identifier in one of the data packets included in the first set of data packets that is located at the top of the first input buffer; examining one or more source identifiers in one or more data packets of the second set of data packets to determine which of the second subset of source components each of the one or more data packets originated from; in response to determining that the destination identifier identifies the first destination, selecting between the first input buffer and the second input buffer as a source of data to the first destination according to the one or more source identifiers, and arbitrating among the plurality of source components for transmitting data to the first destination. a processing circuit configured to A hardware module including:

2. the processing circuit is configured to transmit, when selecting the second input buffer, a second data packet located at the head of the second input buffer among the data packets included in the second data packet set; 2. The hardware module of claim 1, wherein the one or more source identifiers are source identifiers of data packets included in the second set of data packets that were transmitted from the second input buffer before transmitting the second data packets.

3. including storage configured to store a first indicator indicating the number of the source components from which data was last transmitted from the second input buffer; 2. The hardware module of claim 1, wherein selecting between the first input buffer and the second input buffer comprises performing a weighted round robin between the first input buffer and the second input buffer, the second input buffer being weighted by the number of the source components to which data was last sent from the second input buffer.

4. 4. The hardware module of claim 3, further comprising: a set of input buffers configured to receive data originating from the second subset of source components, the set of input buffers including the second input buffer.

5. the storage includes, for each of the source components included in the second subset, a second indicator indicating to which of the set of input buffers data originating from the respective source component was last sent; 5. The hardware module of claim 4, wherein the first indicator is given by the number of source components in the second subset for which the second indicator indicates the second input buffer.

6. A first data packet located at the head of the first input buffer among the data packets included in the first set of data packets originates from a first source component of a first subset of source components identified by a first source identifier; examining the one or more source identifiers includes examining a second source identifier in a second data packet located at the head of the second input buffer among the data packets included in the second set of data packets; 2. The hardware module of claim 1, wherein the processing circuitry is configured to select the second data packet for transmission on the output port as a function of the second source identifier and the first source identifier.

7. The hardware module of claim 6 , wherein the processing circuitry is configured to determine the first source identifier by inspecting the first source identifier in the first data packet.

8. 2. The hardware module of claim 1, further comprising: a register that stores an identifier of one of the plurality of source components that most recently transmitted data to the first destination; and wherein selecting between the first input buffer and the second input buffer comprises sequentially selecting the next of the plurality of source components following the one of the plurality of source components identified in the register.

9. The hardware module of claim 1 , wherein selecting between the first input buffer and the second input buffer comprises applying a round-robin scheme among the plurality of source components.

10. 2. The hardware module of claim 1, wherein the first subset of source components consists of a single source component, and the first input buffer is dedicated to holding data packets originating from the single source component.

11. 2. The hardware module of claim 1, further comprising: a set of input buffers for storing data originating from the second subset of source components, each of the set of input buffers being dedicated to storing data for transmission to a different one of the plurality of destinations, and the second input buffer being dedicated to storing data for transmission to the first destination.

12. 12. The hardware module of claim 11, wherein each of the sets of input buffers is associated with a different virtual channel of an interconnect in which the hardware module serves as a node.

13. 2. The hardware module of claim 1, wherein the processing circuitry is configured to select a first destination as the next destination to which data is to be sent as part of an arbitration scheme for arbitrating among the multiple destinations for sending data on the output port before selecting between the first input buffer and the second input buffer.

14. 2. The hardware module of claim 1, wherein a first bandwidth available for transmitting at the output port exceeds a second bandwidth for receiving data at the first input port.

15. a node on an interconnect, the first input port being a local input port for receiving the first set of data packets from the first subset of the plurality of source components connected to the node; The hardware module of claim 1 , wherein the second input port is configured to receive the second set of data packets from a further node of the interconnect.

16. the second input port configured to receive data for transmission to at least one of the first subset of source components; 2. The hardware module of claim 1, wherein the processing circuitry is configured to provide this data for delivery to the at least one of the first subset of source components on a further output port of the hardware module.

17. 17. An apparatus comprising a hardware module according to any one of claims 1 to 16, the apparatus comprising a plurality of source components and a first destination.

18. 20. The apparatus of claim 17, comprising multiple instances of the hardware module connected together as nodes forming part of an interconnect, each of the instances configured to receive data from a different one or more of the source components.

19. 20. The apparatus of claim 18, further comprising: a further hardware module connected to the first destination, the further hardware module configured to output data from the different ones of the source components connected to each of the plurality of instances of the hardware module to the first destination, wherein a total bandwidth of the different ones of the source components exceeds a bandwidth of the further hardware module for providing data to the first destination.

20. The device of claim 17 , wherein the device is an integrated circuit.

21. 1. A method for receiving data from a plurality of source components and forwarding at least a portion of the data to a first of a plurality of destinations, comprising: receiving a first set of data packets comprising data packets originating from a first subset of the source components and providing the first set of data packets to a first input buffer; receiving a second set of data packets comprising data packets originating from a second subset of the source components and providing the second set of data packets to a second input buffer; examining a destination identifier in one of the data packets included in the first set of data packets that is located at the top of the first input buffer; examining one or more source identifiers in one or more data packets of the second set of data packets to determine which of the second subset of source components each of the one or more data packets originated from; responsive to determining that the destination identifier identifies the first destination, selecting between the first input buffer and the second input buffer as a source of data to the first destination according to the one or more source identifiers, and arbitrating among the plurality of source components for transmitting data to the first destination; Including, the first subset is comprised of one or more of the plurality of source components, and some of the data packets included in the first set of data packets are intended for transmission to different ones of the plurality of destinations; the second subset of source components includes two or more source components, and the first subset is comprised of a different number of source components relative to the second subset. method.

22. 1. A computer program comprising a set of computer readable instructions that cause at least one processor to perform a method, said method comprising: examining a destination identifier in one of the data packets in the first set of data packets located at the head of the first input buffer; examining one or more source identifiers in one or more data packets of the second set of data packets stored in the second input buffer to determine which of a second subset of source components each of the one or more data packets originated from; in response to determining that the destination identifier identifies a first destination of the plurality of destinations, selecting between the first input buffer and the second input buffer as a source of data to the first destination according to the one or more source identifiers, and arbitrating among the plurality of source components for transmitting data to the first destination; Including, the data packets in the first set of data packets originate from a first subset of a plurality of source components, the first subset consisting of one or more of the plurality of source components, and some of the data packets in the first set of data packets are for transmission to different ones of a plurality of destinations; the second subset of the plurality of source components includes two or more source components, and the first subset of source components is comprised of a different number of source components relative to the second subset. Computer program.

Images