A Robust Age Saturation Mechanism for Age-Based Arbitration in Packet Networks

Abstract

A system and method for routing data packets in an interconnect network. Data packets forwarded across the interconnect network each include age data. Routers positioned throughout the interconnect network can control the flow of data packets through the use of aging first-in-first-out (FIFO) queues and age-based arbiters. The age-based arbiters in the routers are configured to prioritize older data packets over newer data packets being pushed from the FIFO queue. Each data packet inserted into the FIFO queue can have its age data updated such that it is converted from age to injection time. When a data packet is read from the FIFO, the age data of the data packet is converted back to age.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Provisional Patent Application No. 17 / 704,677, filed March 25, 2022, which claims the benefit of the filing date of U.S. Provisional Patent Application No. 63 / 314,067, filed February 25, 2022, the disclosure of which is incorporated herein by reference. [Background technology]

[0002] background Routers are typically positioned throughout an interconnection network, such as a distributed network of processors. Each router may include an arbiter that controls the forwarding flow of data packets from the router. The arbiter may resolve conflicts, such as multiple packets competing for access to the same resource. Many arbiter designs are "locally fair" such that each source of packets gets equal access to the resource. However, these locally fair arbiters combined with any asymmetry in the overall network may result in "global unfairness," such as under-service of a packet source. Such global unfairness may ultimately lead to a loss of overall performance of the network. Asymmetries that lead to global unfairness may result from the topology of the network, the topology of virtual networks within the network, or traffic demands on the network. Summary of the Invention [Means for solving the problem]

[0003] overview The present technology is directed to an age-based arbitration system and method.

[0004] Aspects of the technology are directed to a method of routing data packets in an interconnect network, the method including receiving a plurality of data packets at a node in the interconnect network, the node including a set of queues, each received packet of the plurality of data packets including an age, the method further including inputting the set of received data packets of the plurality of data packets into a first queue in the set of queues, and for each data packet in the set of received data packets, replacing an age of the data packet with an injection time corresponding to a local time of the first queue when the data packet was input into the first queue and the age of the data packet, determining an updated age based on a local time of the first queue when the data packet was forwarded from the first queue and the injection time of the data packet, and replacing the injection time of the data packet with the updated age.

[0005] Another aspect of the technology is directed to a system for routing data packets in an interconnection network. The system may include a router including at least one arbiter and a set of queues. The router may be configured to receive a plurality of data packets, each received packet of the plurality of data packets including an age, input the set of received data packets of the plurality of data packets into a first queue in the set of queues, and for each data packet in the set of received data packets, replace the age of the data packet with an injection time corresponding to a local time of the first queue when the data packet was input into the first queue and the age of the data packet, determine an updated age based on a local time of the first queue when the data packet was forwarded from the first queue and the injection time of the data packet, and replace the injection time of the data packet with the updated age.

[0006] In some cases, the updated age may be limited to a maximum age value. In some cases, each data packet that can be forwarded from the first queue is sent to an arbiter.

[0007] In some cases, an arbiter receives the first data packet forwarded from the first queue and at least one other data packet forwarded from one or more of the other queues of the set of queues, In some examples, the arbiter identifies an oldest data packet from the first data packet and the at least one other data packet and forwards the oldest data packet.

[0008] In some cases, the injection time is calculated by subtracting the age of the data packet from the local time of the first queue when the data packet is input to the first queue.

[0009] In some cases, the updated age is determined by subtracting the injection time of the data packet from the local time of the first queue when the data packet is forwarded from the first queue.

[0010] In some examples, the updated age is adjusted by an aging rate based on the value of the updated age relative to the maximum age.

[0011] In some examples, the aging rate is not adjusted when the updated age is less than half the maximum age.

[0012] In some examples, the aging ratio is (i) 0.5 when the updated age is 50%-75% of the maximum age, and (ii) when the updated age is [1-2 -n and 1-2 -n-1 ], 2 -n It is. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an exemplary interconnection network in accordance with an aspect of the present disclosure. [Diagram 2] FIG. 2 illustrates an example of a router, according to an aspect of the present disclosure. [Diagram 3] FIG. 2 illustrates an example of a data packet according to an aspect of the present disclosure. [Figure 4A] 13A-13C illustrate examples of updating age data in a data packet according to aspects of the present disclosure. [Figure 4B] 13A-13C illustrate examples of updating age data in a data packet according to aspects of the present disclosure. [Diagram 5] FIG. 13 illustrates logic for updating age data in a data packet according to an aspect of the disclosure. [Figure 6] 1 is a flow diagram illustrating a data transfer technique for transferring data to a truly wireless device according to an aspect of the disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Detailed Description The techniques described herein are directed to the routing of data packets in an interconnect network. For example, components such as processors in the interconnect network can send and receive data packets to and from other components in the interconnect network. To improve timeliness of delivery of data packets across the interconnect network, the data packets can include age data. Routers positioned throughout the interconnect network control the flow of data packets through the use of aging first-in-first-out (FIFO) queues and age-based arbiters. The age-based arbiters in the routers are configured to prioritize older data packets over newer data packets being pushed from the FIFO queue. Additionally, each data packet inserted into the FIFO queue can have its age data updated to be converted from age to injection time. When a data packet is read from the FIFO, the age data of the data packet is converted back to age.

[0015] As explained above, locally fair arbiters have drawbacks that result in global unfairness. These drawbacks of locally fair arbiters can be addressed by replacing them with age-based arbiters. Age-based arbiters can use the "age" of a data packet, which represents the time the data was inserted into the network, to give priority to older packets when resolving contention when multiple packets are competing for the same resource, such as the same transmission link, the same input, etc.

[0016] A drawback of existing age-based arbiters is that updating the age of each data packet requires careful and precise tuning of parameters and / or global coordination among routers so that the age does not become excessively large. And since each data packet must carry its own age across the network, the number of bits used to store this age is typically limited. A very large age may become very small if the age of a data packet overflows by exceeding the maximum value represented by the bits used to store the age. For example, age data represented by 8 bits can store age values ​​from 0 to 255. If the age of a data packet exceeds 255, the age of the data packet may be reset back to 0, which may lead to unpredictable behavior, including further delays in the delivery of the data packet.

[0017] Furthermore, to prevent overflow, it may require a lot of resources to directly monitor the age of every data packet in the interconnection network, because packets are usually queued in large memory modules such as random access memory (RAM), and the stored data packets cannot be easily modified without significant time delays for reading and rewriting the stored data packets.

[0018] To address these issues using known age-based arbiters, an aging FIFO queue can be used. Each aging FIFO can be constructed from a RAM, such as an SRAM, and can contain tens or even hundreds of data packets. To avoid the issue of needing to continually read and rewrite the data packets to update their respective ages, each data packet inserted into the aging FIFO can have its age converted into an "injection time" relative to a local clock associated with the aging FIFO. The data packet can retain its injection time until it is transferred out of the aging FIFO. At this point, the injection time can be converted back into an age, again using a local clock associated with the aging FIFO. By converting the age of the data packet to the time value when it was added to the aging FIFO, the age of the data packet can be determined again at or before it is transferred out of the aging FIFO, thereby avoiding the need to continually update the age of the data packet as it traverses the FIFO.

[0019] FIG. 1 illustrates an example of an interconnection network 100 that includes a collection of nodes and interconnections. In this regard, FIG. 1 illustrates a 4×4×4 interconnection network 100 that includes four layers of nodes identified by arrows 101, 102, 103, and 104. Each layer of nodes includes 16 nodes positioned within a 4×4 grid. For clarity, only four nodes of each layer are identified. For example, layer 101 includes nodes 111a-111d, layer 102 includes nodes 112a-112b, layer 103 includes nodes 113a-113d, and layer 104 includes nodes 114a-114d.

[0020] A node may include any computing resource capable of communicating with other computing resources. For example, a node may include a computer, a server, a mobile device, a processor, a core, a router, a memory, a card (e.g., an accelerator card), a FIFO or other such queue, etc. In some cases, a node may include a collection of computer resources, such as a processor with a router and a FIFO, etc.

[0021] As further shown in Figure 1, nodes of the interconnect network may be connected to other nodes via interconnects, such as interconnects 121 and 122. For clarity, not all interconnects are shown or referenced in Figure 1. Each interconnect may be any type of connection between nodes, such as a wire, cable, trace, etc. Although each interconnect is shown as a single line, each interconnect may represent any number of connections between nodes.

[0022] Although FIG. 1 illustrates a 4x4x4 interconnection network 100, an interconnection network may include any number of nodes in any configuration. For example, an interconnection network may include nodes in a one-dimensional array, nodes in a two-dimensional array, independent nodes not included in an array, a collection of nodes in any configuration connected to other nodes, etc. Furthermore, each node may be connected to some or all of the other nodes using any number of interconnects. Thus, FIG. 1 is merely an example layout of an interconnection network, and the techniques described should not be considered limited to the interconnection network illustrated in FIG. 1.

[0023] 2 illustrates an example router 200 configured to control a flow of data packets through at least a portion of an interconnect network, such as interconnect network 100. Router 200 includes aging FIFOs 210, 211 that receive data packets directly from nodes. The aging FIFOs 210, 211 can receive data packets from the same node or different nodes. For example, aging FIFO 210 can receive data packets from a first node, as indicated by arrow 212, and aging FIFO 211 can receive data packets from a second node, as indicated by arrow 213.

[0024] Each aging FIFO 210, 211 can forward a data packet to an arbiter 220, as indicated by arrows 214 and 215, respectively. The arbiter 220 can identify the oldest data packet based on the age of the data packet and forward the data packet to another node 230, as indicated by arrow 231.

[0025] The router may be located along an interconnect and / or within other resources. For example, router 200 may be located within a processor, a core, etc. Although router 200 is shown with only two aging FIFOs, a router may include any number of FIFOs and arbiters.

[0026] 3 illustrates an exemplary data packet 301 that may be transferred in an interconnected network, such as interconnected network 100. The data packet includes 16 bits, including bits 311 and 312. For clarity, only bits 311 and 312 are referenced.

[0027] 3, data packet 301 includes age information 303 and data information 305. Age information 303 is represented by 4 bits, including bit 311, and data 305 is represented by 12 bits, including bit 312.

[0028] Data packet 301 is an exemplary data packet, and other data packets having age data may be transferred over the interconnection network. In this regard, the data packets may be of any size, such as 8, 16, 32, 64, 128 bits, etc. Furthermore, the age information and the data information may similarly be represented by any number of bits. For example, the age data may be represented by 8, 16, 32 bits, etc.

[0029] Additionally, the location of the age data relative to other data within a data packet may differ from that shown in Figure 3. For example, the age data may be positioned after other data, between other data, and / or split throughout the data packet.

[0030] As explained above, the age data of a data packet may be replaced with time data when inserted into the aging FIFO. For example, as shown in FIG. 4A, a data packet 410 may be transferred to the aging FIFO 410. Before or within the aging FIFO, the age data of the data packet 410 may be replaced with time data by logic 440, which may be a discrete adder or other such discrete circuit. In this regard, as shown by arrow 430, the logic 440 may receive the data packet 410. As shown by arrow 431, the logic may convert the age data to time data and output the data packet 410 having the time data instead of the age data. The data packet 410 may then be stored in an initial location 412 of the aging FIFO 411.

[0031] After the data packet with time data progresses to the last location 419 of the aging FIFO, the aging FIFO 411 can output the data packet 410, as indicated by arrow 451, and input to additional logic 441, as indicated by arrow 452. The logic 441 can convert the time back to age data representing the current age of the data packet 441, as indicated by arrow 453, and send the data packet to its next destination, which may be an arbiter. In some cases, the time data of the data packet 410 may be modified before being output by the aging FIFO 411.

[0032] Although FIGS. 4A, 4B and 5 show discrete logic for converting the age and time data, in some cases the logic 440 may be implemented by a processor.

[0033] FIG. 5 is a diagram of a process for converting age data of a data packet to time data and vice versa for input to an aging FIFO such as aging FIFO 411. As shown, a data packet includes age data representing "push_age". push_age is the age of the data packet when it is received by the router and / or the aging FIFO. To convert push_age to a time value, push_age is subtracted by an adder from a "local_time" value provided by a local clock associated with the aging FIFO (the "local time counter"). The resulting value is the "push_time", also referred to as the "injection time". The push_age data of a data packet replaces the "push_time" when the data packet is inserted into the aging FIFO.

[0034] As further shown in FIG. 5, the "push_time" data of the packet is converted into an age value "pop_age" that represents the age of the data packet when it is transferred from the aging FIFO. To determine the "pop_age", logic represented by an adder subtracts the "pop_time" from the current local_time, which is the same value as the injection time. The age data of the data packet is then updated with the "pop_age" that indicates the current age of the data packet.

[0035] For example, a data packet may have a "push_age" of 2 when it is injected into the FIFO at time T=10 (as indicated by the local clock). When the data packet is injected into the FIFO, the push_age of 2 is replaced with a push_time having a value of 8: push_time (i.e., injection time)=10(local_time)-2(push_age).

[0036] Upon retrieval at time T=15, the data packet has a pop_age of 7: pop_age=15(local_time)-8(injection time).

[0037] A property of the injection times (i.e., push_times) is that they do not change as packets age. This eliminates the need to continually update the age of a data packet by converting it to an injection time and writing the injection time instead of the age into the FIFO's RAM. When a data packet is read from the RAM, the injection time can be converted back to an age using the current value of the local clock. Because this conversion from age to time and back to age is all done within the aging FIFO, the local clock does not require any synchronization with the local clocks of other aging FIFOs in the network. Furthermore, this local clock approach avoids any kind of global synchronization that may be required in other embodiments.

[0038] A drawback of the age-time-age conversion is that it may be difficult to bound the age of a data packet when it is transferred from the FIFO. Thus, the bit width required for the push and pop times may be larger than that provided by the data packet. To address this, the push time of a data packet may be modified to be a bounded amount less than the largest push time of a data packet that arrived in the queue (i.e., the aging FIFO) before it. This modification is expected to be minimal and similar to the priority inversion encountered in a FIFO. Age-based arbitration favors older data packets, or equivalently, favors data packets with earlier injection times. If a data packet with an earlier push time arrives in the FIFO after a data packet with a later push time, the earlier data packet is blocked behind the later packet and essentially inherits its lower priority. This is a priority inversion.

[0039] Furthermore, if age-based arbitration is performing well, data packets can be expected to arrive at each aging FIFO roughly in order of push time. The net expectation is that enforcing a decay limit push time should not significantly impact the overall performance of the age-based arbitration scheme.

[0040] When inputting a data packet into the aging FIFO, the following sequence of steps may be performed: First, the push time of the data packet may be determined by calculating the local_time of the aging FIFO minus the push_age of the packet.

[0041] If the aging FIFO is empty or the push_time of a data packet is less than the max_push_time associated with the aging FIFO, the max_push_time is replaced with the previously determined value of push_time.

[0042] If the push_time of a data packet is less than max_push_time-T, the value of push_time is updated to be equal to max_push_time-T, where T is a constant that determines the maximum decrease in the sequence of push times, and T>=0.

[0043] When performing the steps described herein, the various times (local_time, push_time, and max_push_time) can all be represented using the same number of bits as the packet age. M can be thought of as the bit width of these quantities. In some cases, the M-bit representation can wrap around (i.e., go beyond the maximum value and back to a lower value). In general, this problem can be avoided by comparing the deltas of the times and relying on the properties of the aging FIFO to ensure that these deltas are always within the range [0,2^M).

[0044] In the next step, push_time can be compared to max_push_time. Instead of comparing them directly, the comparison can be transformed so that it is between differences known to be in [0,2^M): push_time <max_push_time =>local_time-push_age <max_push_time =>local_time-max_push_time <push_age The left hand side of the final equation (local_time - max_push_time) is the age of the youngest packet already in the aging FIFO. This equation is known to be in [0,2^M) since the FIFO avoids age overflow.

[0045] In step 3, push_time is compared to max_push_time-T. This comparison can be translated as follows: push_time <max_push_time-T =>local_time-push_age <max_push_time-T =>local_time-max_push_time+T <push_age It follows that the left-hand side is in [T,2^M+T). Given T<2^M, the final expression can be safely evaluated using M+1-bit arithmetic.

[0046] In this formulation, T is a threshold parameter that determines the maximum decrease. To prevent the age from overflowing, the value of the data packet at the head of the FIFO is examined to determine the age of the next entry to be popped (i.e., forwarded). This age is referred to herein as the pop age.

[0047] pop_age=local_time-pop_time The local time can then be incremented each cycle that pop_age plus T is less than the maximum age. By doing so, none of the queued packets can overflow, since the bounded decay ensures that no packet has an age more than T greater than the age of the head packet. Thus, the pop age is in [0,2^M], so we can ignore wrapping and express all time instants as M bit quantities.

[0048] To further extend the range of representable ages, the precision can be reduced as the age increases through age slowing. In age slowing, the aging rate is reduced as the data packet approaches the maximum age. For example, the aging rate of a data packet can be reduced by a factor of two depending on the current age. If the age is less than half the maximum, the aging rate is consistent with the previous section. If the age is between half and three-quarters of the maximum, the aging rate is halved, and so on, as shown in Table 1.

[0049] [Table 1]

[0050] In the case of M-bit age field, the effective maximum age is M2 due to the age slowdown. M The aging rate values ​​in Table 1 are merely examples and other rates may be used.

[0051] In some cases, the number of bits required to store the age of a data packet can be determined using the following formula:

[0052]

number

[0053] 6 shows a flow diagram for routing data packets in an interconnect network. As shown in block 701, a plurality of data packets are received at a node in the interconnect network. The node may include a set of queues, and each received packet of the plurality of data packets includes an age.

[0054] As indicated at block 703, a received set of data packets of the plurality of data packets may be inserted into a first queue in the set of queues.

[0055] As shown in block 705, for each data packet in the set of received data: (i) replacing the age of the data packet with an injection time corresponding to the local time of the first queue when the data packet was input to the first queue and the age of the data packet; (ii) the updated age is determined based on a local time of the first queue when the data packet is forwarded from the first queue and an injection time of the data packet; (iii) The injection time of the data packet may be replaced with an updated age.

[0056] Unless otherwise stated, the above alternatives are not mutually exclusive and may be implemented in various combinations to achieve specific advantages. These and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, and therefore the above description of the embodiments should be interpreted as illustrative, and not limiting, of the subject matter defined by the claims. In addition, the definition of examples described herein, as well as phrases such as "such as," "including," and the like, should not be interpreted as limiting the subject matter of the claims to specific examples, but rather, these examples are intended to illustrate only one of many possible embodiments. Moreover, the same reference numbers in different drawings may identify the same or similar elements.

Claims

1. 1. A method for routing data packets in an interconnection network, the method comprising: receiving a plurality of data packets at a node in the interconnect network, the node including a set of queues, each received packet of the plurality of data packets including an age, the method comprising: inputting a set of received data packets of the plurality of data packets into a first queue in the set of queues; for each data packet in the set of received data packets, replacing the age of the data packet with a local time of the first queue when the data packet was input to the first queue and an injection time corresponding to the age of the data packet; determining an updated age based on the local time of the first queue when the data packet was forwarded from the first queue and the injection time of the data packet; replacing the injection time of the data packet with the updated age; The method further comprising:

2. The method of claim 1 , further comprising limiting the updated age to a maximum age value.

3. 3. The method of claim 1, wherein each data packet forwarded from the first queue is sent to an arbiter.

4. 3. The method of claim 1, wherein an arbiter receives a first data packet forwarded from the first queue and at least one other data packet forwarded from one or more of the other queues of the set of queues.

5. The arbiter Identifying an oldest data packet from the first data packet and the at least one other data packet; The method of claim 4 , further comprising forwarding the oldest data packet.

6. 3. The method of claim 1, wherein the injection time is calculated by subtracting the age of the data packet from the local time of the first queue when the data packet was input to the first queue.

7. 3. The method of claim 1, wherein the updated age is determined by subtracting the injection time of the data packet from the local time of the first queue when the data packet is forwarded from the first queue.

8. The method of claim 7 , wherein the updated age is adjusted by an aging rate based on a value of the updated age relative to a maximum age.

9. The method of claim 8 , wherein the aging rate is not adjusted when the updated age is less than half of the maximum age.

10. The aging rate is (i) 0 when the update age is equal to the maximum age; (ii) 0.5 when the updated age is between 50% and 75% of the maximum age; (iii) the updated age divided by the maximum age is 1-2 for any integer n>1; －ｎ ~1-2 －ｎ－１ When －ｎ The method according to claim 8, wherein

11. 1. A system for routing data packets in an interconnect network, the system comprising: a router including at least one arbiter and a set of queues, said router comprising: receiving a plurality of data packets, each received packet of the plurality of data packets including an age; The router inputting a set of received data packets of the plurality of data packets into a first queue in the set of queues; for each data packet in the set of received data packets, replacing the age of the data packet with a local time of the first queue when the data packet was input to the first queue and an injection time corresponding to the age of the data packet; determining an updated age based on the local time of the first queue when the data packet was transmitted from the first queue and the injection time of the data packet; replacing the injection time of the data packet with the updated age; The system is configured as follows.

12. The system of claim 11 , wherein the router is further configured to limit the updated age to a maximum age value.

13. 13. The system of claim 11 or 12, wherein each data packet forwarded from the first queue is sent to an arbiter.

14. 13. The system of claim 11 or 12, wherein the at least one arbiter receives a first data packet forwarded from the first queue and at least one other data packet forwarded from one or more of the other queues of the set of queues.

15. The at least one arbiter Identifying an oldest data packet from the first data packet and the at least one other data packet; The system of claim 14 further comprising forwarding the oldest data packet.

16. 16. The system of claim 15, wherein the injection time is calculated by subtracting the age of the data packet from the local time of the first queue when the data packet was input to the first queue.

17. 17. The system of claim 16, wherein the updated age is determined by subtracting the injection time of the data packet from the local time of the first queue when the data packet was forwarded from the first queue.

18. The system of claim 17 , wherein the updated age is adjusted by an aging rate based on a value of the updated age relative to a maximum age.

19. 20. The system of claim 18, wherein the aging rate is not adjusted when the updated age is less than half of the maximum age.

20. The aging rate is (i) 0 when the update age is equal to the maximum age; (ii) 0.5 when the updated age is between 50% and 75% of the maximum age; (iii) the updated age divided by the maximum age is 1-2 for any integer n>1; －ｎ ~1-2 －ｎ－１ When －ｎ 20. The system of claim 18, wherein:

Images