Forwarding based on guaranteed latency

Abstract

Apparatus and techniques are provided for on-time packet transmission through a series of computing nodes in a network. A node receives a packet from an input network interface, enforces an earliest allowed time for transmission of the packet to a next node based on data extracted from an information field in the packet, determines a transmission time for the packet that is no earlier than the earliest allowed transmission time, stores an indication of a minimum latency for the packet at the next node in the information field of the packet based on the determined transmission time, and transmits the packet to the next node through an output network interface.

Description

Technical Field

[0001] This invention generally relates to communication networks, and more specifically to the transmission of data packets over a network. Background Technology

[0002] Data packets are formatted units of data carried by computing nodes (such as servers, routers, or switches) in a computer network. Data packets can be received at nodes in different traffic flows and can have different priorities. Based on the node's capabilities and these priorities, data packets are scheduled for transmission to the next node. Typically, end-to-end latency, or the time it takes for a data packet to travel from the sending node to the receiving node, can vary. However, this can be problematic for applications requiring precise data packet delivery. Summary of the Invention

[0003] According to one aspect of the present invention, an apparatus for a node in a network is provided. For example, the apparatus may include circuitry forming part of the node. The apparatus includes: a non-transient memory including instructions; one or more network interfaces for receiving data packets; one or more output network interfaces for transmitting data packets; and one or more processors communicating with and coupled to the non-transient memory and the plurality of input network interfaces and the output network interfaces. The one or more processors execute the instructions to: receive data packets from the one or more input network interfaces; enforce an earliest allowed time for sending the data packets to the next node in the network based on data extracted from an information field in the data packets; determine a transmission time for the data packets not earlier than the earliest allowed transmission time; store, according to the determined transmission time, an indication of the minimum delay of the data packets at the next node in the information field of the data packets; and send the data packets to the next node via the output network interface.

[0004] Optionally, in the foregoing aspect, in order to calculate the indication of the minimum delay, the one or more processors further execute the instruction to calculate: LATENCY – (Ttrans – Tarr + MinDelay), where LATENCY is a value known to the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, and MinDelay is the minimum delay identified by the data extracted from the information field in the data packet.

[0005] Optionally, in the foregoing aspects, in order to calculate the indication of the minimum delay, the one or more processors further execute the instruction to calculate: LATENCY – (Ttrans – Tarr – MinDelay) – SerDelay, where LATENCY is a value known to the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, MinDelay is the minimum delay identified by the data identifier extracted from the information field in the data packet, and the serialization delay of the data packet, SerDelay, is the size of the data packet (in bits) divided by the transmission rate of the output network interface (in bits per second).

[0006] Optionally, in some of the above aspects, the node pre-configures the LATENCY prior to the arrival time of the data packet.

[0007] Optionally, in some of the foregoing aspects, the one or more processors further execute the instructions to extract a PRIORITY value from the information field of the data packet and determine the LATENCY based on the PRIORITY value. For example, the LATENCY can be determined by reading it from a pre-configured table of each PRIORITY delay value by indexing the PRIORITY value.

[0008] Optionally, in some of the foregoing aspects, the one or more processors further execute the instructions to determine the PRIORITY value based on a sequence of hop priority information fields and a hop count index in the data packet; to determine the index of the item in the sequence used as the PRIORITY value based on the value of the hop count index after the data packet is received; and to increment the hop count index in the data packet before the data packet is sent to the next hop.

[0009] Optionally, in the foregoing aspects, the one or more processors further execute the instructions to determine the minimum latency based on a clock asynchronous with the clock of the next node.

[0010] Optionally, in the foregoing aspects, the one or more processors further execute the instructions to determine the transmission time and the arrival time based on a clock whose timestamps have only local significance and which starts counting from the time the node is powered on (e.g., from 0).

[0011] Optionally, in the above aspects, the data packet is in a service flow that includes a sequence of related data packets; the data packet contains one or more fields that together constitute a service flow identifier, the value of which is unique to the data packets of the service flow; the one or more processors further execute the instructions to determine the minimum latency independently of the service flow identifier.

[0012] Optionally, in some of the above aspects, the one or more input network interfaces are used to receive multiple service flows; the LATENCY is not greater than the sum of the burst size of each data packet and the maximum data packet size in the multiple service flows divided by the rate at which data packets are serialized to the output network interface.

[0013] Optionally, in the above aspects, after the enforcement of the earliest allowed time for transmitting the data packet, the one or more processors execute the instruction to enqueue the data packet into a pre-existing scheduler, and the scheduler determines the transmission time.

[0014] Optionally, in the foregoing aspects, the one or more processors further execute the instructions to delay the data packet passing through a timed Push-On, First-Out (PIFO) queue, wherein the PIFO level of the data packet corresponds to the earliest allowed transmission time, which is the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet, and the head of the PIFO queue is served no earlier than its level.

[0015] Optionally, in some of the foregoing aspects, the one or more processors further execute the instructions to: a) use a separate timed push-on, first-out (PIFO) queue for each value of PRIORITY; b) select the PIFO to which the data packet is to be inserted based on the PRIORITY, wherein: c) the PIFO level of the data packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet; d) the header of each timed PIFO is served no earlier than its level, at which point there is no higher priority timed PIFO whose header can be served.

[0016] Optionally, in some of the foregoing aspects, the one or more processors further execute the instructions to: a) use a separate timing FIFO for each combination (PRIORITY, NTERFACE) of possible values ​​of PRIORITY and valid incoming NTERFACE index; b) select the FIFO to which the packet is to be inserted and select the interface from which the packet is received, based on the PRIORITY, wherein: c) the timing FIFO rank of the packet is the earliest transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the packet; d) the header of the timing FIFO is serviced if: i) the rank of the header of the FIFO is equal to or earlier than the current time; ii) there is no FIFO with a higher PRIORITY that can be serviced; iii) there is no FIFO with the same PRIORITY but an earlier header rank.

[0017] Optionally, in the above aspects, the data extracted from the information field includes additional latency experienced at the node.

[0018] Optionally, in the above aspects, the data extracted from the information field includes a minimum latency value.

[0019] According to a second aspect of the present invention, a method for a node in a network is provided. The method includes: receiving a data packet from one or more input network interfaces; enforcing an earliest allowed time for sending the data packet to a next node in the network based on data extracted from an information field in the data packet; determining a transmission time for the data packet that is not earlier than the earliest allowed transmission time; storing an indication of minimum delay in the information field of the data packet; and sending the data packet to the next node via an output network interface.

[0020] Optionally, in the foregoing aspects, the method further includes calculating the indication of the minimum delay, wherein the calculation includes calculating: LATENCY – (Ttrans – Tarr – MinDelay), where LATENCY is a known value of the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, and MinDelay is the minimum delay identified by the data extracted from the information field in the data packet.

[0021] Optionally, in the foregoing aspects, the method further includes calculating the indication of the minimum delay, wherein the calculation includes calculating: LATENCY – (Ttrans – Tarr – MinDelay) – SerDelay, where LATENCY is a known value of the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, MinDelay is the minimum delay identified by the data identifier extracted from the information field in the data packet, and SerDelay is the serialization delay of the data packet.

[0022] Optionally, in some of the foregoing aspects, the method further includes pre-configuring the LATENCY for the node prior to the arrival time of the data packet.

[0023] Optionally, in some of the above aspects, the method further includes extracting a PRIORITY value from an information field in the data packet and determining the LATENCY based on the PRIORITY value.

[0024] Optionally, in the above aspects, the method further includes: determining the PRIORITY value based on a sequence of hop priority information fields and a hop count index in the data packet, wherein the value of the hop count index after receiving the data packet determines the index of the item in the sequence used as the PRIORITY value, and the hop count index is incremented in the data packet before the data packet is sent to the next hop.

[0025] Optionally, in any of the foregoing aspects, the method further includes determining the minimum delay based on a clock asynchronous with the clock of the next node.

[0026] Optionally, in any of the foregoing aspects, the method further includes determining the transmission time and arrival time based on a clock whose timestamp has only local significance and which starts counting from the time the node is powered on.

[0027] Optionally, in some of the foregoing aspects, the method further includes determining the minimum latency independently of the service flow identifier, wherein the data packet is located in a service flow comprising a sequence of related data packets, the data packet containing one or more fields that together constitute the service flow identifier, the value of the identifier being unique to the data packets of the service flow.

[0028] Optionally, in some of the above aspects, the plurality of input network interfaces are used to receive a plurality of service streams, and the method further includes determining LATENCY as a value not greater than the sum of the burst size of each data packet and the maximum data packet size in the plurality of service streams divided by the rate at which data packets are serialized to the output network interface.

[0029] Optionally, in some of the foregoing aspects, the method further includes, after the enforcement of the earliest allowed time for transmitting the data packet, queuing the data packet into a pre-existing scheduler, wherein the scheduler determines the transmission time.

[0030] Optionally, in some of the foregoing aspects, the enforcement of the earliest allowed transmission time includes delaying the data packets through a timed Push-On, First-Out (PIFO) queue, wherein the PIFO class of the data packets corresponds to the earliest allowed transmission time summed with the arrival time and the minimum delay of the data identifier extracted from the information field in the data packets, and the head of the PIFO queue is served no earlier than its class.

[0031] Optionally, in some of the foregoing aspects, the method further includes: using a separate timed push-on, first-out (PIFO) queue for each value of PRIORITY; selecting a PIFO to insert the data packet into based on the PRIORITY, wherein the PIFO level of the data packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet; and servicing the header no earlier than the level of the header of each timed PIFO, at which point there is no higher priority timed PIFO for which the header can be served.

[0032] Optionally, in some of the foregoing aspects, the method further includes: using a separate timing FIFO for each possible combination (PRIORITY, INTERFACE) of the PRIORITY and the valid incoming INTERFACE index; selecting the FIFO to which the data packet is to be inserted and the interface from which the data packet is received, based on the PRIORITY, wherein the timing FIFO rank of the data packet is the earliest transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet; and servicing the header of the timing FIFO when: the rank of the header of the FIFO is equal to or earlier than the current time; there is no FIFO with a higher PRIORITY that can be served; and there is no FIFO with the same PRIORITY but a higher header rank.

[0033] Optionally, in the above aspects, the data extracted from the information field includes additional latency experienced at the node.

[0034] Optionally, in the above aspects, the data extracted from the information field includes a minimum latency value.

[0035] According to a third aspect of the invention, an apparatus includes: one or more input network interfaces at a current node in a network, the one or more input network interfaces being configured to receive a data packet, the data packet including an indication of a first minimum delay to be enforced at the current node before sending the data packet to a next node in the network; and an output network interface at the current node, the output network interface being configured to transmit the data packet to the next node based on the first minimum delay at a time no earlier than the earliest allowed time for transmitting the data packet at the current node, the transmitted data packet including an indication of a second minimum delay to be enforced at the next node before transmitting the data packet in the network.

[0036] Optionally, in the above aspects, the second minimum delay provides a specified delay between the earliest allowed time for transmitting the data packet at the current node and the earliest allowed time for transmitting the data packet at the next node.

[0037] Optionally, in some of the above aspects, the data packet includes a priority value; the specified delay is based on the priority value.

[0038] Optionally, in some of the above aspects, the data packet includes a hop count index; the priority value is based on the hop count index.

[0039] Optionally, in some of the above aspects, the second minimum delay is a decreasing function of the difference between the transmission time of the data packet at the current node and the earliest allowed time for transmitting the data packet at the current node.

[0040] Optionally, in some of the above aspects, the indication of the first minimum latency includes an information field in the data packet that stores the first minimum latency.

[0041] Optionally, in some of the above aspects, the indication of the first minimum latency includes an information field in the data packet that stores additional latency.

[0042] According to a fourth aspect of the invention, a method includes: receiving a data packet at one or more input network interfaces of a current node in a network, the data packet including an indication of a first minimum delay to be enforced at the current node before sending the data packet to a next node in the network; and transmitting the data packet to the next node at an output network interface at the current node, based on the first minimum delay, at a time no earlier than the earliest allowed time for transmitting the data packet at the current node, the transmitted data packet including an indication of a second minimum delay to be enforced at the next node before transmitting the data packet in the network.

[0043] According to a fifth aspect of the invention, a non-transient memory includes instructions for causing a computer to perform the following steps: receiving a data packet from an input network interface; enforcing an earliest allowed time to send the data packet to the next node in the network based on data extracted from an information field in the data packet; determining a transmission time for the data packet that is no earlier than the earliest allowed transmission time; storing, according to the determined transmission time, an indication of the minimum delay of the data packet at the next node in the information field of the data packet; and sending the data packet to the next node via an output network interface.

[0044] This invention provides a brief overview of some concepts, which will be further described in the specific embodiments. This invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that address any or all the deficiencies mentioned in the background art. Attached Figure Description

[0045] Various aspects of the invention are illustrated by way of example and are not limited to the accompanying drawings, in which similar reference numerals denote similar elements.

[0046] Figure 1A The time delay range between the lower bound (LB) and the upper bound (UB) is shown.

[0047] Figure 1B An example of a timeliness guarantee is shown, which has an upper bound (UB) but no lower bound (LB) on the delay (i.e., LB = 0).

[0048] Figure 1C An example of a minimum delay guarantee is shown, which has a lower bound (LB) but no upper bound (UB) on the delay.

[0049] Figure 1D The best-effort model is shown, where the upper bound (UB) is not guaranteed.

[0050] Figure 1E An example of a time-guarantee is shown, where the lower bound (LB) is equal to the upper bound, or more generally, the time range between LB and UB is relatively narrow.

[0051] Figure 2 An exemplary communication system for sending data is shown.

[0052] Figure 3A An exemplary network 300 comprising a series of nodes is shown.

[0053] Figure 3B This shows node 400 (e.g.) Figure 3A A schematic diagram illustrating exemplary details of any node in the network.

[0054] Figure 4A Examples describing the delays experienced by packets at Node(i) and Node(i+1) include: the corresponding scheduling delays SchedDelay(i) and SchedDelay(i+1), and the corresponding hop delay Hop_Latency(i) ≤ MAX + unscheduled delay.

[0055] Figure 4B Examples describing the delays experienced by data packets at Node(i) and Node(i+1) include: the corresponding mandatory minimum delays MinDelay(i) and MinDelay(i+1), the corresponding scheduling delays SchedDelay(i) and SchedDelay(i+1), and the corresponding hop delay Hop_Latency(i) = MAX + unscheduled delay.

[0056] Figure 4B1 Described with Figure 4B The consistent time period is MAX.

[0057] Figure 4B2 Describes the method used to determine Figure 4B An exemplary process step of MinDelay(i+1) in the example.

[0058] Figure 4C Examples describing the delays experienced by data packets at Node(i) and Node(i+1) include: the corresponding mandatory minimum delays MinDelay(i) and MinDelay(i+1), the corresponding additional delays AddDelay(i) and AddDelay(i+1), the corresponding scheduling delays SchedDelay(i) and SchedDelay(i+1), and the corresponding hop delay Hop_Latency(i).

[0059] Figure 4C1 Described with Figure 4C The consistent time period is LATENCY(i).

[0060] Figure 4C2 Describes the method used to determine Figure 4C An exemplary process step of MinDelay(i+1) in the example.

[0061] Figure 4DExamples describing the delays experienced by data packets at Node(i) and Node(i+1) include: the corresponding mandatory minimum delays MinDelay(i) and MinDelay(i+1), the corresponding additional delays AddDelay(i) and AddDelay(i+1), the corresponding scheduling delays SchedDelay(i) and SchedDelay(i+1), the corresponding serialization delays SerDelay(i) and SerDelay(i+1), and the corresponding hop delay Hop_Latency(i).

[0062] Figure 4D1 Described with Figure 4D The consistent time period is LATENCY(i).

[0063] Figure 4D2 Describes the method used to determine Figure 4D An exemplary process step of MinDelay(i+1) in the example.

[0064] Figure 5A Is with Figures 4B to 4C2 A flowchart illustrating a consistent, delay-based, exemplary process for forwarding data packets.

[0065] Figure 5B It uses a push-in, first-out (PIFO) buffer for execution. Figure 5A The flowcharts for the exemplary processes of steps 503 and 504 are shown below.

[0066] Figure 5C It uses a First-In, First-Out (FIFO) buffer for execution. Figure 5A The flowcharts for the exemplary processes of steps 503 and 504 are shown below.

[0067] Figure 6A Described with Figures 5A to 5C An exemplary format of data packets consistent with the process.

[0068] Figure 6B Described Figure 6A An example format for table field 618 of the data packet.

[0069] Figure 7 Described with Figure 5A Consistent from Figure 3A An example of the process executed by Node(i) and Node(i+1).

[0070] Figure 8 Described with Figure 5A , 5B Consistent with 6A Figure 3AAn example of the process performed by Node(i), where a PIFO buffer is used.

[0071] Figure 9 Described with Figure 5A , 5C Consistent with 6A Figure 3A An example of the process performed by Node(i), where a FIFO buffer is used.

[0072] Figure 10 Described with Figure 8 An example of a consistent PIFO buffer.

[0073] Figure 11 Described with Figure 9 An example of a consistent FIFO buffer. Detailed Implementation

[0074] The invention will now be described with reference to the accompanying drawings, which generally relate to methods and apparatus (also referred to as devices) for managing latency when transmitting data packets on a time-sensitive network. It should be understood that the present embodiments of the invention can be implemented in many different forms, and the scope of the claims should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to make the invention thorough and complete, and to fully convey the inventive concept to those skilled in the art. In fact, the invention is intended to cover alternatives, modifications, and equivalents to these embodiments that are included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following detailed description of the present embodiments of the invention, many specific details are set forth to provide a thorough understanding. However, it will be apparent to those skilled in the art that the present embodiments of the invention can be practiced without these specific details.

[0075] Networks offering deterministic, guaranteed end-to-end latency with minimal latency fluctuations (jitter) and packet loss are useful in many applications. Exemplary applications include virtual reality / augmented reality (VR / AR), which may have strict limitations on maximum motion photon time, such as avoiding dizziness and a diminished user experience, both of which can be caused by longer latency and potentially severely reduce user acceptance. Another example is the haptic internet, which has strict limitations on the latency of tactile feedback, as a lack of control or sluggish control would render many applications infeasible. Further examples include industrial controllers, which may have strict limitations on feedback control loops, as well as applications such as vehicle-to-everything (V2X), remote-controlled robots, drones, and the Internet of Things (IoT) networks. Other examples include robotic collaboration (e.g., robots collaboratively lifting objects) and high-precision measurements (e.g., remote polling at precise intervals).

[0076] Such a network is preferably designed to provide on-time guarantees for data packet delivery. On-time guarantees are an example of service level objectives (SLOs). Figure 1A-1E The diagram illustrates the concept of delay-based packet forwarding for various SLOs. Each diagram describes the allowed delay for sending a packet across the network from a sending network device (or node) to a receiving network device. This can refer to the delay between adjacent nodes in a continuous sequence, or the delay between the sending node that initiates the packet and the end node that acts as the end user of the packet.

[0077] Figure 1A This shows the latency range between the lower bound ("LB") and the upper bound ("UB"). LB and UB are the earliest and latest times that packets are allowed to arrive at their destination, respectively. A non-zero LB can be used to prevent the receiving node from being overloaded by too many packets simultaneously coming from the sending node. UB ensures that the data in the received packet is still timely for the receiving node and that the application using the packet has sufficient responsiveness. However, if the range between LB and UB is too large, packet latency can vary significantly, leading to jitter or other inconsistent performance.

[0078] Figure 1BAn example of a timeliness guarantee is shown, which has an upper bound (UB) but no lower bound (LB) on delay (i.e., LB = 0). In this case, there is no minimum delay requirement, but there is still a requirement not to exceed UB. Because LB = 0, the receiving node may be overloaded by too many packets simultaneously coming from the sending node. This approach may overwhelm the node's buffering capacity, resulting in delays that affect other packets that need to arrive at their destination within the upper bound (UB) or as quickly as possible.

[0079] Figure 1C An example of a minimum latency guarantee is shown, which has a lower bound (LB) but no upper bound (UB) on the latency. In this case, there is a non-zero minimum latency, but no maximum latency. The drawback is that packets may fail to be received, even though the data in the packets is still timely for the receiving node.

[0080] Figure 1D The best-effort model is shown, where the upper bound (UB) is not guaranteed. There is also no minimum LB in this case. This approach may be affected by the aforementioned drawbacks.

[0081] Figure 1EAn example of a timeliness guarantee is shown, where the lower bound (LB) equals the upper bound, or more generally, the time range between LB and UB is relatively narrow. A relatively narrow time range can be, for example, at most 1%, 5%, or 10% of the baseline value. This time range represents the allowed packet latency variation or jitter and can be set as a Quality of Service (QoS) parameter. Allowable jitter may vary depending on the application. For example, in an exemplary medical application, remote surgery with touch feedback may require 3–10 milliseconds of latency with jitter <2 milliseconds. In an exemplary industrial application, a power grid system may require approximately 8 milliseconds of latency with jitter of a few microseconds. For example, a few microseconds might refer to at most 5 or 10 microseconds. In banking applications, high-frequency trading systems may require <1 millisecond of latency and a few microseconds of jitter. In avionics applications, avionics full-duplex switched ethernet (AFDX) networks may require 1–128 milliseconds of latency with jitter of a few microseconds. In automotive applications, advanced driver assistance systems may require latency of 100-250 microseconds, while powertrain chassis control systems may require latency of <10 microseconds, with jitter in the latter case. In infotainment applications, augmented reality systems may require latency of 7-20 milliseconds. Professional audio / video systems may require latency of 2-50 milliseconds, with jitter in both cases in the latter case. Using the system and method described below, data packets can be transmitted from a sending node to a receiving node via one or more intermediate nodes that exhibit the least variation in node-to-node latency.

[0082] As mentioned above, this approach may be ideal in many applications. In this case, data packets have deterministic or predefined delays. This allows data packets to be transmitted in an orderly and predictable manner according to a predefined schedule.

[0083] Some methods that provide timeliness guarantees include the Internet Engineering Task Force (IETF)'s large-scale deterministic IP networks. However, this approach requires time and / or frequency synchronization between clocks in the nodes carrying the packets. Other methods, such as UBS / TSN-ATS, require flow-by-flow signaling between nodes and the controller. UBS / TSN-ATS refers to IEEE 802.1 Time-Sensitive Networking Asynchronous Traffic Shaping (TNS-ATS), which uses urgency-based scheduling (UBS). In these methods, whenever a packet flow is created, eliminated, or a flow parameter is changed, that parameter must be signaled from the controller to every node along the path, resulting in performance impact.

[0084] The technique proposed in this paper addresses the aforementioned and other issues. In one aspect, a desired delay can be implemented for packet transmission between nodes. The desired delay can be, for example, a specified or predetermined delay. To implement the desired delay for each packet, a minimum delay and a corresponding earliest allowed transmission time can be enforced at each node upon receiving the packet. The minimum delay can be carried in the information field of the packet. When the packet is transmitted from a node, a new value for the minimum delay is calculated and stored in the field for use by the next node. The minimum delay can be considered a forced delay because the packet is forced to wait at least this time period before transmission. It is worth noting that the packet does not need to explicitly carry the minimum delay. For example, the packet (in addition to the forced minimum delay) can carry an indication of the additional delay the packet has already experienced, which can then be used to determine the minimum delay at the next node, as further described below. For example, the additional delay can include an additional delay (AddDelay), a scheduling delay (SchedDelay), and a serialization delay (SerDelay), such as... Figure 4D As described in [the document]. For example, additional latency could be... Figure 4D Trans(i)–Tearliest(i), or Trans(i)–Tearliest(i)+SerDelay(i).

[0085] Delay can represent consecutive nodes (e.g.) Figure 3A The time interval between the corresponding processing points in Node(i) and Node(i+1) in the dataset. For example, also refer to... Figure 4C and 4C1 The delay LATENCY(i) can include an additional delay AddDelay(i), the time SchedDelay(i) for scheduling the data packet at Node(i) for transmission, and a minimum delay MinDelay(i+1) enforced at Node(i+1) when the data packet is received. For example, the additional delay could relate to the time taken to transmit the data packet within Node(i). For example, this could be... Figure 3B The time it takes for the data packet to be sent from the delay circuit 425 to the queue 440.

[0086] Scheduling delay can vary depending on factors such as the number of other data packets competing for limited transmission bandwidth at a node.

[0087] When the additional latency and / or scheduling latency at a certain node is relatively large, the minimum latency of the next node decreases accordingly. Conversely, when the additional latency and / or scheduling latency at a certain node is relatively small, the minimum latency of the next node increases accordingly. Therefore, a substantially consistent total latency (or delay) can be achieved between two nodes.

[0088] Latency can be set at each node based on a pre-configured value or based on information carried in the data packet. In some cases, latency can be based on a priority field in the data packet, a hop count index field in the data packet, and / or the input network interface through which the data packet is received at the node.

[0089] Minimum latency can be enforced in various ways. One approach is to calculate the earliest allowed transmission time for a data packet upon receipt. Another approach uses a countdown timer.

[0090] In one approach, the packet delay is minimized for a short period of time, and then the packet is scheduled for transmission while competing with other packets for access to the output network interface. For example, the packet can be stored in a buffer with a dequeue time corresponding to the earliest allowed transmission time. Once the dequeue time is reached, the packet is scheduled for transmission on the output network interface.

[0091] Figure 2 An exemplary communication system 100 for transmitting data is illustrated. Embodiments of this technology can be used in communication system 100, but are not limited thereto. For example, communication system 100 includes user equipment 110A, 110B, and 110C, radioaccess network (RAN) 120A and 120B, core network 130, public switched telephone network (PSTN) 140, Internet 150, and other networks 160. Additional or alternative networks include private and public packet networks, including corporate intranets. Although a number of these components or elements are shown in the figures, any number of these components or elements may be included in system 100.

[0092] In one embodiment, the communication system 100 may include a wireless network, which may be a fifth-generation (5G) network. The 5G network includes at least one 5G base station that communicates with user equipment using orthogonal frequency-division multiplexing (OFDM) and / or non-OFDM with a transmission time interval (TTI) of less than 1 millisecond (e.g., 100 or 200 microseconds). The base station may be an evolved NodeB (eNB) in a fourth-generation (4G) long-term evolution (LTE) network, or a next-generation NodeB (gNB) in a fifth-generation (5G) new radio (NR) network. Furthermore, the network may include a network server for processing information received from communication devices via the BS.

[0093] System 100 enables multiple users to transmit and receive data and other content. System 100 may implement one or more channel access methods, such as, but not limited to, code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), or single-carrier FDMA (SC-FDMA).

[0094] User equipment (UE) 110A, 110B, and 110C are used to operate and / or communicate in system 100. For example, a UE may be used to transmit and / or receive wireless or wired signals. Each UE represents any suitable end user equipment and may include (or may be referred to as) such devices as user equipment / device, wireless transmit / receive unit, mobile station, fixed or mobile subscriber unit, pager, cellular phone, personal digital assistant (PDA), smartphone, laptop, computer, touchpad, wireless sensor, wearable device, consumer electronics device, device-to-device (D2D) user equipment, machine-type user equipment or user equipment with machine-to-machine (M2M) communication capability, iPad, tablet computer, mobile terminal, laptop embedded equipped (LEE), laptop mounted equipment (LME), USB converter, or other non-limiting examples of user equipment.

[0095] In the described embodiments, RAN 120A and 120B each include one or more base stations (BS) 170A and 170B. Each BS is used to establish a radio connection with one or more UEs among a plurality of UEs to enable access to the core network 130, PSTN 140, Internet 150, and / or other networks 160. For example, a BS may include a base transceiver station (BTS), a Node-B (NodeB), an evolved NodeB (eNB), a next-generation NodeB (gNB), a home NodeB, a home eNodeB, a site controller, an access point (AP), or a wireless router, or one or more servers, routers, switches, or other processing entities in a wired or wireless network.

[0096] In one embodiment, BS 170A is part of RAN 120A, which may include one or more other BSs, elements, and / or devices. Similarly, BS 170B is part of RAN 120B, which may include one or more other BSs, elements, and / or devices. Each of the BSs is used to transmit and / or receive radio signals within a specific geographic area or region (sometimes referred to as a "cell"). In some embodiments, multiple-input multiple-output (MIMO) technology may be used, with each cell having multiple transceivers.

[0097] BS 170A and 170B communicate with one or more UEs among UEs 110A-110C via one or more air interfaces using wireless communication links. These air interfaces can use any suitable wireless access technology.

[0098] The BS and UE are used to implement the LTE communication standard (LTE), LTE Advanced (LTE-A), and / or LTE multimedia broadcast multicast service (MBMS). In other embodiments, base stations 170A and 170B and user equipment 110A-110C are used to implement Universal Mobile Telecommunications Service (UMTS), High Speed ​​Packet Access (HSPA), or HSPA+ standards and protocols. Of course, other multiple access schemes and radio protocols can also be used.

[0099] The RAN communicates with the core network 130 to provide voice, data, application, voice over internet protocol (VoIP), or other services to UEs 110A-110C. The RAN and / or core network 130 can communicate directly or indirectly with one or more other RANs (not shown). The core network 130 can also serve as a gateway access for other networks (e.g., PSTN 140, Internet 150, and other networks 160). Additionally, some or all of the UEs in UE 110 may include the ability to communicate with different wireless networks via different wireless links using different wireless technologies and / or protocols.

[0100] The RAN may also include millimeter and / or microwave access points (APs). An AP may be part of a BS or located remotely from the BS. An AP may include, but is not limited to, a connection point (mmW or millimeter-wave CP) or a BS capable of mmW communication (e.g., an mmW base station). An mmW AP may transmit and receive signals in a frequency range, such as 24 GHz to 100 GHz, but is not required to operate across this entire range. As used herein, the term "base station" refers to a base station and / or a wireless access point.

[0101] Although Figure 2 An example of a communication system is shown, but various changes can be made to the system. For example, communication system 100 may include any number of user equipment, base stations, networks, or other components of any suitable configuration. Furthermore, the term "user equipment" can refer to any type of wireless device that communicates with a wireless network node in a cellular or mobile communication system.

[0102] Networks 130, 140, 150, and / or 160 may be packet-switched networks that transmit formatted data units in data packets between nodes, as further described below. The embodiments presented below relate to sending such data packets on a network and managing the latency of such transmissions.

[0103] Figure 3A An exemplary network 300 comprising a series of nodes is shown. The nodes include Node(i–1) to Node(i+3). In one method, a data packet is transmitted from Node(i–1) as a sending node and received at Node(i+3) as a receiving node. The data packet passes sequentially through Node(i), Node(i+1), and Node(i+2) before arriving at Node(i+3).

[0104] For example, the sending and receiving nodes can be servers or other terminal computing devices. The intermediate nodes Node(i) to Node(i+2) can be routers, network switches, servers, or other networking devices, which forward data packets from the sending node to the receiving node in sequence, one node at a time.

[0105] Data links connect each pair of consecutive nodes. For example, data link L1 connects Node(i–1) and Node(i), data link L2 connects Node(i) and Node(i+1), data link L3 connects Node(i+1) and Node(i+2), and data link L4 connects Node(i+2) and Node(i+3). Each data link may include a wired or wireless communication path. In one approach, each link has a corresponding known delay. However, the link delay is not necessarily known for the techniques disclosed herein to be used. The propagation time of consecutive data packets on the link may vary by a small, known amount, such as a change of <1% relative to the baseline delay. For example, the propagation time may be... Figure 4D It is described as Prop.

[0106] The network control / management plane 310 can be communicatively coupled to each node, as shown by the dashed line. The control / management plane can be used to perform various control path and / or control plane functions, such as implementing routing and label distribution protocols for forwarding packets.

[0107] In a series of nodes traversed by a data packet, you can define the current node, the previous node, and the next node. The current node represents the node currently processing the data packet. The previous node is the node from which the current node receives data packets. The next node is the node to which the current node sends data packets.

[0108] Figure 3B This shows node 400 (e.g.) Figure 3A This is a schematic diagram illustrating exemplary details of any node in the network. This node can be used to implement embodiments of the technology disclosed herein. Node 400 may include multiple input network interfaces 410a, 410b, and 410c and an output network interface 430. Multiple output network interfaces are also possible. In this configuration, multiple traffic flows can be received simultaneously on different input network interfaces, leading to access contention for the output network interfaces. Each input network interface may be identified by a corresponding INTERFACE index; for example, in this case, the INTERFACE indices of input network interfaces 410a, 410b, and 410c are 1, 2, or 3, respectively.

[0109] In a packet-switched network, a service flow can refer to a series of related data packets sent from a sending node to one or more receiving nodes. For example, a service flow may include data packets in transport connections such as Transmission Control Protocol (TCP) or User Datagram Protocol (UDP). A service flow can also refer to a series of related data packets in a media stream, such as a video or audio stream. The service flow to which a data packet belongs can be identified from one or more information fields in the data packet (usually in the packet header). For example, see... Figure 6A The business flow ID field is 619.

[0110] Although the input and output network interfaces are shown as separate input and output sections, each interface can be used for either input or output operations. Similarly, although separate receiver 412 and transmitter 450 are described, these functions can be combined into a transceiver.

[0111] The data packets received at the input network interface are provided to receiver 412 and then to processor 420. The receiver can perform tasks such as deserializing the data packets from a series of bits into a data structure that can be stored in storage component 422.

[0112] The processor 420 may include one or more processing circuits 421 and a storage component 422. In one method, one processing circuit is disposed on an inlet interface circuit board, and another processing circuit is disposed on an outlet interface circuit board.

[0113] Storage component 422 may be based on available memory technologies and may include cache 422a, such as volatile RAM like SRAM or DRAM, and long-term storage component 422b, such as non-volatile memory like flash NAND memory or other memory technologies. Storage component 422 can be used to store data and instructions for implementing the packet forwarding techniques described herein. The storage component may be an example of a non-transitory storage device or a non-transitory memory.

[0114] The processing circuitry can be used to execute instructions to perform the processing described herein.

[0115] Other components implemented by processor 420 include clock 423, delay circuitry 425, and scheduler 426. Delay circuitry 425 can enforce a minimum delay on a data packet before it can be sent to the next node. In one approach, delay circuitry enforces minimum delay by storing data packets before the earliest allowed transmission time (Tearliest). For example, delay circuitry may include a PIFO buffer. A push-in, first-out (PIFO) buffer or queue is a priority queue data structure that allows data packets to be "pushed" or enqueued to any position in the queue, but only dequeued from the head of the queue. For delay circuitry, data packets can be scheduled to be dequeued at times corresponding to the desired delay.

[0116] The data packet can then be sent to one of the queues in queue 440 for scheduling to be output on output I / F 430. The data packet is stored in the queue before the output I / F is freed, and the data packet has a transmission priority. This is an example of a two-stage queuing process. In another approach, the delay circuit 425 is not used, and the data packet is stored in one of the queues in queue 440, where the data packet waits for Tearliest, and can then wait further until the output I / F is freed, and the data packet has a transmission priority. This is an example of a single-stage queuing process.

[0117] Queues can include one or more queues, also known as buffers, such as push-in, first-out (PIFO) or first-in, first-out (FIFO) queues. See also Figure 10 and 11 Queues typically store packets from multiple input traffic flows before they can be transmitted using scheduling techniques implemented by the scheduler 426. Various scheduling techniques can be used, such as round-robin if traffic flows have the same priority, or weighted round-robin if traffic flows have different priorities. Other more advanced scheduling techniques are also possible. The scheduler can be a real-time per-packet scheduler that releases packets no earlier than the earliest allowed transmission time.

[0118] The transmitter 450 can perform tasks such as serializing data packets into a series of bits for transmission over a data link.

[0119] According to some embodiments, storage component 422 may include non-transitory memory storing computer-readable instructions that are executed by processor 420, or typically by one or more processors, to implement embodiments of the present technology. Embodiments of the present technology described below may also be implemented at least in part using hardware logic components, such as, but not limited to, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOC) systems, complex programmable logic devices (CPLDs), dedicated computers, etc.

[0120] exist Figures 4A to 4D2 In the middle, the horizontal direction represents the time when the data packet is stored on the node on the common time stamp.

[0121] Figure 4A Examples describing the delays experienced by packets at Node(i) and Node(i+1) include: the corresponding scheduling delays SchedDelay(i) and SchedDelay(i+1), and the corresponding hop delay Hop_Latency(i) ≤ MAX + unscheduled delay.

[0122] At Node(i), a packet is received at arrival time Tarr(i) and transmitted at transmission time Ttrans(i). Subsequently, at Node(i+1), a packet is received at arrival time Tarr(i+1) and transmitted at transmission time Ttrans(i+1), and so on. The scheduling delay is known or assumed to be less than or equal to the maximum value MAX. In the example below, using MinDelay, if the scheduling delay of Node(i) is too large, MinDelay(i+1) of Node(i+1) can be set to 0, but this is insufficient to compensate for the scheduling delay, and therefore the required node-to-node delay cannot be achieved. In this case, packets may be dropped.

[0123] Scheduling delay SchedDelay(i) represents the time a packet waits to be sent on the output network interface, such as due to contention with other packets. SchedDelay(i) becomes longer when traffic is heavy. This results in a smaller MinDelay(i+1), as discussed later. Figure 4BAs described, SchedDelay(i+1) is generally independent of SchedDelay(i). For example, in one possible scenario, a competing traffic flow at Node(i) that results in a higher SchedDelay(i) will not affect SchedDelay(i+1). Instead, other competing traffic arriving at Node(i+1) will affect SchedDelay(i+1).

[0124] In this scenario, the hop delay is less than or equal to the sum of the MAX and the unscheduled delay, which includes, for example, serialization delay and the propagation time of data packets on the link between nodes. This is as follows: Figure 1B An example of timely guarantee is shown.

[0125] Figure 4B Examples describing the delays experienced by data packets at Node(i) and Node(i+1) include: the corresponding mandatory minimum delays MinDelay(i) and MinDelay(i+1), the corresponding scheduled delays SchedDelay(i) and SchedDelay(i+1), and the corresponding hop delay Hop_Latency(i) = MAX + unscheduled delay. This is as follows... Figure 1E The example of a time-guarantee shown is where the hop delay equals the sum of MAX and the unscheduled delay. The unscheduled delay refers to the delay or delay component of Hop_Latency(i) other than ScheduledDelay(i), and may include SerDelay, packet propagation time between nodes, and other possible delays.

[0126] To achieve this, when a data packet is received, a minimum delay, MinDelay(i), is enforced before allowing transmission of the packet. The minimum delay can be set by the node from which the data packet was received and can be read or extracted from the information field in the data packet header. See, for example, [link to relevant documentation]. Figure 6AField 613. For example, for delayed data packets, the packets can be stored in one of the queues in delay circuit 425 and / or queue 440. Typically, the packet storage continues for a period corresponding to the minimum delay without further processing. However, during this minimum delay, some processing can be performed on the packets, such as updating the header used for routing the packets. The packets are not scheduled for transmission until after the minimum delay has elapsed. At Node(i), Tearliest(i) represents the earliest allowed transmission time of the packet, obtained by adding the arrival time Tarr(i) to MinDelay(i). At Node(i+1), Tearliest(i+1) represents the earliest allowed transmission time of the packet, obtained by adding Tarr(i+1) to MinDelay(i+1), and so on.

[0127] Scheduling delay represents the time a data packet waits to be sent on the output network interface, such waiting being due to contention with other data packets. This delay may initially be unknown, but the transmission time, Ttrans(i) in Node(i) or Ttrans(i+1) in Node(i+1), can be determined as the current time when the data packet is ready to be transmitted. A data packet is ready to be transmitted when, for example, it is ready to be sent and no other higher-priority data packets are waiting to be transmitted. Typically, a data packet is ready to be sent when the earliest allowed transmission time is reached or sometime thereafter.

[0128] In this example, MinDelay(i+1) is less than MinDelay(i). This illustrates how the minimum delay can vary across consecutive nodes traversed by a packet. Typically, the minimum delay of a given node is adjusted to account for the Ttrans–Tearliest time interval of the previous node from which it received the packet. The smaller the minimum delay of the given node, the larger the Ttrans–Tearliest time interval of the previous node. In other words, the minimum delay of a given node is a decreasing function of the Ttrans–Tearliest time interval of the previous node.

[0129] Figure 4B1 Described with Figure 4B The consistent time period is MAX. MAX = ScheduledDelay(i) + MinDelay(i+1) + NSL (non-scheduled delay). Conceptually, this can be viewed as the required delay for each hop, where the ScheduledDelay of early nodes is unpredictable, and the MinDelay of later nodes is adjusted to offset the variation in ScheduledDelay.

[0130] Figure 4B2 Describes the method used to determine Figure 4BThe exemplary process steps for MinDelay(i+1) are as follows. These steps include directly or indirectly measuring SchedDelay(i) on Node(i). The next step includes signaling to Node(i+1) in a packet (e.g., in a field of the packet further described herein) that MinDelay(i+1) = MAX – the measured SchedDelay(i). Another step includes enforcing the delay MinDelay(i+1) on Node(i+1).

[0131] Figure 4C Examples describing the delays experienced by data packets at Node(i) and Node(i+1) include: the corresponding mandatory minimum delays MinDelay(i) and MinDelay(i+1), the corresponding additional delays AddDelay(i) and AddDelay(i+1), the corresponding scheduling delays SchedDelay(i) and SchedDelay(i+1), and the corresponding hop delay Hop_Latency(i). As mentioned above, the additional delay is optional and can represent the time taken to send a data packet from one location to another within a node. For example, this could be... Figure 3B The delay is the time it takes for a data packet to be sent from the delay circuit 425 to the queue 440. This delay may be unknown. In this case, the delay circuit itself can implement the queue, so that the data packet is stored in two consecutive queues: in the delay circuit 425 and then in the queue 440.

[0132] In an embodiment where the buffer is used to store the minimum duration of packet delay, scheduling can occur immediately thereafter without sending the packet to another location and incurring additional delay.

[0133] For Node(i), time interval t1 is the time between Tearliest(i) and Ttrans(i). Time interval t1 represents AddDelay(i) + ScheduleDelay(i), or Ttrans(i) – Tearliest(i). t1 is an example of an additional delay, which is the delay after MinDelay. For Node(i+1), time interval t2 is the time between Tarr(i+1) and Tearliest(i+1).

[0134] Hop_Latency(i) = AddDelay(i) + ScheduleDelay(i) + MinDelay(i+1) + Unscheduled delay.

[0135] Figure 4C1 Described with Figure 4CThe consistent time interval is LATENCY(i). LATENCY(i) = AddDelay(i) + ScheduleDelay(i) + MinDelay(i+1). Similar to... Figure 4B1 The MAX in this context can be conceptually considered as the required latency (or delay) per hop.

[0136] Figure 4C2 Describes the method used to determine Figure 4C The following are exemplary process steps for MinDelay(i+1). These steps provide computation on Node(i) once Ttrans(i) is known. The first step computes t1 = AddDelay(i) + ScheduleDelay(i) = Ttrans(i) – Tarr(i) – MinDelay(i), and the second step computes MinDelay(i+1) = LATENCY(i) – AddDelay(i) – ScheduleDelay(i) = LATENCY(i) – t1 = LATENCY(i) – (Ttrans(i) – Tarr(i) – MinDelay(i)). Therefore, MinDelay(i+1) equals LATENCY(i) minus the additional delay. The goal is to achieve Hop_Latency(i) = LATENCY(i) whenever (AddDelay(i) + ScheduleDelay(i)) ≤ LATENCY(i).

[0137] SchedDelay(i) and AddDelay(i) are unknown initially, but the (possible) maximum value of LATENCY(i) can be calculated. Typically, LATENCY(i) is large enough that LATENCY(i) > (Ttrans(i) – Tarr(i) – MinDelay(i)). This ensures that MinDelay(i+1) ≥ 0 is possible, while simultaneously achieving Hop_Latency(i) = LATENCY(i). Therefore, this technique achieves the goal of creating a fixed Hop_Latency(i) for each packet as LATENCY(i). The formula for t1 calculates AddDelay(i) + SchedDelay(i) based on the measured values ​​of Tarr(i) and Ttrans(i). Then, MinDelay(i+1) can be calculated and sent to Node(i+1).

[0138] Figure 4DExamples describing the delays experienced by data packets at Node(i) and Node(i+1) include: the corresponding mandatory minimum delays MinDelay(i) and MinDelay(i+1), the corresponding additional delays AddDelay(i) and AddDelay(i+1), the corresponding scheduling delays SchedDelay(i) and SchedDelay(i+1), the corresponding serialization delays SerDelay(i) and SerDelay(i+1), and the corresponding hop delay Hop_Latency(i).

[0139] In this approach, the serialization delay SerDelay(i) specifies the time it takes to send a data packet on the link between nodes. This approach can be useful, for example, when the nodes' output network interfaces have different transmission rates. A node can determine SerDelay(i) based on the packet size (length) (e.g., in bits) divided by the transmission rate of the output network interface (in bits per second). In another approach, SerDelay(i) can be determined based on the packet size of each data packet. The size can be read from fields in the packet. For example, IP packets have a total length field. This is a 16-bit field that indicates the total packet size in bytes, including the header and payload data.

[0140] like Figure 4C As shown, for Node(i), time interval t1 is the time between Tearliest(i) and Ttrans(i). Time interval t1 represents AddDelay(i) + SchedDelay(i), or Ttrans(i) – Tearliest(i). For Node(i+1), time interval t2 is the time between Tarr(i+1) and Tearliest(i+1).

[0141] Hop_Latency(i) = AddDelay(i) + SchedDelay(i) + SerDelay(i) + Prop. + MinDelay(i+1). Prop. is the propagation time, which is the time it takes for the data packet to be sent on the link between nodes after being serialized.

[0142] Figure 4D1 Described with Figure 4D The consistent time period is LATENCY(i). LATENCY(i) = AddDelay(i) + SchedDelay(i) + SerDelay(i) + MinDelay(i+1).

[0143] Figure 4D2 Describes the method used to determine Figure 4DThe following are exemplary process steps for MinDelay(i+1). These steps provide computation on Node(i) once Ttrans(i) is known. The first step computes t1 = AddDelay(i) + ScheduleDelay(i) = Ttrans(i) – Tarr(i) – MinDelay(i), and the second step computes MinDelay(i+1) = LATENCY(i) – AddDelay(i) – ScheduleDelay(i) – SerDelay(i) = LATENCY(i) – t1 – SerDelay(i) = LATENCY(i) – (Ttrans(i) – Tarr(i) – MinDelay(i)) – SerDelay(i). The goal is to achieve Hop_Latency(i) = LATENCY(i) whenever (AddDelay(i) + ScheduleDelay(i) + SerDelay(i)) ≤ LATENCY(i).

[0144] Suppose Ttrans(i) is the time when the packet begins serialization to the output network interface, i.e., the first bit of the packet has been transmitted, and Tarr(i+1) is the time when the packet completely arrives at Node(i+1), i.e., the last bit of the packet has arrived. In this case, by considering the serialization delay, the serialization delay is longer for larger packets, and the goal of achieving delay LATENCY(i) can be met even if the packets in the traffic flow have different packet sizes. Alternatively, if Ttrans(i) is the timestamp when the last bit of the packet has been serialized to the output network interface, then... Figure 4C For example, Ttrans can be obtained by adding SerDelay(i) to the current time at the start of serialization.

[0145] Figure 5A Is with Figures 4B to 4C2 A flowchart illustrating a consistent, delay-based packet forwarding process. The processing is discussed in conjunction with an exemplary node (i). Step 500 includes, at Node (i), the input network interface (I / F) (e.g., Figure 3B In step 410a), a data packet is received from the previous node Node(i–1), and the corresponding arrival time Tarr(i) is stored. Step 501 includes extracting (e.g., reading) the minimum delay MinDelay(i) from the information field in the data packet. For example, see... Figure 6AField 613. In one approach, if MinDelay(i) < 0, the packet may be dropped. This can happen when Ttrans–Tearliest is too long on the previous node. Dropped packets can be detected by an error detection process and retransmitted by the sending node. Alternatively, if MinDelay(i) might be less than 0, the delay can be optionally set to zero in the packet. Similarly, a negative MinDelay(i) may cause the next node to reduce MinDelay(i+1) accordingly, making it possible for the packet to make up for the previous excessive delay.

[0146] Step 502 involves calculating the earliest allowed transmission time of the data packet as Tearliest(i) = Tarr(i) + MinDelay(i). One possible approach is to calculate Tearliest(i) as a clock time. For example, this clock time could be a time of day or an amount of time since the node was activated. The node then ensures that the data packet is saved at least until this time is reached. Step 503 involves delaying further processing (e.g., scheduling) of the data packet at least until Tearliest(i).

[0147] In one approach, one or more processors execute instructions after step 503 to enqueue packets into a pre-existing scheduler, which is responsible for executing step 504. For example, the pre-existing scheduler could be a simple queue.

[0148] Instead of calculating Tearliest(i), a countdown timer can be started, which will count down from MinDelay(i) to zero, at which point data packet transmission is permitted. Therefore, steps 502 and 503 more generally relate to enforcing the earliest permitted transmission time for data packets.

[0149] Step 504 involves scheduling data packets to the output network I / F (e.g., Figure 3B The transmission of 430) and the determination of the transmission time Ttrans(i) are used as the determined transmission time. As mentioned above, scheduling can involve performing a process that allows data packets to access the output network I / F. The scheduling process can arbitrate access to the output network I / F from competing data packets in the corresponding competing traffic flow. That is, the scheduling process schedules the access of data packets in the competing data packets to the output network interface.

[0150] For example, scheduling can identify the specific time a data packet is transmitted as the time when the packet is due for transmission on the output network I / F. At this time, the output network I / F is available because no other data packets are being transmitted, and the packet has a higher priority than other competing data packets.

[0151] In one approach, scheduling occurs whenever the outgoing interface has just finished serializing the previous packet and the scheduler now needs to find the next packet. At this time, the scheduler searches for a PIFO or FIFO that best satisfies one or more of the conditions described herein (e.g., first-rank and highest priority).

[0152] However, this decision can also be made earlier than the previous packet's serialization. In this approach, the scheduler selects the packet most suitable for transmission at a given time, but this packet may not be the best when the previous packet's serialization is complete, because another higher-priority packet may arrive at the same time.

[0153] Step 504 can be in response to packet priority, as discussed further below.

[0154] Step 505 includes, when Ttrans(i) is known, calculating a new minimum delay value MinDelay(i+1) based on LATENCY(i) – (Ttrans(i) – Tarr(i) – MinDelay(i)), such as combining Figure 4C2 As discussed above. In one approach, Ttrans(i) is known at the moment the data packet is ready to be sent, such as when the data packet begins serialization. Ttrans(i) could be the current time when the data packet begins serialization. In another approach, Ttrans(i) is known before the data packet is ready to be sent. This could happen, for example, when the scheduler pre-sets a specific time for transmitting data packets.

[0155] Step 505 may include determining a transmission time for the data packet that is no earlier than the earliest allowed transmission time. This may include, for example, recording the time on a clock when the data packet is ready to be sent, or determining the time when the data packet is ready to be sent without necessarily recording the corresponding time on a clock.

[0156] As discussed, LATENCY(i) is the required latency in the processing at Node(i) and Node(i+1). See also Figure 7 For example, in the case of LATENCY(i) = t1 + t2, where t1 is the processing time at Node(i) and t2 is the processing time at Node(i+1).

[0157] In one approach, LATENCY is no greater than the sum of the burst size of each packet and the maximum packet size across multiple traffic flows, divided by the rate at which packets are serialized to the output network interface. The burst size refers to the amount of data allowed to be transmitted at the peak bandwidth rate. For example, suppose the packet size for each i-th traffic flow is b. iThe unit digit, m, represents the maximum packet size. At a given time, packets from each traffic flow are buffered on the node, while another packet is pre-serialized for transmission. LATENCY is the time required to transmit both the packets from each traffic flow and the pre-serialized packets. The serialization rate (SR) is applied to ∑ i sum(b i ) + m units, therefore LATENCY ≤ (∑ i sum(b i )+m) / SR.

[0158] In another method of step 505, such as combining Figure 4D The discussion considers the serialization time, such that MinDelay(i+1) is calculated based on LATENCY(i)–(Ttrans(i)–Tarr(i)–MinDelay(i))–SerDelay(i).

[0159] Alternatively, in another approach, the additional delay for the current node can be calculated and inserted into the information field before the data packet is sent to the next node. As mentioned above, the next node can then use this additional delay to determine the minimum delay required by that node. For example, MinDelay(i+1) can be calculated by subtracting the additional delay from LATENCY(i) by the next node Node(i+1).

[0160] Step 506 involves storing MinDelay(i+1) (or additional delay, as described in the previous paragraph) in the information field of the data packet and sending the data packet to the next node Node(i+1) on the output network I / F. Therefore, the information field is updated with the new value of the minimum delay.

[0161] Figure 5B It uses a push-in, first-out (PIFO) buffer for execution. Figure 5A Flowcharts of exemplary procedures for steps 503 and 504 are provided. These steps involve enforcing the earliest allowed transmission time for data packets. See also... Figure 8 Exemplary implementation methods and Figure 10 An example buffer in the example. Step 510 includes determining the packet priority based on the packet's priority field and / or hop count index field. For example, see... Figure 6A Fields 614, 616, and 618 in the data packet field. A default priority can also be used, for example, when no packet field indicates a different priority. Step 511 includes selecting a PIFO buffer with a priority based on packet priority. For example, Figure 10The buffers have priorities from K to 1, with 1 being the highest priority. Buffers can be selected from multiple available buffers with different priorities and optionally corresponding to different LATENCY values. This allows different traffic flows and / or data packets to be processed with different priorities. Higher-priority packets will be sent from the node to the output network interface earlier than lower-priority packets.

[0162] Step 512 includes enqueuing the packet into the buffer at a scheduled dequeue time, which is also Tearliest(i), where Tearliest(i) = Tarr(i) + MinDelay(i). The scheduled dequeue time is the time when the packet can be removed from the buffer and transmitted if the output network interface is available. For example, the scheduled dequeue time can be stored as a timestamp associated with the packet. Step 513 includes determining, at the scheduled dequeue time, whether the packet has, for example, a higher transmission priority than other competing packets.

[0163] If step 514 is true (T), for example, the data packet has a priority, then step 515 is reached, where the data packet is ready to be sent, and a time Ttrans(i) is determined for transmission. Figure 5A It is used in subsequent step 505. Alternatively, Ttrans(i) can be determined in advance, as described above. If step 514 is determined to be false, for example, the packet has no priority, the implementation waits until the packet does indeed have a priority.

[0164] Figure 5C It uses a First-In, First-Out (FIFO) buffer for execution. Figure 5A Flowcharts of exemplary procedures for steps 503 and 504 are provided. These steps involve enforcing the earliest allowed transmission time for data packets. See also... Figure 9 Exemplary implementations and Figure 11 An example buffer in the example. Step 520 includes determining the packet priority based on the packet's priority field. For example, see... Figure 6A Field 614 in the table. Step 521 includes selecting a priority FIFO buffer based on a combination of packet priority and / or packet input network I / F. For example, Figure 11 The buffers in the buffer have priorities K to 1 for each of the different input network interfaces 1 to Ex. For example, the identifier (INTERFACE index) of the input network interface can be recorded by the processor.

[0165] For a given priority in the priority field of a data packet, a buffer can be selected from multiple different buffers, which can be associated with different input network interfaces. This allows different traffic flows and / or data packets to be processed with different priorities and corresponding LATENCY values.

[0166] Step 522 includes enqueuing the data packet into the buffer at a scheduled dequeue time, which is also Tearliest(i), where Tearliest(i) = Tarr(i) + MinDelay(i). Step 523 includes determining whether the data packet has a transmission priority at the scheduled dequeue time.

[0167] If step 524 is true (T), for example, the data packet has a priority, then step 525 is reached, where the data packet is ready to be sent, and a time Ttrans(i) is determined for transmission. Figure 5A It is used in subsequent step 505. Alternatively, Ttrans(i) can be determined in advance, as described above. If step 524 is determined to be false, for example, the packet has no priority, the implementation waits until the packet does indeed have a priority.

[0168] Figure 6A Described with Figures 5A to 5C The data packet follows an exemplary format consistent with the process. Data packet 600 includes a header 610 and a payload 630. For example, the data packet may conform to the Internet Protocol (IP). The header may include various fields, including one or more of fields 611-619. Each field can be provided by allocating a specified number of bits. In some embodiments, some fields may not be needed, while additional fields, not shown, may be included in the data packet and used.

[0169] Field 611 provides the source address, which is the network address of the sending node. Field 612 provides the destination address, which is the network address of the destination node. Field 613 provides the minimum latency to be implemented at the node. As discussed herein, in some embodiments, field 613 may alternatively provide the latency the packet has already experienced at the previous node, which can then be used to calculate the minimum latency at the next node. Field 614 provides the packet priority, which can be used to select the buffer that enforces the minimum latency, where the priority corresponds to the latency. Field 616 provides the hop count index (hc), which is the number of hops the packet has made since it was transmitted from the sending node. A hop is the next node in a series of nodes to which the packet is sent. Field 617 provides the maximum hop count (maxhop), which is the maximum number of hops the packet is allowed to make.

[0170] Field 618 provides the table with priority and hop count indexes. For example, see [link to table]. Figure 6B This allows for custom priorities to be defined for each node and each hop of a packet. This allows for fine-tuning of the effective priorities on a packet path. For example, suppose three priorities P1, P2, and P3 are provided, where P1 > P2 > P3. The packet priority can be alternated between P1 and P2 at consecutive nodes to provide an effective priority between P1 and P2. For example: P1 on hop 1 and Node(i), P2 on hop 2 and Node(i+1), P1 on hop 3 and Node(i+2), and so on. Figure 6B As shown. You can choose to truncate the table at each hop, thus removing unnecessary information from subsequent hops.

[0171] Since latency corresponds to priority, latency can be customized for each node and each hop of a packet. For example, suppose three latency levels, Lat1, Lat2, and Lat3, are provided, corresponding to P1, P2, and P3 respectively, where Lat1 > Lat2 > Lat3. The latency of a packet can alternate between Lat1 and Lat2 at consecutive nodes to provide an effective latency between Lat1 and Lat2. For example: Lat1 at hop 1 and Node(i), Lat2 at hop 2 and Node(i+1), Lat1 at hop 3 and Node(i+2), and so on.

[0172] In another option, the packet priority and latency are fixed at each of the nodes the packet passes through.

[0173] Typically, when a sending node creates a new data packet, it can determine the time budget for sending the packet to the end node and accordingly determine the latency and / or priority of each node. In embodiments using a set of PIFO or FIFO buffers, a buffer can be selected based on the packet's latency and / or priority.

[0174] The sending node can use the service flow ID field 619 to identify the service flow to which the data packet belongs. As discussed, in embodiments of this technology, latency guarantees do not depend on this field. Typically, a data packet may contain one or more fields that together constitute a service flow identifier, the value of which is unique to data packets within a service flow.

[0175] Payload 630 includes data 631. The data is used by the application and can have a fixed or variable length in different data packets.

[0176] Figure 6B Described Figure 6AAn exemplary format of table field 618 for the data packet. This table includes values ​​for hop count indices (hc), such as 1, 2, and 3, and corresponding priorities P1, P2, and P3, as previously discussed. In one method, one or more processors execute instructions to determine a priority based on a sequence of per-hop priority information fields (e.g., P1-P3) and the hop count index (field 616) in the data packet. The index value received after the data packet is received determines the index of an item in the priority sequence, which is then used as the PRIORITY value. The hop count index in the data packet is incremented before the data packet is sent to the next hop.

[0177] Alternatively, some information from the packet fields (e.g., Table 618) can be used, for example, to pre-configure the node before receiving the packet.

[0178] Figure 7 Described with Figure 5A Consistent from Figure 3A An example of the process executed by Node(i) and Node(i+1). In input I / F 410a( Figure 3A The system provides data packets. Process 702 receives and forwards the data packets. Receiving may include deserializing the data packets, as discussed. Forwarding may involve routing functions, such as looking up the address of the next node to which the data packet should be sent and updating the corresponding fields in the data packet.

[0179] Procedure 703 stores the arrival time of the data packet, Tarr(i) = tnow, where tnow is the current time provided by clock 423 (see also...). Figure 3B ).

[0180] In this example, step 703 is executed after step 702. However, the order may differ. In a distributed node, where packets can be processed by different processors, step 702 can be executed on a separate processor, such as on the ingress interface board, while the remaining processing at the node for steps 703 and the following steps is executed on the egress interface board. By executing step 703 after step 702, only the egress interface board must be modified to implement the techniques discussed herein. Furthermore, the ingress and egress interface boards can have separate clocks, allowing Tarr(i) to be obtained in step 703 using the clock on the egress interface board and providing greater consistency for Ttrans(i) in step 706.

[0181] If, in certain circumstances, step 702 introduces a significant variable delay, such as an unknown but bounded delay in sending data packets from the ingress interface board to the egress interface board, it is advantageous to alternatively perform step 703 in the ingress interface board before step 702. In this case, a modified flow can be provided where step 702 changes from the case of receiving but not forwarding data packets as shown, step 703 stores Tarr(i) = tnow as shown, and another step is inserted between steps 703 and 704 to forward data packets, step 704 as shown. This may be more efficient than the original flow when forwarding has a variable delay. Figure 7 An error occurs in the stream. In the modified stream, the variable delay is compensated by step 704, which is performed relative to the timestamp of step 703. This assumes that steps 703 and 706 use the same clock.

[0182] In one approach, means for providing nodes in the network described herein may include a separate egress board or a combination of an egress board and an ingress board.

[0183] Procedure 704 extracts and enforces the minimum delay MinDelay(i) represented by the parameter pak.delay, which may be included in or determined from a field in the received data packet. As discussed, procedure 705 provides possible additional delays. Procedure 706 schedules the data packet for transmission and determines Ttrans = tnow, i.e., the current time provided by clock 423, at which point the data packet is ready to be transmitted. Procedure 707 calculates a new value for the minimum delay as: pak.delay = LATENCY(i) – (Ttrans(i) – Tarr(i) – pak.delay), updates the delay field with the new pak.delay value, and transmits the data packet. As discussed, transmission may include serializing the data packet on output I / F 430. Thus, on link L2 between nodes (see also...) Figure 3A The package 710, which includes pak.delay, is provided on the website.

[0184] Node(i+1) performs similar processing to Node(i), but uses a new pak.delay value. A data packet is provided at input I / F 721. Procedure 722 receives and forwards the data packet. Procedure 723 stores the arrival time of the data packet, Tarr(i) = tnow, where tnow represents the current time provided by clock 738. Advantageously, synchronization between the clocks of the nodes is not required, such as synchronization between clocks 423 and 738. Clocks can be unsynchronized in terms of time, phase, and frequency. This avoids the overhead costs involved when such synchronization is required, such as in large-scale deterministic IP networks of the Internet Engineering Task Force (IETF).

[0185] Clocks in a network can be synchronized using various techniques. One technique is the Precision Time Protocol (PTP), in which clocks synchronize with the master clock by exchanging network messages. When node clocks are frequency-synchronized, each node can forward data packets within a specific time slot based on a period identifier carried in the data packet. For asynchronous clocks, such time slots and period identifiers are not required, nor is it necessary to exchange messages with the master clock.

[0186] In one approach, one or more processors execute instructions to determine Ttrans and Tarr based on a clock whose timestamps have only local significance (e.g., within the node). The clock can start counting from when the node powers on, for example, starting from 0.

[0187] Procedure 724 extracts and enforces the minimum delay MinDelay(i+1) represented by the parameter pak.delay. Procedure 725 provides possible additional delays, such as... Figures 4A-4D As described. Procedure 726 schedules the data packet for transmission and determines Ttrans = tnow, i.e., the current time provided by clock 738, at which point the data packet is ready for transmission. Procedure 727 calculates a new value for the minimum delay as: pak.delay = LATENCY(i+1) – (Ttrans(i+1) – Tarr(i+1) – pak.delay), updates the delay field with the new pak.delay value, and transmits the data packet. As discussed, transmission may include serializing the data packet on output I / F 728.

[0188] As described above, LATENCY(i) can be equal to the sum of the following: (a) the packet processing time at Node(i) after process 704 until the packet is transmitted in step 707, and (b) the packet processing time at Node(i+1) in steps 722-724. These are the corresponding processing points of the nodes at the earliest allowed transmission time of the packet, such as Tearliest(i) and Tearliest(i+1). In this example, the example of LATENCY(i) = t1 + t2 does not include the serialization time of the packet on L2, denoted by SerDelay(i). Alternatively, LATENCY(i) can include the serialization time.

[0189] Figure 8 Described with Figure 5A , 5B Consistent with 6A Figure 3A An example of the process executed by Node(i) using a PIFO buffer. See also: Figure 10 The data packet is provided on input I / F 410a. Procedure 702 receives and forwards the data packet. Procedure 703 stores the arrival time of the data packet, Tarr(i) = tnow. Procedure 801 extracts the arrival time of the data packet from pak.delay( Figure 6A The field 613 in the text represents the minimum delay MinDelay(i), and the maximum jump field (maxhop, Figure 6A Field 617 in the data), hop count index (hc, Figure 6A Field 616 in the table and the table of hc and priority (hopprio, Figure 6A (Table field 618 in the table).

[0190] Procedure 802 uses a set of per-priority PIFO queues to delay and schedule packets. Procedure 707 calculates the new minimum delay value as: pak.delay = LATENCY(i+1) – (Ttrans(i) – Tarr(i) – pak.delay), updates the delay field with the new pak.delay value, and transmits the packets as discussed. Procedure 803 provides further details of procedures 802 and 707.

[0191] Transmission may include serializing data packets on output I / F 430. Thus, data packet 710a is provided on link L2, whose fields include 1) pak.delay and 2) per-hop priority data, such as maxhop, hc and hopprio [hc = 1...maxhop].

[0192] In the detailed process 803, the command sets the priority value k = pak.hopprio[pak.hc], where k can range from 1 to K. That is, Figure 6B the table field 618 represented by hopprio in Figure 6B is read using the current hop count index pak.hc of the data packet.

[0193] If the next command (whether (pak.hc < pak.maxhop) pak.hc++) does not reach the maximum allowed hop count maxhop, the next command increments the hop count index by one hop.

[0194] The next command sets the rank of the data packet to: pak.rank = tnow + pak.delay. This rank is the Tearliest(i) value.

[0195] Then, the data packet is enqueued into pifo[k], which represents the PIFO corresponding to the priority k, and its rank is represented by pak.rank. As described above, a specific k-th PIFO buffer is selected from multiple available PIFO buffers based on the priority. K PIFO buffers in the range from PIFO K with the lowest priority and the highest maximum delay pifo[K].max to PIFO 1 with the highest priority and the lowest maximum delay pifo[1].max are described.

[0196] The data packets are dequeued in rank order. The rank can represent the timestamp at which the data packet can be dequeued. The data packet at the PIFO head is always the next timestamp that can be dequeued. When three conditions are met, the data packet is dequeued from the n-th PIFO buffer pifo[n]. First, the output interface is free to receive another data packet, so that no other data packet is currently being transmitted. ("Output I / F free"). Second, the rank of the PIFO is less than the current timestamp "pifo[n].head.rank ≤ tnow", which means that the timestamp Tearliest of the first or head data packet in the PIFO is equal to or older than the current timestamp. In other words, the earliest transmission time of the head data packet in the PIFO has been reached. The head data packet of the PIFO is the next data packet to be released or dequeued from the PIFO. The third condition is that there is no other PIFO with a higher priority where the earliest transmission time of the head data packet has been reached. This is represented by the command: "For all j < n: pifo[j].head.rank ≥ tnow OR empty(pifo[j])".

[0197] Therefore, packets are obtained from the highest priority pifo[1], and only from lower priority PIFOs (e.g., pifo[2], pifo[3], ...) if pifo[1] does not have packets ready to be sent.

[0198] The next command (pak.delay = pixel[k].max – (tnow – pak.rank) updates the minimum delay, which can be, for example, LATENCY(i) – (Ttrans(i) – Tearliest(i)). The term “pifo[i].max” can be the maximum intentional delay time pre-configured by pixel[i]. Specifically, for the highest priority PIFO buffer PIFO[1], each traffic flow has a burst size in bits. The maximum delay for these flows is LATENCY(i). The scheduling delay SchedDelay(i) is the sum of these burst sizes divided by the link rate, that is, the time required for all flows to burst simultaneously and then continuously send all these bursts. This can be expressed by the following equation:

[0199] (1)MAX(1)=sum(priority[1]burst flow) / I / F bit rate;

[0200] (2)MAX(j)=MAX(j–1)+sum(prio[j]burst flow) / I / F bit rate.

[0201] In one embodiment, one or more processors execute instructions to delay data packets by periodically pushing them into a first-out (PIFO) queue, wherein: a) the PIFO level of a data packet corresponds to the earliest transmission time of the sum of the arrival time and the minimum delay extracted from the information field in the data packet; b) the header of the PIFO buffer is served no earlier than its level.

[0202] In another embodiment, one or more processors execute instructions to: a) target Figure 6A Each value of PRIORITY defined in the table uses a separate timing PIFO, b) select the PIFO to which the packet is to be inserted based on PRIORITY, where: c) the PIFO level of the packet is the earliest transmission time, corresponding to the sum of the arrival time and the minimum delay extracted from the information field in the packet, d) the header of each timing PIFO is served no earlier than its level, at which point no header can be served by a higher priority timing PIFO.

[0203] Figure 9 Described with Figure 5A , 5C Consistent with 6A Figure 3A An example of the process executed by Node(i) using a FIFO buffer. See also: Figure 11 The data packet is provided on input I / F 410a. Procedure 702 receives and forwards the data packet. Procedure 703 stores the arrival time of the data packet, Tarr(i) = tnow. Procedure 901 extracts the arrival time of the data packet from pak.delay( Figure 6A Field 613 in the text represents the minimum delay MinDelay(i) and the priority field ( Figure 6A Field 614 in the text.

[0204] Procedure 902 uses a set of per-priority FIFO queues to delay and schedule packets. Procedure 707 calculates the new minimum delay value as: pak.delay = LATENCY(i+1) – (Ttrans(i) – Tarr(i) – pak.delay), updates the delay field with the new pak.delay value, and transmits the packets as discussed. Procedure 903 provides further details of procedures 902 and 707.

[0205] Transmission may include serializing data packets on output I / F 430. Therefore, data packet 710b is provided on link L2, with fields including 1) pak.delay and 2) priority data.

[0206] In detailed procedure 903, the command (e = input I / F or INTERFACE index) sets the parameter e that identifies the input I / F on which the received data packet is located, where e can be in the range of 1 to Ex.

[0207] The next command sets the packet's rank to: pak.rank = tnow + pak.delay. This is the Tearliest(i) value.

[0208] Then, the data packet is enqueued into fifo[k,e], which represents the FIFO corresponding to priority k and input I / F e, whose rank is represented by pak.rank. For each value of priority k, there are Ex FIFO buffers available. Alternatively, for each value of e, K FIFO buffers are described. For example, for e=1, FIFO(K,1) has the lowest priority[K] and the highest maximum delay, and FIFO(1,1) has the highest priority[1] and the lowest maximum delay. For e=Ex, FIFO(K,Ex) has the lowest priority[K] and the highest maximum delay, and FIFO(1,Ex) has the highest priority[1] and the lowest maximum delay.

[0209] As discussed, packets are dequeued in rank order. A packet is dequeued from fifo[k,e] when three conditions are met. First, the output interface is free to receive another packet (“Output I / F Free”). Second, the rank of the FIFO is less than the current timestamp “fifo[k,e].head.rank≤tnow”, meaning that the timestamp of the first or head packet in the FIFO is equal to or older than the current timestamp. In other words, the earliest transmission time of the head packet of the FIFO has been reached. The third condition is that there are no other higher-priority FIFOs where the earliest transmission time of the head packet has been reached. This is indicated by the command: “For all j <k:fifo[j,*].head.rank≥tnow||empty(fifo[j,*)”。

[0210] The next command updates the minimum delay: “pak.delay = priority[pak.priority].max – (tnow – pak.rank)”, which corresponds to LATENCY(i) – (Ttrans(i) – Tearliest(i)). The term “priority[pak.priority].max” can be the maximum intentional delay time pre-configured via fifo[k,e]. Specifically, for the highest priority FIFO, FIFO[1,Ex], each traffic flow has a burst size in bits. The maximum delay for these flows is LATENCY(i). The scheduling delay SchedDelay(i) is the sum of these burst sizes divided by the link rate.

[0211] In one embodiment, one or more processors execute instructions to enqueue packets into a first-in, first-out (FIFO) buffer, where the scheduled dequeue time corresponds to the sum of the arrival time and the minimum delay extracted from the information field in the packet.

[0212] One or more processors also execute instructions to: a) use a separate timing FIFO for each possible combination (PRIORITY, INTERFACE) of the PRIORITY and valid incoming INTERFACE index; b) select the FIFO to which to insert the packet and the interface from which to receive the packet based on the PRIORITY, wherein: c) the FIFO rank of the packet is the earliest transmission time, corresponding to the sum of the arrival time and the minimum delay extracted from the information field in the packet; e) the header of the timing FIFO is served if: i) the header of the FIFO is equal to or earlier than the current time, in which case the current time is equal to or later than the FIFO header rank; ii) there is no FIFO with a higher PRIORITY that can be served; iii) there is no FIFO with the same PRIORITY but a higher header rank.

[0213] Figure 10 Described with Figure 8 An example of a consistent PIFO buffer. As mentioned earlier, a Push-In, First-Out (PIFO) buffer or queue is a priority queue data structure that allows packets to be "pushed" or enqueued at any position in the queue, but only dequeued from the head of the queue. Conversely, packets need to be enqueued at the bottom of a First-In, First-Out (FIFO) queue and dequeued from the head of the queue. PIFO makes packet scheduling more flexible, while FIFO may be easier to implement using existing hardware.

[0214] A set of K buffers 1000 is described, including PIFO[K] to PIFO[1]. Each buffer has a corresponding input path 1003_K to 1003_1 connected to multiplexer 1002 and a corresponding output path 1020_K to 1020_1 connected to demultiplexer 1030. The multiplexer can route packets to a buffer on input path 1001. For example, as combined with Figure 8 The discussion states that, for a given input data packet, one of the K PIFO queues can be selected based on the packet's priority field.

[0215] Each buffer can store multiple data packets. For example, PIFO[1] stores data packets 1010-1017. New data packets received on the input path of the buffer can be stored at any desired position relative to the current data packet in the buffer, as indicated by arrow 1004 of PIFO[1]. Header data packet 1017 is output on output path 1020_1 to demultiplexer 1030 for transmission on output path 1031. The demultiplexer can select a PIFO buffer at a given time.

[0216] Figure 11 Described with Figure 9 An example of a consistent FIFO buffer. As previously mentioned, packets need to be added to the bottom or tail of a FIFO queue and dequeued from the head of the queue. A set of K×Ex buffers 1100 includes FIFO[1,1] to FIFO[1,Ex], ..., FIFO[K,1] to FIFO[K,Ex]. In other words, for each input I / F e = 1...Ex, there are K FIFO buffers. Each buffer has a corresponding input path 1103_[1,1] to 1103_[1,Ex], ..., 1103_[K,1] to 1103_[K,Ex] connected to multiplexer 1102 and a corresponding output path 1120_[1,1] to 1120_[1,Ex], ..., 1120_[K,1] to 1120_[K,Ex] connected to demultiplexer 1130. The multiplexer can route packets to a buffer on input path 1101. For example, as combined with Figure 9 The discussion states that, for a given input data packet, one of the K×Ex FIFO queues can be selected based on the packet's priority field and the input I / F identifier.

[0217] Each buffer can store multiple data packets. For example, PIFO[1] stores data packets 1110-1117. New data packets received on the input path of the buffer are stored at the tail position of the buffer, as indicated by arrow 1103_[K,Ex] of FIFO[K,Ex]. Header data packet 1117 is output on output path 1120_[K,Ex] to demultiplexer 1130 for transmission on output path 1131. The demultiplexer can select a FIFO buffer at a given time.

[0218] Some embodiments of the technology described herein can be implemented using hardware, software, or a combination of both. The software used is stored in one or more processor-readable storage devices described above to program one or more processors to perform the functions described herein. Processor-readable storage devices can include computer-readable media, such as volatile and non-volatile media, removable and non-removable media. For example, but not limited to, computer-readable media can include computer-readable storage media and communication media. Computer-readable storage media can be implemented using any method or technology to store computer-readable instructions, data structures, program modules, or other data and other information. Examples of computer-readable storage media include RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disk (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Computer-readable media do not include propagated, modulated, or transient signals.

[0219] Communication media typically embody computer-readable instructions, data structures, program modules, or other data in the form of propagated, modulated, or transient data signals (e.g., carrier waves or other transmission mechanisms), and include any information transmission medium. The term "modulated data signal" refers to a signal in which one or more characteristics are set or altered in a manner that encodes information in the signal. For example, but not limited to, communication media include wired media such as wired networks or direct wired connections, and wireless media such as RF and other wireless media. Combinations of the foregoing terms are also included within the scope of computer-readable media.

[0220] In alternative embodiments, some or all of the software may be replaced by dedicated hardware logic components. Examples, but not limited to, illustrative types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), and special-purpose computers. In one embodiment, software implementing one or more embodiments (stored on a storage device) is used to program one or more processors. The one or more processors may communicate with one or more computer-readable media / storage devices, peripheral devices, and / or communication interfaces.

[0221] It should be understood that the subject matter of this invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to make the subject matter of the invention thorough and complete, and to fully convey the inventive scope to those skilled in the art. In fact, the purpose of the subject matter of this invention is to cover alternatives, modifications, and equivalents to these embodiments, which are included within the scope and spirit of the subject matter of the invention as defined by the appended claims. Moreover, in the following detailed description of the subject matter of the invention, many specific details are set forth to provide a thorough understanding of the subject matter of the invention. However, it will be apparent to those skilled in the art that the subject matter of the invention can be practiced without these specific details.

[0222] This document describes various aspects of the invention in conjunction with flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products provided by embodiments of the invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create mechanisms for implementing the functions / actions detailed in the blocks of the flowcharts and / or block diagrams.

[0223] The description of this invention is presented for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the invention in any way disclosed. Various modifications and alterations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. Various aspects of the invention were chosen and described to better explain the principles and practical applications of the invention and to enable others skilled in the art to understand the invention and the various modifications suited to the intended particular use.

[0224] This invention has been described in conjunction with various embodiments. Other variations and modifications to the disclosed embodiments can be understood and implemented in conjunction with the accompanying drawings, the summary of the invention, and the appended claims, and all such variations and modifications should be interpreted as being covered within the scope of the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0225] For the purposes of this document, it should be noted that the dimensions of the various features depicted in the figures are not necessarily drawn to scale.

[0226] For the purposes of this document, the description may use the terms "an embodiment," "one embodiment," "some embodiments," or "another embodiment" to describe different or the same embodiments.

[0227] For the purposes of this document, a connection can be a direct connection or an indirect connection (e.g., through one or more other components). In some cases, when a component is said to be connected or coupled to another component, that component may be directly connected to the other component or indirectly connected to the other component through an intermediate component. When a component is said to be directly connected to another component, there is no intermediate component between that component and the other component. If two devices are directly or indirectly connected so that they can send electronic signals to each other, then the two devices are "communicating".

[0228] In this document, the term "based on" can be understood as "at least partially based on".

[0229] For the purposes of this document, the use of numerical terms such as “first” object, “second” object, and “third” object without additional context may not imply an order of objects, but rather can be used for identification purposes to distinguish different objects.

[0230] The detailed description above has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the subject matter claimed herein to the precise forms disclosed. Various modifications and alterations can be made in accordance with the above teachings. The described embodiments were chosen to better explain the principles of the disclosed technology and its practical application, thereby enabling those skilled in the art to better utilize the techniques of the various embodiments and various modifications suited to the intended particular purpose. The scope is intended to be defined by the appended claims.

[0231] Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary ways of implementing the claims.

Claims

1. An apparatus for a node in a network, characterized in that, The device includes: Non-transitory memory, including instructions; One or more input network interfaces for receiving data packets; Output network interface for transmitting data packets; One or more processors that communicate with and are coupled to the one or more input network interfaces and the one or more output network interfaces, the one or more processors executing the instructions to: A data packet is received from one of the one or more input network interfaces; the data packet includes an information field, and data extracted from the information field includes a minimum latency value; Based on the minimum delay value extracted from the information field of the data packet, the earliest allowed transmission time for sending the data packet to the next node in the network is enforced. Determine the transmission time of the data packet to be no earlier than the earliest allowed transmission time; Based on the determined transmission time, an indication of the new minimum delay of the data packet at the next node is stored in the information field of the data packet, wherein the indication information of the new minimum delay is determined based on the minimum delay and the transmission time; The data packet is sent to the next node through the output network interface.

2. The apparatus according to claim 1, characterized in that: In order to calculate the indication of the new minimum delay, the one or more processors further execute the instruction to calculate: LATENCY – (Ttrans – Tarr – MinDelay), where LATENCY is a value known to the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, and MinDelay is the minimum delay identified by data extracted from the information field in the data packet.

3. The apparatus according to claim 1, characterized in that: To calculate the indication of the new minimum delay, the one or more processors further execute the instruction to calculate: LATENCY – (Ttrans – Tarr – MinDelay) – SerDelay, where LATENCY is a value known to the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, MinDelay is the minimum delay identified by data extracted from the information field in the data packet, and SerDelay is the serialization delay of the data packet.

4. The apparatus according to claim 2 or 3, characterized in that: The node is pre-configured with the LATENCY prior to the arrival time of the data packet.

5. The apparatus according to claim 2 or 3, characterized in that: The one or more processors also execute the instructions to extract the PRIORITY value from the information field in the data packet and determine the LATENCY based on the PRIORITY value.

6. The apparatus according to claim 5, characterized in that: The one or more processors further execute the instructions to determine the PRIORITY value based on the sequence of per-hop priority information fields and the hop count index in the data packet; The value of the hop count index after receiving the data packet determines the index of the item in the sequence used as the PRIORITY value; The hop count index is incremented in the data packet before it is sent to the next hop.

7. The apparatus according to any one of claims 1 to 3 or 6, characterized in that: The one or more processors also execute the instructions to determine the new minimum latency based on a clock asynchronous with the clock of the next node.

8. The apparatus according to claim 2, 3, or 6, characterized in that: The one or more processors also execute the instructions to determine the transmission time and the arrival time based on a clock whose timestamps have only local significance and which starts counting from the time the node is powered on.

9. The apparatus according to any one of claims 1 to 3 or 6, characterized in that: The data packet is in a service flow that includes a sequence of related data packets; The data packet contains one or more fields that together constitute a service flow identifier, the value of which is unique to the data packet of the service flow; The one or more processors also execute the instructions to determine the new minimum latency independently of the traffic flow identifier.

10. The apparatus according to claim 2, 3, or 6, characterized in that: The one or more input network interfaces are used to receive multiple service flows; The LATENCY is no greater than the sum of the burst size of each data packet and the maximum data packet size in the plurality of service flows, divided by the rate at which data packets are serialized to the output network interface.

11. The apparatus according to any one of claims 1 to 3 or 6, characterized in that: After the earliest allowed transmission time for transmitting the data packet is enforced, the one or more processors execute the instruction to enqueue the data packet into a pre-existing scheduler, and the scheduler determines the transmission time.

12. The apparatus according to claim 2, 3, or 6, characterized in that: The one or more processors also execute the instructions to delay the data packet passing through a timed push-on, first-out (PIFO) queue, wherein the PIFO level of the data packet corresponds to the earliest allowed transmission time summed to the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet, and the head of the PIFO queue is served no earlier than its level.

13. The apparatus according to claim 5, characterized in that: The one or more processors further execute the instructions to: a) use a separate timed push-on, first-out (PIFO) queue for each value of PRIORITY; b) select the PIFO to which the packet is to be inserted based on the PRIORITY; wherein: c) the PIFO level of the packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the packet; d) the header of each timed PIFO is served no earlier than its level, at which point there is no higher priority timed PIFO whose header can be served.

14. The apparatus according to claim 5, characterized in that: The one or more processors further execute the instructions to: a) use a separate timing FIFO for each combination (PRIORITY, NTERFACE) of possible values ​​of PRIORITY and valid incoming NTERFACE index; b) select the FIFO to which the packet is to be inserted and select the interface from which the packet is received, based on the PRIORITY; wherein: c) the timing FIFO rank of the packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the packet; d) the header of the timing FIFO is serviced if: i) the rank of the header of the FIFO is equal to or earlier than the current time; ii) there is no FIFO with a higher PRIORITY that is serviced; iii) there is no FIFO with the same PRIORITY but an earlier header rank.

15. The apparatus according to any one of claims 1 to 3, 6, 13, or 14, characterized in that: The data extracted from the information field includes the additional latency experienced at the previous node.

16. A method for nodes in a network, characterized in that, The method includes: Receive data packets from one or more input network interfaces; the data packets include an information field, and data extracted from the information field includes a minimum latency value; Based on the minimum delay value extracted from the information field of the data packet, the earliest allowed transmission time for sending the data packet to the next node in the network is enforced. Determine the transmission time of the data packet to be no earlier than the earliest allowed transmission time; The information field of the data packet stores an indication of the new minimum delay, wherein the indication information of the new minimum delay is determined based on the minimum delay and the transmission time; The data packet is sent to the next node via the output network interface.

17. The method according to claim 16, characterized in that, Also includes: The calculation of the new minimum delay includes calculating: LATENCY – (Ttrans – Tarr – MinDelay), where LATENCY is a known value of the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, and MinDelay is the minimum delay identified by data extracted from the information field in the data packet.

18. The method according to claim 16, characterized in that, Also includes: The calculation of the new minimum delay includes calculating: LATENCY – (Ttrans – Tarr – MinDelay) – SerDelay, where LATENCY is a known value of the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, MinDelay is the minimum delay identified by data extracted from the information field in the data packet, and SerDelay is the serialization delay of the data packet.

19. The method according to claim 17 or 18, characterized in that, Also includes: The LATENCY is pre-configured for the node before the arrival time of the data packet.

20. The method according to claim 17 or 18, characterized in that, Also includes: Extract the PRIORITY value from the information field in the data packet, and determine the LATENCY based on the PRIORITY value.

21. The method according to claim 20, characterized in that, Also includes: The PRIORITY value is determined based on the sequence of hop priority information fields and the hop count index in the data packet, wherein the value of the hop count index after the data packet is received determines the index of the item in the sequence used as the PRIORITY value, and the hop count index is incremented in the data packet before the data packet is sent to the next hop.

22. The method according to any one of claims 16 to 18 or 21, characterized in that, Also includes: The new minimum delay is determined based on a clock that is asynchronous with the clock of the next node.

23. The method according to claim 17, 18, or 21, characterized in that, Also includes: The transmission time and the arrival time are determined based on a clock whose timestamp has only local significance and which starts counting from the time the node is powered on.

24. The method according to any one of claims 16 to 18 or 21, characterized in that, Also includes: The new minimum latency is determined independently of the service flow identifier, wherein the data packet is located in a service flow that includes a sequence of related data packets, the data packet containing one or more fields that together constitute the service flow identifier, the value of which is unique to the data packets of the service flow.

25. The method according to claim 17, 18, or 21, characterized in that, The one or more input network interfaces are used to receive multiple service flows, and the method further includes: LATENCY is defined as a value no greater than the sum of the burst size and the maximum packet size in the plurality of traffic flows, divided by the rate at which packets are serialized to the output network interface.

26. The method according to any one of claims 16 to 18 or 21, characterized in that, Also includes: After the earliest allowed transmission time for transmitting the data packet is enforced, the data packet is queued into a pre-existing scheduler, wherein the scheduler determines the transmission time.

27. The method according to claim 17, 18, or 21, characterized in that: The enforcement of the earliest allowed transmission time includes delaying the data packets through a timed Push-On, First-Out (PIFO) queue, wherein the PIFO level of the data packets corresponds to the earliest allowed transmission time summed with the arrival time and the minimum delay of the data identifier extracted from the information field in the data packets, and the head of the PIFO queue is served no earlier than its level.

28. The method according to claim 20, characterized in that, Also includes: Use a separate timed push-on, first-out (PIFO) queue for each value of PRIORITY; The PIFO to which the data packet is to be inserted is selected based on the PRIORITY, wherein the PIFO level of the data packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet; The head is serviced no earlier than the level of the head of each timed PIFO, at which point there is no higher priority timed PIFO for which the head can be serviced.

29. The method according to claim 17, characterized in that, Also includes: Use a separate timed FIFO for each possible combination of PRIORITY and valid incoming INTERFACE index; The FIFO to which the data packet is to be inserted is selected according to the PRIORITY, and the interface from which the data packet is received is selected, wherein the timing FIFO level of the data packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet; Service is provided for the head of a timed FIFO in the following circumstances: the rank of the head of the FIFO is equal to or earlier than the current time, there is no FIFO with a higher rank that can be serviced, and there is no FIFO with the same rank but a higher head rank.

30. The method according to claim 17, 18, 21, 28, or 29, characterized in that: The data extracted from the information field indicates the additional latency experienced at the previous node.

31. An apparatus, characterized in that, include: One or more input network interfaces at the current node in the network, the one or more input network interfaces being used to receive data packets, the data packets including an indication of a first minimum delay to be enforced at the current node before sending the data packets to the next node in the network, the indication of the first minimum delay including an information field in the data packets storing the first minimum delay; The output network interface at the current node is configured to transmit the data packet to the next node at a time no earlier than the earliest allowed transmission time for transmitting the data packet at the current node, based on a first minimum delay. The transmitted data packet includes an indication of a second minimum delay to be enforced at the next node before transmitting the data packet in the network. The second minimum delay provides a specified delay between the earliest allowed transmission time for transmitting the data packet at the current node and the earliest allowed transmission time for transmitting the data packet at the next node.

32. The apparatus according to claim 31, characterized in that: The data packet includes a priority value; The specified delay is based on the priority value.

33. The apparatus according to claim 32, characterized in that: The data packet includes a hop count index; The priority value is based on the hop count index.

34. The apparatus according to any one of claims 31 to 33, characterized in that: The second minimum delay is a decreasing function of the difference between the transmission time of the data packet at the current node and the earliest allowed transmission time of the data packet at the current node.

35. The apparatus according to any one of claims 31 to 34, characterized in that: The indication of the first minimum latency includes an information field in the data packet that stores additional latency.

36. A method, characterized in that, The method includes: A data packet is received at one or more input network interfaces of the current node in the network, the data packet including an indication of a first minimum delay to be enforced at the current node before sending the data packet to the next node in the network, the indication of the first minimum delay including an information field in the data packet storing the first minimum delay; At the output network interface of the current node, the data packet is transmitted to the next node at a time no earlier than the earliest allowed transmission time for transmitting the data packet at the current node, based on the first minimum delay. The transmitted data packet includes an indication of a second minimum delay to be enforced at the next node before transmitting the data packet in the network. The second minimum delay provides a specified delay between the earliest allowed transmission time for transmitting the data packet at the current node and the earliest allowed transmission time for transmitting the data packet at the next node.

37. The method according to claim 36, characterized in that: The data packet includes a priority value; The specified delay is based on the priority value.

38. The method according to claim 37, characterized in that: The data packet includes a hop count index; The priority value is based on the hop count index.

39. The method according to any one of claims 36 to 38, characterized in that: The second minimum delay is a decreasing function of the difference between the transmission time of the data packet at the current node and the earliest allowed transmission time of the data packet at the current node.

40. The method according to any one of claims 36 to 39, characterized in that: The indication of the first minimum latency includes an information field in the data packet that stores additional latency.

41. A non-transitory computer-readable medium for storing computer instructions, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: Receive data packets from the input network interface; the data packets include an information field, and data extracted from the information field includes a minimum latency value; Based on the minimum delay value extracted from the information field of the data packet, the earliest allowed transmission time for sending the data packet to the next node in the network is enforced. Determine the transmission time of the data packet to be no earlier than the earliest allowed transmission time; Based on the determined transmission time, an indication of the new minimum delay of the data packet at the next node is stored in the information field of the data packet, wherein the indication information of the new minimum delay is determined based on the minimum delay and the transmission time; The data packet is sent to the next node via the output network interface.

42. The non-transitory computer-readable medium according to claim 41, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: The calculation of the new minimum delay includes calculating: LATENCY – (Ttrans – Tarr – MinDelay), where LATENCY is a value known to the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, and MinDelay is the minimum delay identified by data extracted from the information field in the data packet.

43. The non-transitory computer-readable medium according to claim 41, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: The calculation of the new minimum delay includes calculating: LATENCY – (Ttrans – Tarr – MinDelay) – SerDelay, where LATENCY is a value known to the node, Ttrans is the determined transmission time of the data packet, Tarr is the arrival time of the data packet, MinDelay is the minimum delay identified by data extracted from the information field in the data packet, and SerDelay is the serialization delay of the data packet.

44. The non-transitory computer-readable medium according to claim 42 or 43, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: Extract the PRIORITY value from the information field in the data packet, and determine the LATENCY based on the PRIORITY value.

45. The non-transitory computer-readable medium according to claim 44, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: The PRIORITY value is determined based on the sequence of hop priority information fields and the hop count index in the data packet, wherein the value of the hop count index after the data packet is received determines the index of the item in the sequence used as the PRIORITY value, and the hop count index is incremented in the data packet before the data packet is sent to the next hop.

46. ​​The non-transitory computer-readable medium according to claim 41, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: After the earliest allowed transmission time for transmitting the data packet is enforced, the data packet is queued into a pre-existing scheduler, wherein the scheduler determines the transmission time.

47. The non-transitory computer-readable medium according to any one of claims 42 to 43 or 45, characterized in that: The enforcement of the earliest allowed transmission time includes delaying the data packets through a timed Push-On, First-Out (PIFO) queue, wherein the PIFO level of the data packets corresponds to the earliest allowed transmission time summed with the arrival time and the minimum delay of the data identifier extracted from the information field in the data packets, and the head of the PIFO queue is served no earlier than its level.

48. The non-transitory computer-readable medium according to claim 44, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: Use a separate timed push-on, first-out (PIFO) queue for each value of PRIORITY; The PIFO to which the data packet is to be inserted is selected based on the PRIORITY, wherein the PIFO level of the data packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet; The head is serviced no earlier than the level of the head of each timed PIFO, at which point there is no higher priority timed PIFO for which the head can be serviced.

49. The non-transitory computer-readable medium according to claim 44, characterized in that, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the following steps: Use a separate timed FIFO for each possible combination of PRIORITY and valid incoming INTERFACE index; The FIFO to which the data packet is to be inserted is selected according to the PRIORITY, and the interface from which the data packet is received is selected, wherein the timing FIFO level of the data packet is the earliest allowed transmission time, corresponding to the sum of the arrival time and the minimum delay of the data identifier extracted from the information field in the data packet; Service is provided for the head of a timed FIFO in the following circumstances: the rank of the head of the FIFO is equal to or earlier than the current time, there is no FIFO with a higher rank that can be serviced, and there is no FIFO with the same rank but a higher head rank.

50. The non-transitory computer-readable medium according to any one of claims 41 to 43, 45, 46, 48, or 49, characterized in that: The data extracted from the information field indicates the additional latency experienced at the previous node.

Images