Deterministic flow transmission method and apparatus

Abstract

The present invention provides a method and device for deterministic flow transmission. The embodiment of the present invention can realize adding deterministic flow forwarding to existing best-effort flow forwarding, and further, the distribution thread, DFT, and CFT in the network device ensure that the deterministic flow forwarding is not affected by the best-effort flow, and the compatibility between the existing best-effort flow forwarding and the deterministic flow forwarding is guaranteed.

Description

[Technical Field]

[0001] The present application relates to network communication technology, and in particular to a deterministic flow transmission method and apparatus. [Background technology]

[0002] A deterministic network (DetNet) can provide deterministic service functions for bearer services within a network domain. These deterministic service functions may include delay, packet loss rate, etc. As an example, a deterministic network implemented based on a local area network, i.e., a Time Sensitive Network (TSN), uses Cyclic Queuing and Forwarding (CQF) to ensure that the transmission delay of the entire forwarding path is determined at the forwarding layer.

[0003] For convenience of explanation, a service flow that is transmitted through a deterministic network and has deterministic service functions such as delay and packet loss rate is called a deterministic flow. For service flows other than deterministic flows, existing forwarding devices, such as software forwarding devices and hardware forwarding devices implemented based on switching chips or NPs, forward them using a best-effort forwarding method, and other service flows other than deterministic flows can also be called best-effort flows. Summary of the Invention

[0004] The present invention provides a deterministic flow transmission method, system, and apparatus for implementing deterministic flow transmission of Cycle Specified Queuing and Forwarding (CSQF) based on segment routing (SR).

[0005] According to an embodiment of the present invention, there is provided a deterministic flow transmission method applied to a network device, comprising: Distributing packets in a packet receive queue bound to the distribution thread by an activated distribution thread on the device, whereby packets belonging to a deterministic flow in the packet receive queue are distributed to a corresponding deterministic flow buffering queue (DBQ) and packets belonging to a best effort flow are distributed to a corresponding best effort flow queue (BTQ), wherein the packets in the packet receive queue are packets received externally via a local interface of the network device; by an activated deterministic forwarding thread (DFT) on the device, queueing packets in a DBQ bound to the DFT into a corresponding deterministic flow queue (DTQ); By an activated periodic transfer thread (CFT) on the device, packets in a DTQ bound to the CFT are queued into a corresponding periodic assignment queue (CSQ), and the CFT selects a packet from a transmission queue (SQ) and transfers the packet through an out-interface for transferring packets, the SQ being the CSQ currently pointed to by a transmission queue pointer (SQP) corresponding to the CFT; A deterministic flow transmission method is provided.

[0006] According to an embodiment of the present invention, there is provided an electronic device including a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions executable by the processor, the processor being used to execute the machine-executable instructions to perform the steps of the method disclosed above. An electronic device is also provided.

[0007] As can be seen from the above technical solutions, the present embodiment realizes adding deterministic flow forwarding to the existing best-effort flow forwarding.

[0008] Furthermore, the above-mentioned distribution thread, DFT, and CFT ensure that deterministic flow forwarding is not affected by best-effort flows, ensuring interoperability between existing best-effort flow forwarding and deterministic flow forwarding. [Brief explanation of the drawings]

[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present invention and, together with the description, serve to explain the principles of the invention. [Figure 1] FIG. 1 is a schematic diagram illustrating the structure of best effort forwarding. [Figure 2] 1 is a flowchart illustrating a method provided in an embodiment of the present invention. [Figure 3] FIG. 1 is a schematic diagram illustrating the structure of deterministic flow forwarding and best-effort flow forwarding provided in an embodiment of the present invention. [Figure 4] 1 is a flowchart illustrating an implementation of a distribution thread provided in an embodiment of the present invention. [Figure 5] 1 is a flowchart illustrating a DFT implementation provided in an embodiment of the present invention. [Figure 6] 1 is a schematic diagram illustrating the relationship between the fast forwarding table and OutIfInfo and SRLists provided in an embodiment of the present invention. FIG. [Figure 7] 1 is a flowchart illustrating the realization of a CFT provided in an embodiment of the present invention. [Figure 8] FIG. 1 is a schematic diagram illustrating the relationship between CSQP and CSQ provided in an embodiment of the present invention. [Figure 9] FIG. 1 is a schematic diagram showing the structure between SQ, RQ and TQ provided in an embodiment of the present invention. [Figure 10] FIG. 2 is a schematic diagram illustrating interactions between threads provided in an embodiment of the present invention. [Figure 11] 1 is a structural diagram illustrating an apparatus provided in an embodiment of the present invention; [Figure 12]1 is a structural diagram illustrating an electronic device provided in an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0010] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, like numbers in different drawings refer to the same or similar elements unless otherwise noted. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus, devices, and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0011] The terms used herein are merely for the purpose of describing particular embodiments and are not intended to limit the present invention. As used herein and in the appended claims, the singular forms "a," "the," and "said" are intended to include the plural forms unless the context clearly dictates otherwise.

[0012] In order to enable those skilled in the art to better understand the technical solutions provided in the embodiments of the present invention, and to make the above-mentioned objectives, features and advantages of the embodiments of the present invention more apparent and understandable, a forwarding process designed based on best-effort forwarding is described below.

[0013] For software forwarding devices (e.g., low-end and mid-end routers), packets are typically forwarded according to a best-effort forwarding policy as shown in Figure 1.

[0014] As shown in FIG. 1, after receiving a packet, a hardware controller (e.g., a controller on an interface board) first stores the packet in a corresponding packet receiving queue (RcvQueue: Receiving Queue) in memory according to the packet feature parameters of the packet. Optionally, taking the packet feature parameters as an example where a packet quintuple is used as the packet feature parameters, the hardware controller may perform a hash operation on the packet quintuple to obtain a result of the operation, search the packet receiving queue in memory for a packet receiving queue whose queue number is the result of the operation, and store the packet in the searched packet receiving queue. Note that this is merely an example of how to store a packet in a corresponding packet receiving queue in memory based on the packet feature parameters of the packet, and is not intended to be limiting.

[0015] In one example, after receiving a packet, the above-mentioned hardware controller generates an interrupt to an interrupt controller, and the interrupt call generated by the interrupt controller invokes an interrupt handler in the driver software (e.g., a CPU) to invoke at least one best-effort forwarding thread (BFT).

[0016] Here, the activated BFT takes a packet from the bound packet receiving queue, converts the taken packet into a packet of a specified structure, and sends it to the platform software (also called a platform interface).

[0017] The platform software consults the FIB table to determine the out-interface for forwarding the packet, and calls the out-interface (Transmod) of the driver software to send the packet to the hardware queue corresponding to the out-interface for transmission. Optionally, in this embodiment, packets in the hardware queue with a higher priority are transmitted first. After transmitting the packet, the free buffer pool (Free Buffer Pool) may be called to further release the storage resources occupied by the transmitted packet.

[0018] The forwarding process for forwarding packets according to the best-effort forwarding method (which may also be called a best-effort flow) has been described above.

[0019] However, the current best-effort flow forwarding process lacks support for deterministic networks, making deterministic flow forwarding unrealizable.

[0020] Based on this, an embodiment of the present invention provides a deterministic flow transmission method that overlays deterministic flow forwarding on best-effort flow forwarding, ensuring compatibility between best-effort flow forwarding and deterministic flow forwarding while ensuring that deterministic flow forwarding is not affected by best-effort flows. The deterministic flow transmission method provided in the embodiment of the present invention is described below.

[0021] 2 is a flowchart illustrating a deterministic flow transmission method provided in an embodiment of the present invention. The flow is applied to a network device. The embodiment of the present invention does not particularly limit the specific structure of the network device. For example, the network device may be a low-end or mid-end router, or a high-end router that requires high performance and supports more CPU cores. The embodiment of the present invention is not particularly limited because the embodiment of the present invention has a wide range of application and scalability.

[0022] As described above, the embodiment of the present invention superimposes deterministic flow forwarding on top of best-effort flow forwarding. To make it easier to understand the method provided in the embodiment of the present invention, we will first describe the following changes that are newly added to superimpose deterministic flow forwarding on top of best-effort flow forwarding.

[0023] 1) Adding a distribution thread (DisThr): Deterministic flows are highly sensitive to delay and jitter, and are highly susceptible to platform service processing, such as best-effort flow forwarding. To prevent platform service processing from affecting the forwarding of deterministic flows, in the embodiment of the present invention, a distribution thread is newly added to the network device described above.

[0024] Optionally, in this embodiment, the number of distribution threads is set according to the number of packet receiving queues, and one distribution thread is bound to at least one packet receiving queue. Specifically, the distribution thread is used to distribute packets in the bound packet receiving queue, and distributes packets belonging to deterministic flows in the packet receiving queue to corresponding deterministic flow buffering queues (DetNet-flow Buffering Queues, DBQs), and distributes packets belonging to best-effort flows to corresponding best-effort flow queues (BTQs).

[0025] Optionally, DBQs and packet receive queues have a one-to-one correspondence; for example, packets belonging to a deterministic flow in packet receive queue 1 are distributed to DBQ1, and packets belonging to a deterministic flow in packet receive queue 2 are distributed to DBQ2. DBQs are used to buffer packets belonging to deterministic flows that are distributed by the distribution thread. The work of the distribution thread to distribute packets is decoupled from the work of the platform to process packets belonging to deterministic flows, so the distribution thread processes a single process, which is highly efficient and non-blocking, while the process of determining which packets belong to a deterministic flow is independent.

[0026] Optionally, BTQs and packet receiving queues have a one-to-one correspondence; for example, packets belonging to the best-effort flow in packet receiving queue 1 are distributed to BTQ1, and packets belonging to the best-effort flow in packet receiving queue 2 are distributed to BTQ2. BTQs are used to buffer packets belonging to the best-effort flow distributed by the distribution thread. The work of distributing packets by the distribution thread is separated from the work of the platform processing packets belonging to the best-effort flow, preventing the platform's uncertainty processing work from blocking the distribution thread.

[0027] As can be seen from the above, the process by which the distribution thread distributes packets is a non-blocking process that ensures that packets belonging to deterministic flows are distributed in a timely manner, even if there are uncertainties due to platform processing.

[0028] To prevent the distribution of deterministic flows from being delayed due to an excessive number of best-effort flows, the specific implementation requires that the above-mentioned distribution threads occupy sufficient CPU resources. For example, one distribution thread exclusively occupies as many CPU cores as possible, and the CPU cores allocated to a distribution thread cannot be allocated to other threads.

[0029] 2) Adding a Deterministic Flow Forwarding Core Module. Optionally, embodiments of the present invention may add a deterministic flow forwarding core module to the driver software of the network device. In one example, the deterministic flow forwarding core module includes at least a deterministic forwarding thread (DFT) and a cycle forwarding thread (CFT).

[0030] Optionally, in this embodiment, the number of DFTs is set based on the number of DBQs and the deterministic flow load. In an application, one DFT is bound to at least one DBQ. The DFT is used to queue packets in the bound DBQ into the corresponding deterministic flow queue (DetNet flow Queue, DTQ).

[0031] Optionally, in this embodiment, the number of CFTs is set based on the number of DTQs and the deterministic flow load. In an application, one CFT is bound to at least one DTQ. The CFT is used to queue packets in the bound DTQ into a corresponding Cycle Specified Queue (CSQ). Optionally, in this embodiment, the CFT is further bound to at least one CSQ of at least one interface, and is used to retrieve packets from the specified CSQ within a specified cycle and forward the packets via an out-interface for forwarding the packets.

[0032] 3) Modifying the existing BFT: Conventionally, the BFT retrieves packets from the packet reception queue and forwards them, but in this embodiment, the BFT is modified to retrieve packets from the bound BTQ and forward them.

[0033] 4) Modify the Transmitting Module (TransMod) to enable it to provide deterministic flow forwarding (e.g., increase new hardware adaptations for deterministic flow forwarding). Optionally, in one embodiment, during deterministic flow forwarding, packets belonging to deterministic flows may be designated at the out-interface as having the highest priority to ensure that packets belonging to deterministic flows are forwarded first, and at the same time, rate limiting may be implemented for the forwarding of best-effort flows.

[0034] Above, we have briefly explained the following changes that are newly added to overlay deterministic flow forwarding on top of best-effort flow forwarding. The above changes enable the addition of deterministic flow forwarding while maintaining compatibility with existing best-effort QOS forwarding, and ensure that deterministic flow forwarding is not affected by best-effort flow forwarding. Figure 3 shows, as an example, the structure of a modified framework based on the framework shown in Figure 1.

[0035] Based on the above modifications, the method provided in the embodiment of the present invention will be described below.

[0036] As shown in FIG. 2, the flow may include the following steps:

[0037] Step 201: A distribution thread is woken up on the device to distribute packets in a packet receiving queue bound to the distribution thread, thereby distributing packets belonging to a deterministic flow in the packet receiving queue to a corresponding DBQ and packets belonging to a best-effort flow to a corresponding BTQ.

[0038] As described above, each packet receiving queue stores packets received from the outside via the local interface of the network device.

[0039] How the distribution thread distributes packets in the bound packet receiving queue can be seen in the implementation flow of the distribution thread, which will be explained later with an example in FIG. 4, so a detailed explanation will be omitted here.

[0040] In this embodiment, the distribution thread may be started by an interrupt or by other methods such as polling, and this embodiment is not particularly limited.

[0041] Step 202: The activated DFT on the device places packets in the DBQ bound to the DFT into the corresponding DTQ.

[0042] In this embodiment, the DFT may be started at an appropriate time by a distribution thread, as will be described below with reference to an example. Of course, the DFT may be started in other ways, and this embodiment is not particularly limited thereto.

[0043] How the DFT places packets in the bound DBQ into the corresponding DTQ can be seen in the DFT implementation flow explained later with an example in FIG. 5, so a detailed explanation is omitted here.

[0044] Step 203: The activated periodic transfer thread (CFT) on the device puts the packets in the DTQ bound to the CFT into a corresponding cyclic specific queue (CSQ), and the CFT selects a packet from a transmission queue (SQ) and transfers the packet through an out-interface for transferring the packet, where the SQ is the CSQ currently pointed to by the transmission queue pointer (SQP) corresponding to the CFT.

[0045] In this embodiment, the CFT can be started by the DFT at an appropriate time, as described below with examples. Of course, the CFT may be started in other ways, such as starting the CFT directly by default after creation, or starting the CFT by default when a specified event is detected after creation, and this embodiment is not particularly limited. The specified event here refers to an event that requires packets to be transferred to the CSQ in a timely manner. Examples of such events will be described later, so their description will be omitted here.

[0046] How the CFT places packets in the DTQ bound to the CFT into the corresponding CSQ, selects packets from the sending queue (SQ), and forwards the packets via the out-interface for forwarding packets can be seen in the CFT implementation flow explained below with an example in Figure 7, so the explanation will be omitted here.

[0047] In this embodiment, each interface corresponds to multiple CSQs, for example, three CSQs, where each CSQ includes at least: (1) Cyclic Specific Queue Descriptors (CSQD): Used to describe CSQs. (2) Buffer Descriptor Queue (BDQ): In this embodiment, a queue element of the BDQ describes information about one data block, and may include at least information about the memory block in which the data resides, the data start position, the data length, etc., and this embodiment is not particularly limited.

[0048] Optionally, in this embodiment, each periodic queue descriptor may include at least the following: Queue Header (QH): Used to point to the corresponding BDQ. Head: Refers to the current head position of the queue. When the CSQ is empty, Head is 0, indicating that the next packet to enter the queue will occupy the 0th position. Head increases by 1 each time data enters the queue. QLen: This is the queue length, which indicates the number of packets that can be stored in the queue. When the length of Head reaches QLen, it indicates that the queue is full. CurBytes: The number of bytes currently queued. MaxBytes: The current queue capacity in bytes.

[0049] In this embodiment, the CFT also manages a cyclic specific queue pointer structure (abbreviated as CSQP). The CSQP is composed of three members. SQP: Sending Queue Pointer. The CSQ pointed to by the SQP indicates the CSQ being sent in the current cycle (this CSQ is also called a sending queue (SQ)). RQP: Receiving Queue Pointer. The CSQ pointed to by the RQP indicates the CSQ being received in the current cycle (this CSQ is also called a receiving queue (RQ)). TQP: Tolerating Queue Pointer, the CSQ pointed by TQP represents the CSQ that is allowed to be received within the current period after the RQ is full (this CSQ is also called the Tolerating Queue TQ). Based on the above description, Figure 8 illustrates a schematic diagram showing the relationship between the CSQP and the CSQ.

[0050] This completes the flow shown in Figure 2.

[0051] According to the flow shown in FIG. 2, this embodiment realizes adding deterministic flow forwarding to the existing best-effort flow forwarding.

[0052] Furthermore, the above-mentioned distribution thread, DFT, and CFT ensure that deterministic flow forwarding is not affected by best-effort flows, ensuring interoperability between existing best-effort flow forwarding and deterministic flow forwarding.

[0053] The flow of implementing the distributed thread will be explained below using the flow shown in FIG.

[0054] Please refer to Figure 4, which is a flowchart showing the implementation of the distribution thread provided in the embodiment of the present invention. In this embodiment, the distribution thread has a corresponding thread polling flag (denoted as bPoll_1) set, and bPoll_1 is initially set to FALSE. bPoll_1 is also FALSE when the distribution thread is started. In this embodiment, the distribution thread can be started by an interrupt or other method.

[0055] As shown in FIG. 4, the launched distribution thread may perform the following steps:

[0056] Step 401: The distribution thread traverses the bound packet receiving queues in order, determines the traversed packet receiving queue as the current queue, checks whether there are packets in the current queue, and executes step 402 if there are packets in the current queue; if there are no packets in the current queue, executes step 407.

[0057] Step 402, set bPoll_1 to TRUE, traverse the packets in the current queue, determine the traversed packet as the current packet, and execute step 403 if the current packet is identified as belonging to a deterministic flow, or execute step 405 if the current packet is identified as belonging to a best-effort flow.

[0058] Step 402 is executed when it is determined that there is a packet in the current queue. When there is a packet in the current queue, setting bPoll_1 to TRUE indicates that the distribution thread is currently entering polling processing. In order to reduce the overhead of thread scheduling and prevent the wake-up operation from affecting the polling of the distribution thread, it is necessary to instruct the distribution thread to turn off the specified function set thereto. Here, turning off the specified function is intended to instruct the distribution thread to prevent it from entering a sleep state.

[0059] Step 403: Put the current packet into the DBQ corresponding to the current queue, and then execute step 404.

[0060] In this embodiment, a deterministic property (DetProperty) is added to packets belonging to a deterministic flow. For example, the deterministic property is not fixed, and is not specifically specified in current specifications. Instead, it is comprehensively set based on actual requirements, such as by setting a value for a field such as DSCP (Differentiated Services Code Point) to express the deterministic property. In this embodiment, the deterministic property is not particularly limited. However, once the deterministic property is set and applied to step 403, it is identified whether the deterministic property is added to the current packet. If the deterministic property is added, the current packet is identified as belonging to a deterministic flow and placed in a DBQ corresponding to the current queue. Finally, packets belonging to a deterministic flow are distributed to DBQs.

[0061] Optionally, as an embodiment, in this embodiment, the DBQs and the packet receiving queues have a one-to-one correspondence, for example, a packet in the packet receiving queue numbered 1 is placed in the DBQ numbered 1, a packet in the packet receiving queue numbered 2 is placed in the DBQ numbered 2, etc. Finally, based on the correspondence between the DBQs and the packet receiving queues described above, it is realized that the current packet identified as belonging to the deterministic flow is placed in the DBQ corresponding to the current queue.

[0062] Step 404: If the DFT startup flag set for the DFT bound to the DBQ is FALSE at this time, set the DFT startup flag to TRUE and start the DFT whose DFT startup flag is TRUE. Then, determine whether there are any untraversed packets in the current queue. If there are any untraversed packets in the current queue, continue traversing the untraversed packets. Return to step 402, the step of determining the traversed packet as the current packet. If there are no untraversed packets in the current queue, execute step 407.

[0063] In this embodiment, the distribution thread sets a corresponding DFT activation flag for each DFT. Initially, the DFT activation flag for each DFT is set to FALSE.

[0064] Step 404 activates the corresponding DFT in a timely manner, and the activated DFT can place the packets in the bound DBQ into the corresponding DTQ in a timely manner. Details can be found in the subsequent DFT flow, so the explanation is omitted here.

[0065] At this time, if the DFT activation flag set for the DFT bound to the DBQ is TRUE, step 404 may directly traverse the untraversed packets if there are any untraversed packets in the current queue, and return to step 402, where the traversed packets are determined to be the current packet; if there are no untraversed packets in the current queue, step 407 may be executed.

[0066] Step 405: Put the current packet into the BTQ corresponding to the current queue, and then execute step 406.

[0067] Optionally, as an embodiment, in this embodiment, the BTQs and the packet receiving queues have a one-to-one correspondence, for example, a packet in the packet receiving queue numbered 1 is placed in the BTQ numbered 1, a packet in the packet receiving queue numbered 2 is placed in the BTQ numbered 2, etc. Finally, based on the correspondence between the BTQs and the packet receiving queues described above, it is realized that the current packet identified as belonging to the best-effort flow is placed in the BTQ corresponding to the current queue.

[0068] Step 406: At this time, if the BFT startup flag set for the BFT bound to the above-mentioned BTQ is FALSE, set the BFT startup flag to TRUE and start the BFT whose BFT startup flag is TRUE, then determine whether there are any untraversed packets in the current queue, if there are any untraversed packets in the current queue, traverse the untraversed packets, and return to the step in step 402 of determining the traversed packet as the current packet, and if there are no untraversed packets in the current queue, execute step 407.

[0069] Step 406 allows the BFT bound to the above-mentioned BTQ to be activated in a timely manner, and the activated BFT can transfer packets in the BTQ in a timely manner.

[0070] Step 407: If there is no packet in the current queue or there is no untraversed packet in the current queue, detect whether there is an untraversed packet receiving queue among all packet receiving queues bound to the distribution thread, and if there is an untraversed packet receiving queue, continue traversing other packet receiving queues bound to the distribution thread and not traversed, and return to the step of determining the traversed packet receiving queue as the current queue in step 401; otherwise, execute step 408.

[0071] In step 408, for a DFT whose DFT startup flag is TRUE, the DFT is started and the DFT startup flag of the DFT is set to FALSE, and for a BFT whose BFT startup flag is TRUE, the BFT is started and the BFT startup flag of the BFT is set to FALSE. Then, step 409 is executed.

[0072] Step 408 is executed here to wake up the DFTs and BFTs that were previously started but became dormant while the distribution thread was traversing the packet receive queue, so that both deterministic flows and best-effort flows can be forwarded in a timely manner. Of course, if the DFTs whose DFT start-up flags are TRUE are not currently dormant (e.g., still running), step 408 can simply set the DFT start-up flags of the DFTs whose DFT start-up flags are TRUE to FALSE directly. The principle of the BFTs whose BFT start-up flags are TRUE is similar, so it will not be repeated.

[0073] Step 409, if bPoll_1 is TRUE, set bPoll_1 to FALSE, and return to above step 401; if bPoll_1 is FALSE, turn on the above-mentioned designation function, which is used to instruct the distribution thread to wait to be woken up.

[0074] This completes the flow shown in Figure 4.

[0075] The flow of realizing the distribution process has been explained using the flow shown in FIG.

[0076] The flow of realizing DFT will be explained below using the flow shown in FIG.

[0077] Referring to FIG. 5, a flowchart illustrating the implementation of DFT provided in an embodiment of the present invention is shown. In this embodiment, the DFT may be initiated by a distributed thread (see the flow shown in FIG. 4 above for details). In this embodiment, the DFT may be initiated by other methods, which are not particularly limited here. In this embodiment, a corresponding thread polling flag (referred to as a DFT polling flag) is set for the initiated DFT, and the DFT polling flag is FALSE when the DFT is initiated.

[0078] As shown in FIG. 5, the invoked DFT may perform the following steps:

[0079] In step 501, the DFT traverses the bound DBQs in order, determines the traversed DBQ as the current DBQ, checks whether there are packets in the current DBQ, and executes step 502 if there are packets in the current DBQ; if there are no packets in the current DBQ, executes step 505.

[0080] Step 502 sets the DFT polling flag to TRUE, traverses the packets in the current DBQ, determines the traversed packet as the current DBQ packet, and calls the configured packet replication, elimination, and ordering functions (PREOF) to generate a packet output chain if the current DBQ packet is identified as not being a duplicate packet.

[0081] This step 502 is executed when it is detected that there is a packet in the current DBQ. Setting the DFT polling flag to TRUE when there is a packet in the current DBQ indicates that the DFT is now entering polling processing. In order to reduce the overhead of thread scheduling and to prevent wake-up operations from affecting the DFT polling, it is necessary to instruct the DFT to turn off the designated function set at this time. Here, turning off the designated function is to instruct the DFT to prevent it from entering a sleep state.

[0082] In this embodiment, the DFT is configured with a PREOF function. Based on the PREOF function, it is possible to identify whether the current DBQ packet is a duplicate packet. Here, the PREOF function can be used to identify whether the current DBQ packet is a duplicate packet of a packet previously processed by calling the PREOF function. If the current DBQ packet is identified as a duplicate packet, the current DBQ packet is discarded. If there are untraversed packets in the current DBQ, the traversal of the untraversed packets continues, and the process returns to step 502, which determines the traversed packet as the current DBQ packet. Optionally, there are many ways to identify whether the current DBQ packet is a duplicate packet based on the PREOF function, such as identifying whether the packet sequence number of the current DBQ packet is smaller than the highest recorded sequence number corresponding to the service flow to which the current DBQ packet belongs, and if so, identifying the current DBQ packet as a duplicate packet. This embodiment does not particularly limit the scope of the method.

[0083] Optionally, in this embodiment, if the current DBQ packet is identified as not being a duplicate packet based on the PREOF function, the PREOF function may be called to generate a packet output chain. How to call the PREOF function to generate a packet output chain will be described below with an example, and therefore will not be described here.

[0084] In this embodiment, after generating the packet output chain, each packet in the packet output chain may be encapsulated and put into a DTQ corresponding to an out-interface for forwarding the packet, details can be seen in step 503.

[0085] Step 503: traverse packets in the packet output chain, set the traversed packet as the target packet, encapsulate the target packet, and put the encapsulated target packet into a DTQ corresponding to an out-interface for forwarding the target packet; check whether the target packet is the last untraversed packet in the packet output chain; if it is the last untraversed packet, execute step 504; if it is not the last untraversed packet, return to the step of traversing the untraversed packets in the packet output chain and set the traversed packet as the target packet.

[0086] Optionally, in this embodiment, a corresponding DTQ is set for each out-interface. For convenience of explanation, for example, the DTQ corresponding to out-interface 1 is denoted as DTQ1, and the DTQ corresponding to out-interface 2 is denoted as DTQ2. Based on this, after determining the out-interface for forwarding the target packet, the target packet is encapsulated and the encapsulated target packet is placed in the DTQ corresponding to the out-interface for forwarding the target packet. For example, if it is determined that the out-interface for forwarding the target packet is out-interface 1, the encapsulated target packet is placed in the DTQ corresponding to the out-interface for forwarding the target packet, i.e., DTQ1.

[0087] How to encapsulate the target packet will be explained below with an example, so the explanation will be omitted here.

[0088] Finally, the packets in the DBQ are put into the DTQ by step 503 described above.

[0089] Step 504: At this time, if the CFT activation flag set for the CFT bound to the above-mentioned DTQ is FALSE, set the CFT activation flag to TRUE and activate the CFT whose CFT activation flag is TRUE. Then, determine whether there are any untraversed packets in the current DBQ. If there are any untraversed packets in the current DBQ, continue traversing the untraversed packets, and return to the step in step 502 of determining the traversed packet as the current DBQ packet. If there are no untraversed packets in the current DBQ, execute step 505.

[0090] In this embodiment, the DFT sets a corresponding CFT activation flag for each CFT. Initially, the CFT activation flag for each CFT is set to FALSE.

[0091] Step 504 promptly activates the CFT bound to the above-mentioned DTQ, and the activated CFT can promptly put packets in the bound DTQ into the corresponding CSQ, select packets from the SQ, and forward them through the packet out interface. For details, see the flow shown in Figure 7 below.

[0092] In addition, if the CFT startup flag set for the CFT bound to the above-mentioned DTQ is TRUE, this step 504 may directly continue traversing the untraversed packets if there are any untraversed packets in the current DBQ, and return to the step in step 502 of determining the traversed packet as the current DBQ packet, and if there are any untraversed packets in the current DBQ queue, execute step 505.

[0093] Step 505: If there is no packet in the current DBQ or there is no untraversed packet in the current DBQ, detect whether there is an untraversed DBQ in all DBQs bound to the DFT, and if there is an untraversed DBQ, continue traversing the untraversed DBQ, and return to the step of determining the traversed DBQ as the current DBQ in step 501; otherwise, execute step 506.

[0094] In step 506, for a CFT whose CFT startup flag is TRUE, the CFT is started and the CFT startup flag of the CFT is set to FALSE. Then, step 507 is executed.

[0095] Step 506 is performed here to wake up any CFTs that were previously invoked but went dormant while the DFT was traversing the DBQ, ensuring the reliability of deterministic flow forwarding.

[0096] Step 507: If the CFT polling flag is TRUE, set the CFT polling flag to FALSE, and return to the above step 501; if the CFT polling flag is FALSE, turn on the above specified function and instruct the DFT to wait for activation.

[0097] This completes the flow shown in Figure 5.

[0098] The flow of DFT implementation has been explained using the flow shown in Figure 5.

[0099] The flow shown in FIG. 5 will now be described how the PREOF function is invoked to generate a packet output chain.

[0100] Optionally, in one embodiment, an example of calling the PREOF function to generate a packet output chain may include:

[0101] Step a11: Identify whether the current DBQ packet is an out-of-order packet. If it is not an out-of-order packet, execute step a12. If it is an out-of-order packet, execute step a13.

[0102] Optionally, in this embodiment, there are many ways to identify whether the current DBQ packet is an out-of-order packet, such as identifying the service flow to which the current DBQ packet belongs, determining that the packet sequence number of the current DBQ packet is greater than the maximum sequence number of packets belonging to the above-mentioned service flow that have been recorded and processed by invoking the PREOF function, and if the packet sequence number of the current DBQ packet is not adjacent to the above-mentioned maximum sequence number, determining that the current DBQ packet is an out-of-order packet; otherwise, determining that the current DBQ packet is not an out-of-order packet.

[0103] Step a12: put the current DBQ packet into the packet output chain mentioned above.

[0104] Step a12 is executed on the assumption that the current DBQ packet is not identified as an out-of-order packet. If the current DBQ packet is identified as not an out-of-order packet, the current DBQ packet may be directly input to the packet output chain, and the maximum recorded sequence number may be updated to the packet sequence number of the current DBQ packet, in order to facilitate determining whether a subsequent DBQ packet is an out-of-order packet.

[0105] Step a13: determine the service flow to which the current DBQ packet belongs; if the current DBQ packet satisfies a condition, put the current DBQ packet, at least one recorded packet that is correctly aligned with the current DBQ packet, and a recorded packet that belongs to the service flow and has been processed by invoking the PREOF function outside the predetermined time window but has not been put into the packet output chain into the packet output chain; if the current DBQ packet does not satisfy the condition, put the recorded packet that belongs to the service flow and has been processed by invoking the PREOF function outside the predetermined time window but has not been put into the packet output chain into the packet output chain, where the condition refers to the correct alignment between the current DBQ packet and at least one recorded packet that belongs to the service flow and has been processed by invoking the PREOF function within the predetermined time window.

[0106] If the current DBQ packet is identified as an out-of-order packet, the service flow to which the current DBQ packet belongs is identified (the specific service flow can be added to the packet), and then it is determined whether the current DBQ packet is properly aligned with at least one recorded packet belonging to the service flow and processed by invoking the PREOF function within the predetermined time window (i.e., whether the sequence number of at least one packet is consecutive with the sequence number of the current DBQ packet). For example, if it is determined that the packet sequence number of the current DBQ packet is 8 and the packet sequence number of a recorded packet belonging to the service flow and processed by invoking the PREOF function within the predetermined time window is 7, this means that the current DBQ packet is properly aligned with at least one recorded packet belonging to the service flow and processed by invoking the PREOF function within the predetermined time window. Note that the predetermined time window can be set according to actual needs, such as to the closest 10 μs from the current time, and is not particularly limited in this embodiment.

[0107] In one embodiment, if the current DBQ packet and at least one recorded packet belonging to the service flow and processed by invoking the PREOF function within the predetermined time window are aligned correctly, the current DBQ packet, the at least one recorded packet that is aligned correctly with the current DBQ packet, and the recorded packets belonging to the service flow and processed by invoking the PREOF function outside the predetermined time window but not yet put into the packet output chain are put into the packet output chain, and finally, a packet output chain is generated.

[0108] In one embodiment, if the current DBQ packet and at least one packet belonging to the recorded service flow and processed by invoking the PREOF function within the predetermined time window are not properly aligned, packets belonging to the recorded service flow and processed by invoking the PREOF function outside the predetermined time window but not yet placed in the packet output chain are placed in the packet output chain. Finally, a packet output chain is generated. Note that if the current DBQ packet and at least one packet belonging to the recorded service flow and processed by invoking the PREOF function within the predetermined time window are not properly aligned, optionally, the timestamp of the current DBQ packet and the timestamp of invoking the PREOF function to process the current DBQ packet may also be recorded.

[0109] Finally, the packet output chain is generated by the above steps a11 to a13.

[0110] The following describes how the target packet is encapsulated in step 503 above.

[0111] Optionally, the encapsulation of the target packet in step 503 above may include:

[0112] Step b11: Determine whether the target packet is an edge access packet. If it is an edge access packet, execute step b12; if it is not an edge access packet, execute step b13.

[0113] Optionally, in this embodiment, determining whether the target packet is an edge access packet may depend on whether the target packet is encapsulated with network information, for example, if the target packet is not encapsulated with network information such as SR information, determining that the target packet is an edge access packet; otherwise, determining that the target packet is not an edge access packet. Note that this is merely an example provided to explain how to determine whether the target packet is an edge access packet, and is not intended to be limiting.

[0114] Step b12: The first type of encapsulation is performed on the target packet.

[0115] This step b12 is performed on the assumption that the target packet is determined to be an edge access packet. If the target packet is determined to be an edge access packet, the target packet may be subjected to a first type of encapsulation as described in step b12.

[0116] Optionally, when performing the first type encapsulation on the target packet, this embodiment introduces a new concept called a fast forwarding table (FastFwdTbl). In one embodiment, the fast forwarding table may be obtained by converting routing information in a deterministic network. DFT can quickly obtain the out-interface of the target packet by querying the fast forwarding table, and then find the associated DTQ (i.e., the DTQ corresponding to the out-interface).

[0117] In this embodiment, deterministic flows are forwarded through the fast forwarding table in order to separate them from the best-effort flow forwarding process, prevent the deterministic flows from being affected by the best-effort flow forwarding, shorten the deterministic flow forwarding process, increase the deterministic flow forwarding speed, and provide guarantees for the deterministic flow forwarding.

[0118] In concrete implementation, the high-speed forwarding table has a smaller capacity and faster lookup than the best-effort FIB table, and there is almost no difference in the overhead of the lookup table, thereby reducing uncertainty.

[0119] In one embodiment, optionally, the fast forwarding table includes at least Destination IP, Source IP, Destination Port, Source Port, Proto (Protocol), Differentiated Services Code Point (DSCP), Out Interface, and Tunnel Index (LIndex).

[0120] As an example of step b12, based on the above-mentioned fast forwarding table, the OutIf and LIndex are found in the fast forwarding table based on packet characteristic information added to the target packet, such as the source IP address and destination IP address. Then, from all the created out-interface information (OutIfInfo), the OutIfInfo corresponding to the OutIf is found. Here, the OutIfInfo includes out-interface information such as at least the MAC address and a pointer to a segment routing list (SRList) based on the out-interface (here, segment routing SR is used as an example). The corresponding SRList is then obtained based on the LIndex and the pointer in the SRList. The target packet is encapsulated based on the SRList and the MAD address in the OutIfInfo. Finally, the first type of encapsulation for the target packet is achieved. Figure 6 shows, as an example, a schematic diagram illustrating the relationship between the fast forwarding table, OutIfInfo, and SRLists.

[0121] Step b13: perform the second type encapsulation on the target packet.

[0122] Step b13 is executed on the assumption that it is determined that the target packet is not an edge access packet. If it is determined that the target packet is not an edge access packet, this means that the target packet has undergone the above-mentioned first type encapsulation. Based on this, in response to the first type encapsulation exemplified in step b12 above, in step b13, OutIf is determined based on the SID in the SRLists in which the target packet is encapsulated, and OutIfInfo corresponding to OutIf is found (e.g., including at least a MAC address, etc.), and based on the obtained OutIfInfo (e.g., including at least a MAC address, etc.), encapsulation replacement may be performed on the target packet, such as replacing the source MAC address in the encapsulated second layer header with the MAC address of the out-interface in OutIfInfo, and this embodiment is not particularly limited.

[0123] Finally, the target packet is encapsulated in step 503 through the above steps b11 to b13.

[0124] The flow of realizing the CFT will be explained below using the flow shown in Figure 7.

[0125] 7 is a flowchart illustrating the implementation of a CFT provided in an embodiment of the present invention. In this embodiment, the CFT can be started by the DFT, as described in step 504 or step 506 of the flow shown in FIG. 5 above. In another embodiment, when a new CFT is created on a network device, the CFT can be started directly. For example, when a new CFT is created on a network device, if the CFT detects that there is an event in the CSQ to which the CFT is bound that requires timely packet forwarding, the CFT can be started directly.

[0126] Before explaining the flow shown in FIG. 7, the settings of each CFT will be explained.

[0127] In one embodiment, three corresponding local variables may be set for each CFT: CFT_Jiffies, CFT_prev_Jiffies, and CFT_Rcv_Jiffies.

[0128] Here, cft_prev_Jiffies can indicate the currently processed period and may be referred to as a current processing period variable, CFT_prev_Jiffies is used to indicate a history period and may be referred to as a history period variable, and CFT_Rcv_Jiffies is used to record the currently processed receiving period and may be referred to as a packet receiving period variable.

[0129] Based on the above description, as shown in FIG. 7, the flow may include the following steps:

[0130] Step 701: After the CFT is started, the current value of CSQF_Jiffies is read and CFT_Jiffies is updated to the current value of CSQF_Jiffies that has been read. Then, step 702 is executed.

[0131] In step 702, it is determined whether the packet transmission period has been updated. If it has been updated, step 703 is executed; if it has not been updated, step 704 is executed.

[0132] As described above, CFT_Jiffies indicates the current processing period variable, and CFT_Prev_Jiffies indicates the historical period variable, so it is possible to determine whether the packet transmission period has been updated based on CFT_Jiffies and CFT_Prev_Jiffies. For example, if CFT_Jiffies and CFT_Prev_Jiffies are not equal, this indicates that a period rotation has been performed, such as a period rotation from the first period to the second period, and the packet transmission period has been updated. If CFT_Jiffies and CFT_Prev_Jiffies are equal, this means that the packet transmission period has not been updated.

[0133] Step 703: Readjust the CSQP corresponding to the CFT and update the value of CFT_Prev_Jiffies to the current value of CFT_Jiffies. Then, step 704 is executed.

[0134] As mentioned above, the CSQP includes at least the SQP, the RQP, and the TQP. Optionally, in one embodiment, the number of the CSQ pointed to by the re-adjusted SQP is determined based on the remainder obtained by modulo operation of CFT_Jiffies and the total number d of CSQs bound to the CFT, where the remainder may be denoted as Rem.

[0135] The number of the CSQ pointed to by the readjusted RQP is determined based on the result obtained by performing a modulo operation on the above d, which is the sum of Rem and the specified jitter period number Jitter. For example, the number of the CSQ pointed to by the readjusted RQP is determined according to (Rem + Jitter) mod d.

[0136] The number of the CSQ to which the readjusted TQP points can be determined according to (d-1+Rem-Jitter) mod d.

[0137] It should be noted that the above-described method for readjusting the CSQP corresponding to the CFT is merely an example and is not intended to be limiting.

[0138] Step 704: traverse the packets in the SQ, determine the traversed packet as the current SQ packet, send the current SQ packet to the out-interface for forwarding and forward it, then determine whether there are any untraversed packets in the SQ, if there are any untraversed packets in the SQ, continue traversing the untraversed packets, and return to the step of determining the traversed packet as the current SQ packet in step 704, if there are no untraversed packets in the SQ, execute step 705.

[0139] Step 705: traverse the DTQs bound to the CFT, determine the traversed DTQ as the current DTQ, check whether there is a packet in the current DTQ, if there is a packet in the current DTQ, execute step 706, if there is no packet in the current DTQ, execute step 708.

[0140] This step 705 is executed when there are no untraversed packets in the SQ (ie, the SQ is empty).

[0141] Step 706 updates the value of CFT_Rcv_Jiffies to the current value of CSQF_Jiffies, traverses the packets in the current DTQ, determines the traversed packet as the current DTQ packet, and determines the target CSQ corresponding to the current DTQ packet based on the cycle parameter (Cycle) attached to the current DTQ packet. Then, step 707 is executed.

[0142] Based on the above CSQF_Jiffies and CFT_Rcv_Jiffies, the value of CFT_Rcv_Jiffies is updated to the current value of CSQF_Jiffies here in order to control the timely execution of CFT threads to reduce scheduling delays.

[0143] In this embodiment, a cycle-specified queue (Cycle) parameter indicating the target CSQ is added to the current DTQ packet, so that it is easy to determine the target CSQ corresponding to the current DTQ packet based on the indication of the Cycle parameter.

[0144] Step 707: If the target CSQ is between SQ and RQ, put the current DTQ packet into RQ; if the target CSQ is between TQ and SQ, put the current DTQ packet into TQ; if the target CSQ is between RQ and TQ, put the current DTQ packet into the target CSQ; then determine whether there are any untraversed packets in the current DTQ; if there are any untraversed packets in the current DTQ, continue traversing the untraversed packets; return to step 706, the step of determining the traversed packet as the current DTQ packet; if there are no untraversed packets in the current DTQ, execute step 708.

[0145] In this embodiment, if the target CSQ is between SQ and RQ, it indicates that the current DTQ packet may arrive late, and the current DTQ packet may be placed in RQ to ensure that the late-arriving current DTQ packet waits a short time to be transferred. On the other hand, if the target CSQ is between TQ and SQ, it indicates that the current DTQ packet may arrive early, and the early-arriving current DTQ packet may be limited to waiting a relatively long time to be transferred, that is, the current DTQ packet may be placed in TQ to ensure that other packets are transferred in a timely manner. Figure 9 shows the structure between SQ, RQ, and TQ as an example.

[0146] Step 708: Determine whether there are any untraversed DTQs among all DTQs bound to the CFT; if there are any untraversed DTQs among all DTQs bound to the CFT, continue traversing the untraversed DTQs and return to step 705, which determines the traversed DTQ as the current DTQ; otherwise, execute step 709.

[0147] Step 709: Detect whether there is an event in the CSQ bound to the CFT that requires packets to be forwarded in a timely manner; if there is no event, the CFT enters a sleep state and waits to be woken up; if there is an event, return to executing step 701 described above.

[0148] Here, the event requiring packet transfer in a timely manner is determined based on the above-mentioned CFT_Jiffies and CFT_Rcv_Jiffies, and optionally, if the difference between the value of CFT_Jiffies and the value of CFT_Rcv_Jiffies is equal to or less than a predetermined difference, it is determined that there is an event requiring packet transfer in a timely manner, and otherwise it is determined that there is no event requiring packet transfer in a timely manner. Note that in this embodiment, the predetermined difference is set in advance, and is not particularly limited to, for example, 10 periods (the period for generating the above-mentioned timer interrupt).

[0149] This completes the flow shown in Figure 7.

[0150] FIG. 7 explains the process that CFT executes after it is started.

[0151] The above provides examples of the processes that are executed after the distribution thread, DFT, and CFT are launched. However, as for the BFT, after it is launched, it can directly obtain packets from the BTQ, forward them to the out-interface for forwarding the packets, and then forward the packets, so detailed explanations are omitted here.

[0152] Optionally, in this embodiment, the following restrictions may be desirable to ensure reliability: (1) Each DBQ generally corresponds to only one distribution thread, and based on the above-mentioned correspondence between DBQs and packet receiving queues, a one-to-one correspondence between DBQs, distribution threads, and packet receiving queues can ultimately be realized, and each distribution thread is responsible for distributing packets belonging to a deterministic flow in a packet receiving queue to the DBQ corresponding to the distribution thread. (2) Regarding DBQ, one DBQ is generally bound to only one DFT. (3) Regarding DTQs, a DTQ is generally bound to one CFT, which processes packets queued by only one DFT and is responsible for processing packets sent from the DTQ. (4) Regarding CSQs, one CSQ generally has packets queued by only one CFT, and the same CFT is responsible for processing packets sent from that CSQ.

[0153] Based on the above explanation, Fig. 10 illustrates an example of the operations between threads based on the above restrictions. Fig. 10 shows an example in which one interface is associated with one CFT, and packets output through each interface are scheduled by the CFT associated with that interface.

[0154] Furthermore, in this embodiment, the DBQ and DTQ are designed as circular queues to enable lock-free operation and improve concurrency.

[0155] In addition, DisThr, DFT, and CFT are real-time priority scheduling threads, where DFT threads and CFT threads do not share CPU cores with other real-time priority threads, nor with threads that occupy CPU resources for long periods of time without releasing them.

[0156] Now that the method provided in the embodiment of the present invention has been described, the apparatus provided in the embodiment of the present invention will be described below.

[0157] 11 is a structural diagram illustrating an apparatus provided in an embodiment of the present invention, the apparatus being applied to a network device, a distribution thread that, after being activated, distributes packets in a packet receiving queue bound to it, thereby distributing packets belonging to a deterministic flow in the packet receiving queue to a corresponding DBQ and distributing packets belonging to a best-effort flow to a corresponding BTQ, wherein the packets in the packet receiving queue are packets received from outside via a local interface of the network device; A deterministic forwarding thread (DFT) is started and puts packets in the bound DBQ into the corresponding DTQ. and a periodic transfer thread (CFT) that, after being activated, places packets in the bound DTQ into a corresponding periodic specified queue (CSQ), selects a packet from a transmit queue (SQ), and forwards the packet through an out-interface for forwarding the packet, wherein the SQ is the CSQ currently pointed to by a transmit queue pointer (SQP) corresponding to the CFT.

[0158] Optionally, the distribution thread distributing packets in a bound packet receive queue comprises: traversing packet receiving queues bound to the distribution thread, determining the currently traversed packet receiving queue as a current queue, and checking whether there are packets in the current queue; If there are packets in the current queue, traverse the packets in the current queue and determine the traversed packet as the current packet; if the current packet belongs to a deterministic flow, put the current packet into a DBQ corresponding to the current queue; if the current packet belongs to a best-effort flow, put the current packet into a BTQ corresponding to the current queue; then, if there are packets in the current queue that have not been traversed, continue traversing the untraversed packets and return to the step of determining the traversed packet as the current packet; If there are no packets in the current queue or there are no untraversed packets in the current queue, when there are untraversed packet receiving queues among all packet receiving queues bound to the distribution thread, continuing to traverse other packet receiving queues bound to the distribution thread and untraversed, and returning to the step of determining the currently traversed packet receiving queue as the current queue.

[0159] Optionally, the distribution thread has a corresponding thread polling flag set, and when the distribution thread is started, the thread polling flag is set to FALSE; If the distribution thread detects that there is a packet in the current queue, further set the thread polling flag to TRUE to instruct the distribution thread to turn off a designated function set thereto, where turning off the designated function is to instruct the distribution thread to prevent the distribution thread from entering a sleep state; After all packet receive queues bound to the distribution thread have been traversed, If the thread polling flag is TRUE, set the thread polling flag to FALSE and return to the step of traversing the packet receiving queue bound to the distribution thread; if the thread polling flag is FALSE, instruct the specified function to be turned on, and turning on the specified function is to instruct the distribution thread to wait to be activated.

[0160] Optionally, when the distribution thread distributes packets belonging to a deterministic flow in the packet receiving queue to a corresponding DBQ, the distribution thread further If the DFT startup flag set for the DFT bound to the DBQ is FALSE, set the DFT startup flag to TRUE and start the DFT whose DFT startup flag is TRUE, and / or After all packet receive queues bound to the distribution thread have been traversed, For a DFT whose DFT startup flag is TRUE, the DFT is started and the DFT startup flag of the DFT is set to FALSE.

[0161] Optionally, packets in a DFT-bound DBQ are placed in the corresponding DTQ. traversing DBQs bound to the DFT, determining the DBQ being traversed as a current DBQ, and checking whether there are packets in the current DBQ; If there are packets in the current DBQ, traverse the packets in the current DBQ, determine the traversed packet as the current DBQ packet, call a configured packet duplication removal and sorting function (PREOF function), and if it is determined that the current DBQ packet is not a duplicate packet, generate a packet output chain including at least the current DBQ packet, encapsulate each DBQ packet in the packet output chain, and put the encapsulated DBQ packets into a DTQ corresponding to an out-interface for forwarding the DBQ packet; then, if there are packets in the current DBQ that have not been traversed, continue traversing the untraversed packets, and return to the step of determining the traversed packet as the current DBQ packet; If there are no packets in the current DBQ or if there are no untraversed packets in the current DBQ, and if there are untraversed DBQs in all DBQs bound to the DFT, continuing to traverse the untraversed DBQs and returning to the step of determining the traversed DBQ as the current DBQ.

[0162] Optionally, said packet output chain comprises: Identifying whether the current DBQ packet is an out-of-order packet, and if the current DBQ packet is identified as not being an out-of-order packet, putting the current DBQ packet into the packet output chain; If the current DBQ packet is identified as an out-of-order packet, the service flow to which the current DBQ packet belongs is determined; and if the current DBQ packet and at least one recorded packet belonging to the service flow and processed by invoking a PREOF function within a predetermined time window are properly aligned, the current DBQ packet, the at least one recorded packet that is properly aligned with the current DBQ packet, and recorded packets belonging to the service flow and processed by invoking a PREOF function outside the predetermined time window but not placed in the packet output chain are placed in the packet output chain; otherwise, the packet that is recorded belonging to the service flow and processed by invoking a PREOF function outside the predetermined time window but not placed in the packet output chain is placed in the packet output chain.

[0163] Optionally, the DFT has a corresponding DFT polling flag set, and when the DFT is invoked, the DFT polling flag is set to FALSE; When the DFT detects that there is a packet in the current DBQ, the DFT sets the DFT polling flag to TRUE to turn off the designated function set for the DFT, and turning off the designated function is to instruct the DFT to prevent itself from entering a sleep state; After all DBQs bound to the DFT have been traversed, If the DFT polling flag is TRUE, set the DFT polling flag to FALSE and return to the step of traversing the DBQ bound to the invoked DFT. If the DFT polling flag is FALSE, turn on the specified function. Turning on the specified function is to instruct the DFT to wait to be invoked.

[0164] Optionally, after the DFT places the DBQ packet into a DTQ corresponding to an out-interface for forwarding the DBQ packet, further: If the CFT startup flag set for the CFT bound to the DTQ is FALSE, set the CFT startup flag to TRUE and start the CFT whose CFT startup flag is TRUE; After all DTQs bound to the DFT have been traversed, for any CFT whose CFT activation flag is set to TRUE, the CFT is activated and the CFT activation flag of the CFT is set to FALSE.

[0165] Optionally, the CFT queues packets in the bound DTQ into a corresponding cycle-specific queue (CSQ), traversing the DTQs bound to said CFT, determining the traversed DTQ as the current DTQ, and checking whether there is a packet in the current DTQ; If there are packets in the current DTQ, traverse the packets in the current DTQ, determine the traversed packet as the current DTQ packet, determine a target CSQ corresponding to the current DTQ packet based on a cycle parameter attached to the current DTQ packet, and if the target CSQ is between SQ and RQ, put the current DTQ packet into RQ, if the target CSQ is between TQ and SQ, put the current DTQ packet into TQ, if the target CSQ is between RQ and TQ, put the current DTQ packet into target CSQ, and then, if there are packets in the current DTQ that have not been traversed, return to the step of traversing the untraversed packets and determining the traversed packet as the current DTQ packet, wherein the RQ is the CSQ currently pointed to by a receive queue pointer (RQP) corresponding to the CFT, and the TQ is the CSQ currently pointed to by an admission queue pointer (TQP) corresponding to the CFT; If there are no packets in the current DTQ or if there are no untraversed packets in the current DTQ, if there are untraversed DTQs in all DTQs bound to the CFT, continue traversing the untraversed DTQs and return to the step of determining the traversed DTQ as the current DTQ.

[0166] Optionally, the CFT queues packets in the bound DTQ into the corresponding cycle specific queue (CSQ) when the SQ is empty.

[0167] Optionally, the CFT selecting a packet from a transmission queue (SQ) and forwarding the packet via an out-interface for forwarding the packet comprises: If it is determined that the packet transmission period has been updated, readjust a period specification queue pointer structure (CSQP) corresponding to the CFT, traverse packets in the SQ pointed to by the readjusted SQP, determine the traversed packet as the current SQ packet, and forward the current SQ packet by calling an out-interface for forwarding the current SQ packet; and then, if there is a packet in the SQ that has not been traversed, continue traversing the untraversed packet and return to the step of determining the traversed packet as the current SQ packet, wherein the CSQP includes at least an SQP; If it is determined that the packet transmission period has not been updated, the method includes traversing the packets in the SQ, determining the traversed packet as the current SQ packet, and forwarding the current SQ packet by calling an out-interface for forwarding the current SQ packet, and then, if there is a packet in the SQ that has not been traversed, continuing to traverse the untraversed packet and returning to the step of determining the traversed packet as the current SQ packet.

[0168] Optionally, the CSQP further comprises a receive queue pointer (RQP) and a tolerance queue pointer (TQP), the RQP pointing to the CSQ designated as the receive queue (RQ) and the TQP pointing to the CSQ designated as the tolerance queue (TQ); Here, the number of the CSQ pointed to by the re-adjusted SQP is determined based on the remainder obtained by modulo operation of the current processing period variable CFT_Jiffies set for the CFT and the number d of CSQs bound to the CFT, The number of the CSQ indicated by the readjusted RQP is determined based on the result obtained by performing a modulo operation on the d by adding the remainder and a specified jitter period number Jitter; The CSQ number that the rebalanced TQP points to is: It is determined according to (d-1+Rem-Jitter) mod d, where Rem is the remainder.

[0169] Optionally, after the CFT queues all packets in all bound DTQs into the corresponding cycle-specific queues (CSQs), Detect whether there is an event in the CSQ bound to the CFT that requires a packet to be forwarded in a timely manner, and if there is an event, select a packet from the transmission queue (SQ) by the CFT and return to the execution of the step of forwarding the packet through the out-interface for forwarding the packet, and if there is no event, wait to be triggered again.

[0170] Optionally, the event requiring packet transfer in a timely manner is determined based on a current processing period variable CFT_Jiffies and a current packet receiving period variable CFT_Rcv_Jiffies set for the CFT, and if a difference between the value of CFT_Jiffies and the value of CFT_Rcv_Jiffies is equal to or less than a predetermined difference, it is determined that there is an event requiring packet transfer in a timely manner, and otherwise it is determined that there is no event requiring packet transfer in a timely manner; When the CFT is activated or when an event that requires timely packet forwarding is detected in a CSQ bound to the CFT, the value of the CFT_Jiffies is updated to the current value of the configured CSQF_Jiffies, and the CSQF_Jiffies is used to indicate that a periodic counting of the designated periodic queuing and forwarding (CSQF) is to be realized based on segment routing; If the current DTQ is checked to see if there are any packets, the value of CFT_Rcv_Jiffies is updated to the value of CFT_Jiffies.

[0171] Optionally, whether the packet transmission period has been updated or not is determined by: It is determined by checking whether the value of the current processing period variable CFT_Jiffies set for the CFT is equal to the value of the history period variable CFT_Prev_Jiffies, and if they are equal, determining that the packet transmission period has not been updated, and if they are not equal, determining that the packet transmission period has been updated; After the cycle-specific queue pointer structure (CSQP) corresponding to the CFT is re-adjusted, the method further comprises: updating the value of the CFT_Prev_Jiffies to the current value of the CFT_Jiffies.

[0172] This completes the description of the apparatus provided in the embodiments of the present invention.

[0173] The embodiment of the present invention further provides a hardware structure of the device shown in FIG.

[0174] Referring to Figure 12, there is shown a structural diagram of an electronic device provided in an embodiment of the present invention. As shown in Figure 12, the hardware structure may include a processor and a machine-readable storage medium, in which machine-executable instructions executable by the processor are stored, and the processor is used to execute the machine-executable instructions to implement the methods disclosed in the above examples of the present invention.

[0175] Based on the same inventive idea as the above-mentioned method, an embodiment of the present invention further provides a machine-readable storage medium storing a plurality of computer instructions, which can be executed by a processor so as to implement the method disclosed in the above example of the present invention.

[0176] By way of example, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that contains or can store information such as executable instructions, data, etc. For example, the machine-readable storage medium may be a RAM (random access memory), a volatile memory, a non-volatile memory, a flash memory, a storage drive (such as a hard drive), a solid-state drive, any storage disk (such as an optical disk, a DVD), or a similar storage medium, or a combination thereof.

[0177] The systems, devices, modules, or units described in the above embodiments may be specifically realized by computer chips, entities, or products having certain functions. A typical realizing device is a computer, and the specific form of the computer may be a personal computer, a laptop computer, a mobile phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email sending / receiving device, a game console, a tablet, a wearable device, or any combination of these devices.

[0178] For the sake of convenience, the above-described device will be described by dividing it into various units according to their functions. However, when implementing the present invention, the functions of each unit may be realized by the same or multiple pieces of software and / or hardware.

[0179] As will be appreciated by those skilled in the art, embodiments of the present invention may be provided as a method, a system, or a computer program product. Thus, the present invention may employ embodiments consisting entirely of hardware, embodiments consisting entirely of software, or embodiments combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product embodied in one or more computer-usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0180] The present invention will be described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, whereby the instructions, executed by the processor of the computer or other programmable data processing device, generate an apparatus for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0181] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture that includes an instruction apparatus that implements the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0182] These computer program instructions may be loaded into a computer or other programmable data processing device, whereby a series of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0183] The above is merely an embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will appreciate that the present invention may undergo various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principles of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A deterministic flow transmission method applied to a network device, comprising: Distributing packets in a packet receive queue bound to the distribution thread by an activated distribution thread on the device, whereby packets belonging to a deterministic flow in the packet receive queue are distributed to a corresponding deterministic flow buffering queue (DBQ) and packets belonging to a best effort flow are distributed to a corresponding best effort flow queue (BTQ), wherein the packets in the packet receive queue are packets received externally via a local interface of the network device; by an activated Deterministic Forwarding Thread (DFT) on the device, queueing packets in a DBQ bound to the DFT into a corresponding Deterministic Flow Queue (DTQ); By an activated periodic transfer thread (CFT) on the device, packets in a DTQ bound to the CFT are queued to a corresponding periodic specified queue (CSQ), and the CFT selects a packet from a transmit queue (SQ) and transmits the packet via an out-interface for packet transmission, the SQ being the CSQ currently pointed to by a transmit queue pointer (SQP) corresponding to the CFT; and fetching, by a launched best effort forwarding thread (BFT) on the device, a packet from a BTQ bound to the BFT and forwarding the packet via an out-interface for packet forwarding; The DFT and the CFT are configured not to share a CPU core with the BFT, Packets belonging to the deterministic flow are assigned the highest priority on an outgoing interface for packet forwarding. A deterministic flow transmission method comprising:

2. Distributing, by the distribution thread, packets in a packet receiving queue bound to the distribution thread, traversing packet receive queues bound to the distribution thread, determining the packet receive queue being traversed as a current queue, and checking whether there are packets in the current queue; If there is a packet in the current queue, traverse the packets in the current queue, determine the traversed packet as the current packet, and if the current packet belongs to a deterministic flow, put the current packet into the DBQ corresponding to the current queue; if the current packet belongs to a best-effort flow, put the current packet into the BTQ corresponding to the current queue; If there is an untraversed packet in the current queue, continuing to traverse the untraversed packets and returning to the step of determining the traversed packet as the current packet; If there is no packet in the current queue or there is no untraversed packet in the current queue, when there is an untraversed packet receiving queue among all packet receiving queues bound to the distribution thread, continuing to traverse the packet receiving queues bound to the distribution thread and untraversed, and returning to the step of determining the traversed packet receiving queue as the current queue.

2. The method of claim 1 .

3. The distribution thread has a corresponding thread polling flag set, and when the distribution thread is started, the thread polling flag is set to FALSE; If it is detected that there is a packet in the current queue, further set the thread polling flag to TRUE to instruct the distribution thread to turn off a designated function set thereto, where turning off the designated function is to instruct the distribution thread to prevent it from entering a sleep state; After all packet receive queues bound to the distribution thread have been traversed, If the thread polling flag is TRUE, setting the thread polling flag to FALSE and returning to the step of traversing the packet receive queue bound to the distribution thread; if the thread polling flag is FALSE, instructing the designated function to be turned on, where turning on the designated function is to instruct the distribution thread to wait to be woken up.

3. The method of claim 2.

4. When the distribution thread distributes packets belonging to a deterministic flow in the packet receiving queue to a corresponding DBQ, If a DFT activation flag set for a DFT bound to the DBQ is FALSE, setting the DFT activation flag to TRUE and activating the DFT whose DFT activation flag is TRUE; and / or After all packet receive queues bound to the distribution thread have been traversed, For a DFT whose DFT activation flag is TRUE, activating the DFT and setting the DFT activation flag of the DFT to FALSE is further included; 4. The method according to claim 2 or 3, characterized in that

5. Queueing packets in a DBQ bound to the DFT into a corresponding Deterministic Flow Queue (DTQ) by an activated Deterministic Forwarding Thread (DFT) on the device, traversing DBQs bound to said DFT, determining the DBQ being traversed as a current DBQ, and checking whether there are packets in said current DBQ; If there are packets in the current DBQ, traverse the packets in the current DBQ, determine the traversed packet as a current DBQ packet, and if the current DBQ packet is identified as not being a duplicate packet by calling a configured packet duplication, removal, and alignment function (PREOF function), generate a packet output chain, encapsulate each DBQ packet in the packet output chain, and put the encapsulated DBQ packets into a DTQ corresponding to an out-interface for forwarding the DBQ packet; If there are untraversed packets in the current DBQ, continuing to traverse the untraversed packets in the current DBQ and returning to the step of determining the traversed packet as the current DBQ packet; If there are no packets in the current DBQ or there are no untraversed packets in the current DBQ, when there are untraversed DBQs in all DBQs bound to the DFT, continue traversing the untraversed DBQs and return to the step of determining the traversed DBQ as the current DBQ.

2. The method of claim 1 .

6. The packet output chain comprises: if the current DBQ packet is identified as not being an out-of-order packet by calling the PREOF function, placing the current DBQ packet into the packet output chain; if the current DBQ packet is identified as an out-of-order packet by invoking the PREOF function, determining a service flow to which the current DBQ packet belongs; if the current DBQ packet satisfies a condition that the current DBQ packet and at least one recorded packet belonging to the service flow and processed by invoking a PREOF function within a predetermined time window are properly aligned, then place the current DBQ packet, the at least one recorded packet properly aligned with the current DBQ packet, and a recorded packet belonging to the service flow and processed by invoking a PREOF function outside the predetermined time window into the packet output chain; if the current DBQ packet does not satisfy the condition, place a recorded packet belonging to the service flow and processed by invoking a PREOF function outside the predetermined time window into the packet output chain.

6. The method of claim 5.

7. The DFT has a corresponding DFT polling flag set, and when the DFT is started, the DFT polling flag is set to FALSE; If it is detected that there is a packet in the current DBQ, further set the DFT polling flag to TRUE to turn off the designated function set for the DFT, and turning off the designated function is to indicate that the DFT is prevented from entering a sleep state; After all DBQs bound to the DFT have been traversed, If the DFT polling flag is TRUE, setting the DFT polling flag to FALSE and returning to the step of traversing the DBQ bound to the activated DFT; if the DFT polling flag is FALSE, turning on the designated function, where turning on the designated function is to instruct the DFT to wait to be activated.

6. The method of claim 5.

8. After the DFT places the DBQ packet into a DTQ corresponding to an out-interface for transferring the DBQ packet, If a CFT activation flag set for a CFT bound to the DTQ is FALSE, setting the CFT activation flag to TRUE and activating the CFT whose CFT activation flag is TRUE; After all DBQs bound to the DFT have been traversed, For a CFT whose CFT activation flag is set to TRUE, activating the CFT and setting the CFT activation flag of the CFT to FALSE is further included; 6. The method of claim 5.

9. Queueing packets in a DTQ bound to a periodic transfer thread (CFT) on the device into a corresponding periodic queue (CSQ) by the CFT, which is started, traversing the DTQs bound to said CFT, determining the DTQ being traversed as the current DTQ, and checking whether there are any packets in said current DTQ; If there is a packet in the current DTQ, traverse the packets in the current DTQ and determine the traversed packet as the current DTQ packet; determine a target CSQ corresponding to the current DTQ packet based on a cycle parameter attached to the current DTQ packet; if the target CSQ is between an SQ and a receive queue (RQ), place the current DTQ packet in the RQ; if the target CSQ is between an acceptance queue (TQ) and an SQ, place the current DTQ packet in the TQ; if the target CSQ is between the RQ and the TQ, place the current DTQ packet in the target CSQ; If there are any untraversed packets in the current DTQ, traversing one untraversed packet and returning to the step of determining the traversed packet as the current DTQ packet, wherein the RQ is the CSQ currently pointed to by the receive queue pointer (RQP) corresponding to the CFT, and the TQ is the CSQ currently pointed to by the admission queue pointer (TQP) corresponding to the CFT; If there is no packet in the current DTQ or there is no untraversed packet in the current DTQ, when there is an untraversed DTQ among all DTQs bound to the CFT, continue traversing one untraversed DTQ and return to the step of determining the traversed DTQ as the current DTQ.

2. The method of claim 1 .

10. By an activated Periodic Transfer Thread (CFT) on the device, packets in a DTQ bound to the CFT are queued into a corresponding Periodic Scheduled Queue (CSQ) when the SQ is empty.

10. The method of claim 1 or 9.

11. Selecting a packet from a transmission queue (SQ) by the CFT and forwarding the packet via an out-interface for packet forwarding includes: If it is determined that the packet transmission period has been updated, readjust a period-specified queue pointer structure (CSQP) corresponding to the CFT, which includes at least an SQP, traverse packets in the SQ pointed to by the readjusted SQP, determine the traversed packet as a current SQ packet, and forward the current SQ packet by calling an out-interface for forwarding the current SQ packet; If there is an untraversed packet in the SQ, traverse the untraversed packet and return to the step of determining the traversed packet as the current SQ packet; If it is determined that the packet transmission period has not been updated, traverse the packets in the SQ, determine the traversed packet as the current SQ packet, and transfer the current SQ packet by calling an out interface for transferring the current SQ packet. If there is an untraversed packet in the SQ, traversing the untraversed packet and returning to the step of determining the traversed packet as the current SQ packet.

10. The method of claim 9.

12. The CSQP further includes a receive queue pointer (RQP) and a acceptance queue pointer (TQP), where the RQP points to the CSQ designated as the receive queue (RQ) and the TQP points to the CSQ designated as the acceptance queue (TQ); Here, the number of the CSQ pointed to by the re-adjusted SQP is determined based on the remainder obtained by modulo operation of the current processing cycle variable CFT_Jiffies set for the CFT and the number d of CSQs bound to the CFT, The number of the CSQ indicated by the readjusted RQP is determined based on the result obtained by performing a modulo operation on the d by adding the remainder and a specified jitter period number Jitter; The CSQ number pointed to by the rebalanced TQP is: (d-1+Rem-Jitter) mod d, where Rem is the remainder.

12. The method of claim 11 .

13. After an activated Periodic Transfer Thread (CFT) on the device places all packets in all DTQs bound to that CFT into corresponding Periodic Scheduled Queues (CSQs), The method further includes detecting whether there is an event in a CSQ bound to the CFT that requires a packet to be forwarded in a timely manner, and if there is an event, returning to the execution of the step of selecting a packet from a transmission queue (SQ) by the CFT and forwarding the packet via an out-interface for packet forwarding, and if there is no event, waiting for the CFT to be activated again.

10. The method of claim 9.

14. The event requiring timely packet transfer is determined based on a current processing period variable CFT_Jiffies and a current packet receiving period variable CFT_Rcv_Jiffies set for the CFT, and if a difference between the value of CFT_Jiffies and the value of CFT_Rcv_Jiffies is equal to or less than a predetermined difference, it is determined that an event requiring timely packet transfer exists, and otherwise it is determined that no event requiring timely packet transfer exists; wherein, when the CFT is activated or when an event that requires timely packet forwarding is detected in a CSQ bound to the CFT, the value of the CFT_Jiffies is updated to the current value of the configured CSQF_Jiffies, and the CSQF_Jiffies is used to indicate that a periodic counting of CSQF is implemented based on segment routing; If the current DTQ is checked for packets, the value of CFT_Rcv_Jiffies is updated to the value of CFT_Jiffies.

14. The method of claim 13.

15. Whether the packet transmission period has been updated or not is determined by It is determined whether or not the value of the current processing period variable CFT_Jiffies set for the CFT is equal to the value of the history period variable CFT_Prev_Jiffies, and if they are equal, it is determined that the packet transmission period has not been updated, and if they are not equal, it is determined that the packet transmission period has been updated. After readjusting the cycle-specific queue pointer structure (CSQP) corresponding to the CFT, updating the value of the CFT_Prev_Jiffies to the current value of the CFT_Jiffies.

12. The method of claim 11 .

16. An electronic device including a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions executable by the processor, the processor being used to execute the machine-executable instructions to perform a method according to any one of claims 1 to 15. An electronic device characterized by:

Images