A joint routing and scheduling method and device based on multi-queue cyclic queuing and forwarding

Abstract

The present invention discloses a joint routing and scheduling method and device based on multi-queue cyclic queuing and forwarding, belonging to the technical field of network traffic scheduling. The method includes designing a multi-queue cyclic queuing and forwarding mechanism for each switch in the time-sensitive network; on the basis of ensuring the network requirements for the transmission of bursty flows, taking the maximum number of time-triggered flow schedules as the goal, modeling the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem, and using deep reinforcement learning for solution to obtain the receiving queue and transmission time slot of the next hop of the traffic flow in the transmission queue of each switch in the time-sensitive network. The multi-queue cyclic queuing and forwarding mechanism proposed by the present invention well handles the mixed flow scheduling problem of bursty flows and time-triggered flows.

Description

Technical Field

[0001] The present invention belongs to the technical field of network traffic scheduling, and in particular, to a joint routing and scheduling method and device based on multi-queue cyclic queuing and forwarding. Background Art

[0002] With the rapid development of industrial Internet of Things, intelligent manufacturing, and big data technologies, the demand for real-time and deterministic transmission of information has become increasingly prominent. Time Sensitive Networking (TSN) has emerged as a new industrial communication technology.

[0003] Time Sensitive Network is based on traditional Ethernet and supports the mixed transmission of periodic flows and aperiodic flows in the network through technologies such as clock synchronization mechanism, traffic shaping mechanism, gate array mechanism, and flow reservation, and can provide end-to-end deterministic, low-latency, and low-jitter transmission guarantee for data. Among them, traffic scheduling technology is a key technology in the current research of Time Sensitive Network. Existing research shows that the TSN traffic scheduling problem is a non-deterministic polynomial problem. Since there are often a large number of bursty flows with high latency requirements in actual industrial scenarios, and such bursty flows are highly dynamic and unpredictable, the mixed traffic scheduling mechanism needs to be focused on. At present, there is a lack of research on the mixed traffic scheduling method that includes bursty flows and time-triggered flows, and the influence of network topology has not been considered. Most studies only consider the case of a single Time Sensitive Network switch. Summary of the Invention

[0004] The purpose of the present invention is to provide a joint routing and scheduling method and device based on multi-queue cyclic queuing and forwarding, which realizes the joint routing and scheduling problem of mixed flows based on the multi-queue cyclic queuing and forwarding mechanism, and maximally guarantees the number of time-triggered flow scheduling while ensuring the network requirements for bursty flow transmission.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a joint routing and scheduling method based on multi-queue cyclic queuing and forwarding, including:

[0007] Designing a multi-queue cyclic queuing and forwarding mechanism for each switch in the Time Sensitive Network; the multi-queue cyclic queuing and forwarding mechanism means that transmission queues are set on the switch to send and receive traffic flows. In each transmission time slot, there is exactly one transmission queue for sending traffic flows, and the remaining transmission queues are used for receiving traffic flows; the traffic flows include time-triggered flows and bursty flows;

[0008] To meet the network requirements for the transmission of bursty flows, with the goal of maximizing the number of time-triggered flow schedules, the joint routing and scheduling problem of time-triggered flows and bursty flows is modeled as a multi-objective constrained optimization problem;

[0009] Determine the routing path of the traffic flows in the time-sensitive network. After each hop switch successfully receives the traffic, solve the multi-objective constrained optimization problem to determine the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

[0010] Preferably, the multi-objective constrained optimization problem is expressed as:

[0011] ,

[0012] ,

[0013] and needs to satisfy the following constraints:

[0014] ,

[0015] where, represents the total number of traffic flows, represents the set of traffic flows, represents the traffic flow , is the value function of the th transmission time slot, represents the number of transmission time slots within a scheduling period, is the weight coefficient, indicates whether the traffic flow is successfully scheduled, represents the network load balancing degree function, then represents the end-to-end transmission delay of the bursty flow from the source end to the destination end , represents the end-to-end delay of the traffic flow from the source node to the destination node, represents the traffic flow at the th node, the selected receiving queue offset value, represents the length of the transmission time slot, represents the traffic flow the total number of nodes passed through on the transmission path, represents the traffic flow the selected receiving queue offset value, represents the link bandwidth, represents the traffic flow size, represents the largest data packet, represents the sending time of the time-triggered flow, Represents the period of the time-triggered flow, Represents the service flow The deadline of, For the service flow The End-to-end delay of the packet, Represents the service flow The number of packets contained, Represents the service flow The End-to-end delay of the packet.

[0016] Preferably, the Is expressed as:

[0017] ,

[0018] Among them, Represents the service flow The Whether the packet has been assigned to the If the Then it represents the service flow The Packet has been assigned to the Transmission time slot.

[0019] Preferably, the network load balancing degree function represents:

[0020] ,

[0021] ,

[0022] ,

[0023] Among them, Represents the Load ratio of the transmission time slot, Represents the average value of the load ratios of all transmission time slots within the scheduling period, Represents the service flow The Packet, Represents the service flow Size of, Represents the service flow The number of packets contained, Represents the service flow The Whether the packet has been assigned to the Transmission time slot, Represents the Capacity size of the transmission time slot.

[0024] Preferably, the service flow The selected receiving queue offset value needs to satisfy the following constraints:

[0025] If the receiving queue is in front of the sending queue, then,

[0026] ,

[0027] If the receiving queue is behind the sending queue, then,

[0028] ,

[0029] where, represents the offset value of the receiving queue , represents the offset value of the sending queue, represents the sending queue;

[0030] When the burst flow is received by the switch of the first hop, select a receiving queue with a receiving queue offset value greater than or equal to 2. After the burst flow is successfully received by the switch of the first hop, always select a receiving queue with a receiving queue offset value of 1.

[0031] Preferably, the scheduling period is set to the least common multiple of the periods of all time-triggered flows.

[0032] Preferably, solving the multi-objective constraint optimization problem includes:

[0033] Set the state parameters of the environment as: ; The includes six attributes of the service flow and is represented as , where is the source node of the service flow , is the destination node of the service flow , is the size of the service flow , is the period of the service flow , is the deadline of the service flow , is the routing path of the service flow ;

[0034] Set the action parameter as: ; where, represents the action taken by the switch and represents the switch at the current transmission time slot The receiving queue for the next-hop selection of the traffic flow in the transmission queue , is the number of switches in the time-sensitive network;

[0035] Set the reward function as:

[0036] ;

[0037] wherein, represents the state , action at the time of the reward function;

[0038] Based on the set state parameters, action parameters, and reward function, use the deep reinforcement learning algorithm to solve, and obtain the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

[0039] Preferably, in the solving process, calculate the priority of each traffic flow in the following manner:

[0040] ,

[0041] wherein, represents the traffic flow priority, represents the weight coefficient, represents the traffic flow period, represents the traffic flow deadline, represents the traffic flow priority characteristic, and respectively represent the maximum number of routing hops and the minimum number of routing hops that the traffic flow passes through from the source node to the destination node;

[0042] Convert the priority to the probability of the traffic flow:

[0043] ,

[0044] wherein, is the probability of the traffic flow ;

[0045] Select the traffic flow with the highest probability for scheduling.

[0046] Preferably, determining the routing path of the traffic flow in the time-sensitive network includes:

[0047] According to the source node and destination node of the traffic flow, based on the shortest path algorithm, plan a path with the fewest number of routing hops passed through as the routing path of the traffic flow.

[0048] In a second aspect, the present invention provides a joint routing and scheduling device based on multi-queue cyclic queuing and forwarding, which is used to implement the above-mentioned joint routing and scheduling method based on multi-queue cyclic queuing and forwarding. The device includes:

[0049] A configuration module, which is used to design a multi-queue cyclic queuing and forwarding mechanism for each switch in the time-sensitive network; the multi-queue cyclic queuing and forwarding mechanism means that transmission queues are set on the switch to send and receive traffic flows. In each transmission time slot, there is one and only one transmission queue to send traffic flows, and the remaining transmission queues are used to receive traffic flows; the traffic flows include time-triggered flows and bursty flows;

[0050] A problem modeling module, which is used to model the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem with the goal of maximizing the number of scheduled time-triggered flows while ensuring the network requirements for bursty flow transmission;

[0051] An optimization scheduling module, which is used to determine the routing paths of traffic flows in the time-sensitive network. After each switch in each hop successfully receives, the multi-objective constrained optimization problem is solved to determine the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

[0052] The beneficial effects achieved by the present invention are as follows:

[0053] Aiming at the joint routing and scheduling problem of mixed flows in the time-sensitive network, the present invention proposes a joint routing and scheduling method based on multi-queue cyclic queuing and forwarding, which uses a deep reinforcement learning network to select the optimal injection transmission time slot and receiving queue, and combines the shortest path planning algorithm to perform routing planning to complete traffic flow scheduling. The multi-queue cyclic queuing and forwarding mechanism proposed by the present invention can well handle the mixed flow scheduling problem of bursty flows and time-triggered flows. Among them, a highly efficient policy sorting network is trained from a large number of mixed flow scheduling processes by using deep reinforcement learning, which significantly improves the final mixed flow scheduling result. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the multi-queue cyclic queuing and forwarding mechanism proposed in the embodiment of the present invention;

[0055] Figure 2 It is a topology diagram of the time-sensitive network in the embodiment of the present invention;

[0056] Figure 3 It is a schematic diagram of the mixed flow joint routing and scheduling process based on deep reinforcement learning in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0057] To make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below in conjunction with the embodiments and the drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but do not limit the present invention.

[0058] Herein, it should also be noted that in order to avoid obscuring the present invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the present invention are shown in the drawings, while other details less related to the present invention are omitted.

[0059] It should be emphasized that the term "comprising / including" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0060] It should be emphasized here that the step marks mentioned hereinafter are not intended to limit the order of the steps. Instead, it should be understood that the steps can be executed in the order mentioned in the embodiments, can be different from the order in the embodiments, or several steps can be executed simultaneously.

[0061] The traditional Cyclic Queuing and Forwarding (CQF) mechanism maintains two queue cycles at each interface of the network device to receive and send traffic flows. Based on the consideration of hybrid flow scheduling, the embodiments of the present invention improve the traditional cyclic queuing and forwarding mechanism and design a multi-queue cyclic queuing and forwarding mechanism as Figure 1 shown. In order to support the transmission of hybrid traffic of bursty flows and time-triggered flows, transmission queues are set on each time-sensitive network switch. The gating on these transmission queues periodically sends and receives traffic flows. In each transmission time slot, there is and only one transmission queue that can send traffic flows, and the remaining transmission queues can only receive traffic flows.

[0062] The time-sensitive network topology is as Figure 2 shown. The time-sensitive network topology is modeled as an undirected graph , where is the set of all time-sensitive network switches and terminals in the network, Figure 2 in which, SW1, SW2, SW3, SW4, SW5, SW6 are switches, is the set of links in the network.

[0063] Specifically, the queue states are divided into two states. The queue state for sending traffic flows is denoted as , where indicates that the queue ; the state for receiving traffic flows is denoted as , , indicates a queue . During the transmission time slot , the state of the queue is recorded as:

[0064] (1)

[0065] (2)

[0066] Wherein indicates the transmission time slot number, is expressed as the only queue for sending traffic flow in the transmission time slot , while represents the remaining queues for receiving traffic flow.

[0067] When the switch port receives traffic flow, in order to determine the queue for the traffic flow, the receiving queue offset value is the value of the sending queue corresponding to the current transmission time slot to the receiving queue . The offset value of the sending queue is recorded as , and based on the sending queue as the reference point, each queue increases by 1 in turn. Based on the definition of the queue offset value, it can be expressed as:

[0068] If the receiving queue is before the sending queue, that is , then,

[0069] (3)

[0070] If the receiving queue is after the sending queue, that is , then,

[0071] (4)

[0072] In addition, when receiving a burst stream, a queue with a queue offset value greater than or equal to 2 should be selected to receive the burst stream, that is:

[0073] (5)

[0074] Wherein represents the distance from the receiving queue to the sending queue. Setting the offset value greater than 2 is to prevent the phenomenon of packet loss due to the inability to fully receive the burst stream within the current time slot, ensuring the reliability of the burst stream transmission.

[0075] After the burst flow is successfully received by the switch at the first hop, the fast-forwarding policy should be followed at the subsequent multi-hop nodes. The burst flow has the highest priority, so the queue with a queue offset value of 1 is always selected for the burst flow to receive, that is:

[0076] (6)

[0077] The purpose is to ensure that the burst flow can be forwarded in the next time slot of the current node.

[0078] While the time-triggered flow selects the queue to receive, so as to meet the requirements of time-triggered flow delay, jitter, etc. without affecting the transmission of the burst flow.

[0079] Based on this, before scheduling the hybrid flow, it is necessary to explore the priority of the flow. When the flow arrives at a certain node, the optimal transmission time slot should also be selected for the flow. The embodiment of the present invention proposes to use the deep reinforcement learning method to explore the problem of the transmission time slot of the flow, and the basis of the probability distribution of the flow is the priority of the flow. The priority calculation formula integrates indicators such as the period, transmission time, deadline, and routing hop count of the traffic in the receiving queue to give different priority metrics for different traffic, so as to optimize resource allocation. Specifically, the priority calculation is as follows:

[0080] (7)

[0081] Among them, represents the priority of the traffic flow , represents the flow sequence number, and the traffic flow includes burst flow and time-triggered flow; represents the weight coefficient; represents the period of the traffic flow ; represents the deadline of the traffic flow ; represents the priority characteristic of the traffic flow , and this item is mainly used to distinguish between burst flow and time-triggered flow. The specific values can be seen in Table 1; and respectively represent the maximum routing hop count and the minimum routing hop count that the traffic flow passes from the source node to the destination node. Therefore this item reflects the influence of the routing hop count on the priority level.

[0082] Table 1 Priority characteristic values of traffic flow

[0083]

[0084] Since the bandwidth resources in the network are limited, it is necessary to arrange the transmission time slot allocation and routing selection in combination with the service flow to be scheduled to ensure reliable traffic scheduling and strive for optimal network resource allocation. Therefore, the embodiments of the present invention are based on a multi-queue cyclic queuing and forwarding mechanism, and use a deep reinforcement learning algorithm to solve the joint routing hybrid flow scheduling problem.

[0085] It should be noted that when solving the joint routing hybrid flow scheduling problem of the embodiments of the present invention, the following constraints need to be considered:

[0086] A. To achieve the periodic scheduling of time-triggered flows, the scheduling period is set to the least common multiple (LCM) of the periods of all time-triggered flows, that is:

[0087] (8)

[0088] Wherein, represents the least common multiple, represents the set of time-triggered flows, represents the period of the time-triggered flow, represents taking the least common multiple of the periods of all time-triggered flows.

[0089] B. In the multi-queue cyclic queuing and forwarding mechanism, the end-to-end transmission delay of the service flow should satisfy the following constraints:

[0090] (9)

[0091] Wherein, represents the service flow from the source node to the end-to-end delay of the destination node, represents the service flow at the selected receive queue offset value on the node, represents the length of the transmission time slot, represents the service flow total number of nodes passed on the transmission path.

[0092] C. The multi-queue cyclic queuing and forwarding mechanism takes each transmission time slot as the smallest unit of resource allocation. Each time-triggered flow should be divisible by the transmission time slot, and the length of the transmission time slot should be at most equal to the greatest common divisor of the periods of all time-triggered flows. Therefore:

[0093] (10)

[0094] (11)

[0095] Wherein, represents the greatest common divisor represents the remainder operation

[0096] D. The length of the receive time slot should be able to accommodate the mixed flow, that is, both the bursty flow and the time-triggered flow can be received. Considering the worst case, when the bursty flow is sent by the upstream node exactly at the end of the transmission time slot, to receive all traffic flows, the receive time slot shall not be less than the sum of a transmission time slot and the interface transmission delay. Therefore, the receive time slot can be described as:

[0097] (12)

[0098] where represents the largest data packet represents the traffic flow size represents the traffic flow selected receive queue offset value represents the link bandwidth

[0099] E. The transmission time of the time-triggered flow shall not exceed the period size of the time-triggered flow to avoid conflicts of time-triggered flows with different periods at the same node. Therefore, it is necessary to satisfy:

[0100] (13)

[0101] where represents the transmission time of the time-triggered flow

[0102] F. The worst-case end-to-end transmission delay of the traffic flow shall not exceed its deterministic delay requirement

[0103] (14)

[0104] where represents the traffic flow deadline

[0105] G. Let be the end-to-end delay of the th data packet of the traffic flow. The jitter of transmitting each data packet of the traffic flow shall not exceed the jitter upper limit requirement of the traffic flow. Therefore, the jitter constraint is described as:

[0106] (15)

[0107] where represents the number of data packets contained in the traffic flow represents the traffic flow th​ The end-to-end delay of a data packet.

[0108] Based on the above constraints, for the joint routing and hybrid flow scheduling problem of the multi-queue cyclic queuing and forwarding mechanism in the embodiments of the present invention, the solution process is as follows:

[0109] Model the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem, that is:

[0110] (16)

[0111]

[0112] ,

[0113] where, represents the number of transmission time slots within a scheduling period, represents the total number of traffic flows, represents the traffic flow set, is the value function of the th transmission time slot, expressed as:

[0114]

[0115] where, is the weight coefficient, , indicates whether the traffic flow is successfully scheduled, represents the network load balancing degree function, represents the end-to-end transmission delay of the bursty flow from the source end to the destination end .

[0116] The traffic flow set is expressed as: , and each traffic flow has six attributes, namely , where is the source node of the traffic flow , is the destination node of the traffic flow , is the size of the traffic flow , is the period of the traffic flow , is the deadline of the traffic flow , is the routing path of the traffic flow .

[0117] If all the data packets included in the traffic flow are successfully scheduled, then The value is 1, otherwise it is 0. Therefore, it can be expressed as:

[0118] (18)

[0119] Among them, , if it means that the th data packet of the service flow is assigned to the

[0120] th transmission time slot. It is represented by the mean square deviation of the time slot load value, that is:

[0121] (19)

[0122] Among them, represents the load ratio of the th transmission time slot, that is,

[0123] (20)

[0124] (21)

[0125] Among them, represents the average value of the load ratios of all transmission time slots within the scheduling period, represents the th data packet of the service flow represents the size of the service flow , represents the number of data packets contained in the service flow , represents whether the th data packet of the service flow is assigned to the th transmission time slot,

[0126] It can be seen from the optimization objective function in Equation (16) that when the number of successfully scheduled flows is higher, the time slot load is more balanced, and the burst flow delay is lower, a higher value benefit can be obtained.

[0127] For the above multi-objective constrained optimization problem, the embodiment of the present invention uses a deep reinforcement learning algorithm to solve it. The deep reinforcement learning network framework is as Figure 3 shown. In the embodiment of the present invention, the state space , the action space and the reward function are as follows:

[0128] State space : The state space mainly refers to the set of states in which the network environment is located. In this embodiment, the state information is mainly composed of two parts: service flow characteristic information and the load ratio of transmission time slots, which is expressed as:

[0129] (22)

[0130] represents the state parameters of the environment.

[0131] Action space : In the transmission time slot, according to the current state of each switch, calculate the receiving queue selected by the next hop of the service flow in the transmission queue of each switch in the current transmission time slot, which is expressed as:

[0132] (23)

[0133] Among them, represents the action, that is, the receiving queue selected by the next hop of the service flow in the transmission queue of each switch in the current transmission time slot, represents the switch the action taken, , is the number of switches.

[0134] The reward function is as follows:

[0135] (24)

[0136] represents the state 、action when the reward function.

[0137] The deep reinforcement learning algorithm is used to solve the above multi-objective constrained optimization problem, as Figure 3 shown, the specific steps are as follows:

[0138] Step 1: Initialize the state parameters of the environment , which includes network status and service flow characteristic information, and input the state parameters of the environment into the policy network. The policy network is responsible for directly outputting the distribution of actions, so as to determine the action strategy adopted in each state.

[0139] Step 2: The policy network directly selects actions according to the environmental information. According to the constraints in the above multi-objective constrained optimization problem, randomly select a queue that meets the constraints and inject a transmission time slot. In this process, it is necessary to calculate the priority of each service flow to be scheduled, and use the softmax function to convert the priority into the probability distribution of the service flow to be scheduled. The probability of each service flow to be scheduled is:

[0140] (25)

[0141] The policy network selects the traffic flow to be scheduled with the highest probability according to the probability distribution of the traffic flow to be scheduled for scheduling. According to the shortest path algorithm, that is, based on the source node and destination node of the traffic flow to be scheduled, a path with the fewest number of routed hops passed through is planned as the routing path of the traffic flow to be scheduled.

[0142] Step 3: Execute the action. After completing the scheduling of the traffic flow to be scheduled, update the network state, store the experience of each interaction in the experience buffer, and obtain the reward and the next state .

[0143] Step 4: The evaluation network estimates the value to guide the improvement of the policy network. The evaluation network calculates the temporal difference error by estimating the value function , that is:

[0144] (26)

[0145] where is the discount factor, is the value function at state , is the value function at state , and is expressed using the Bellman equation, that is:

[0146] (27)

[0147] where represents the expectation operation, represents the reward at time , represents the value function at state .

[0148] Step 5: Update the evaluation network:

[0149] (28)

[0150] where is the weight of the updated evaluation network, is the weight of the evaluation network before update, is the learning rate of the evaluation network, represents the gradient.

[0151] Step 6: Update the policy network:

[0152] (29)

[0153] Among them, and are the policy network parameters before and after update respectively, is the learning rate of the policy network, represents at state selecting action with probability, represents the gradient.

[0154] Step 7: Repeat Steps 2 to 6 until the maximum number of iterations is reached and convergence occurs, and finally output the scheduling policy of the service flow to be scheduled.

[0155] Based on the same inventive concept, the second embodiment of the present invention provides a joint routing and scheduling device based on multi-queue cyclic queuing and forwarding, for implementing the joint routing and scheduling method based on multi-queue cyclic queuing and forwarding in the above embodiment. The device includes:

[0156] A configuration module, used to design a multi-queue cyclic queuing and forwarding mechanism for each switch in the time-sensitive network; the multi-queue cyclic queuing and forwarding mechanism means that transmission queues are set on the switch for sending and receiving service flows. In each transmission time slot, there is one and only one transmission queue for sending service flows, and the remaining transmission queues are used for receiving service flows; the service flows include time-triggered flows and bursty flows;

[0157] A problem modeling module, used to model the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem with the goal of maximizing the number of scheduled time-triggered flows while ensuring the network requirements for bursty flow transmission;

[0158] An optimized scheduling module, used to determine the routing path of the service flows in the time-sensitive network. After each switch in each hop successfully receives, the multi-objective constrained optimization problem is solved to determine the receiving queue and transmission time slot of the next-hop switch of the service flow.

[0159] It should be noted that the device embodiment corresponds to the above method embodiment, and the implementation manners of the above method embodiment are all applicable to the device embodiment and can achieve the same or similar technical effects, so they will not be elaborated here.

[0160] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0161] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0162] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are performed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: still can modify the specific implementation manners of the present invention or make equivalent substitutions, and any modification or equivalent substitution that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A joint routing and scheduling method based on multi-queue cyclic queuing and forwarding, characterized in that, Including: Design a multi-queue cyclic queuing and forwarding mechanism for each switch in the time-sensitive network; The multi-queue cyclic queuing and forwarding mechanism means that M transmission queues are set on the switch to send and receive traffic flows. In each transmission time slot, there is exactly one transmission queue to send the traffic flow, and the remaining transmission queues are used to receive the traffic flow; the traffic flow includes time-triggered flows and bursty flows; On the basis of ensuring the network requirements for bursty flow transmission, with the goal of maximizing the number of time-triggered flow scheduling, the joint routing and scheduling problem of time-triggered flows and bursty flows is modeled as a multi-objective constrained optimization problem, expressed as: And the following constraints need to be satisfied: 0 ≤ g i .offset ≤ g i .prd g i ∈ F tt , F tt ∈ F mix , Among them, U represents the total number of service flows, F mix represents the service flow set, f i represents service flow i, V β (f i , λ, F mix ) is the value function of the β-th transmission time slot, λ represents the number of transmission time slots within a scheduling period, μ1, μ2, μ3 are weight coefficients, Suc(f i , λ) represents whether service flow is successfully scheduled, Bal(F mix , λ) represents the network load balancing degree function, represents the end-to-end transmission delay of the burst flow from the source end v s to the destination end v d , f i .delay represents the end-to-end delay of service flow i from the source node to the destination node, represents the receiving queue offset value selected by service flow i on the n-th node, T trans represents the length of the transmission time slot, N represents the total number of nodes passed by the transmission path of service flow i, represents the receiving queue offset value selected by service flow i, Β represents the link bandwidth, f i .size represents the size of service flow i, max(f i .size) represents the largest data packet, g i .offset represents the sending moment of the time-triggered flow, g i .prd represents the period of the time-triggered flow, f i .deadline represents the deadline of service flow i, f i k .delay is the end-to-end delay of the k-th data packet of service flow i, represents the number of data packets contained in service flow i, f i s .delay represents the end-to-end delay of the s-th data packet of service flow i; Determine the routing path of the traffic flow in the time-sensitive network. After each hop switch successfully receives, solve the multi-objective constrained optimization problem to determine the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

2. The joint routing and scheduling method based on multi-queue circular queuing and forwarding according to claim 1, characterized in that The Suc(f i , λ) is expressed as: where, Φ β (f i s ) ∈ {0, 1} indicates whether the s-th data packet of traffic flow i is assigned to the β-th transmission time slot. If Φ β (f i s ) = 1, it means that the s-th data packet of traffic flow i is assigned to the β-th transmission time slot.

3. A joint routing and scheduling method based on multi-queue cyclic queuing and forwarding according to claim 1, characterized in that The network load balancing degree function represents: Among them, X β represents the load ratio of the β-th transmission time slot, represents the average value of the load ratios of all transmission time slots within the scheduling period, represents the y-th data packet of traffic flow m, f m .size represents the size of traffic flow m, represents the number of data packets contained in traffic flow m, represents whether the y-th data packet of traffic flow m is assigned to the β-th transmission time slot, represents the capacity size of the β-th transmission time slot.

4. The joint routing and scheduling method based on multi-queue cyclic queuing and forwarding according to claim 1, characterized in that The offset value of the receiving queue selected by the traffic flow i needs to satisfy the following constraints: If the receiving queue j is in front of the sending queue, then, Offset j = Offset base + (j - ε%M + M), If the receiving queue j is behind the sending queue, then, Offset j = Offset base + (j - ε%M), Among them, offset j represents the offset value of the receive queue j, Offset base represents the offset value of the transmit queue, and ε%M represents the transmit queue; When the bursty flow is received by the first-hop switch, select a receiving queue with a receiving queue offset value greater than or equal to 2. After the bursty flow is successfully received by the first-hop switch, always select a receiving queue with a receiving queue offset value of 1.

5. The combined routing and scheduling method based on multi-queue cyclic queuing and forwarding according to claim 1, characterized in that, The scheduling period is set as the least common multiple of the periods of all time-triggered flows.

6. The joint routing and scheduling method based on multi-queue cyclic queuing and forwarding according to claim 3, characterized in that Solving the multi-objective constrained optimization problem includes: Set the state parameter s of the environment l as: s l = {f i , X β}, 1 ≤ i ≤ U, 0 ≤ β ≤ λ - 1; The f i includes six attributes of service flow i, expressed as f i = {f i .src, f i .dst, f i .size, f i .prd, f i .deadline, f i .path}, where f i .src is the source node of service flow i, f i .dst is the destination node of service flow i, f i .size is the size of service flow i, f i .prd is the period of service flow i, f i .deadline is the deadline of service flow i, f i .path is the routing path of service flow i; Set the action parameter a t to be: a t = {SW1, SW2, …, SWR N}; where SW j represents the action taken by switch j, representing the receiving queue selected for the next hop of the traffic flow in the transmission queue of switch j in the current transmission time slot, j = 1, 2, …, RN, and RN is the number of switches in the time-sensitive network; Set the reward function as: Among them, r(s l , a t ) represents the reward function when the state is s l and the action is a t ; Based on the set state parameters, action parameters and reward function, use the deep reinforcement learning algorithm to solve, and obtain the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

7. The joint routing and scheduling method based on multi-queue cyclic queuing and forwarding according to claim 6, wherein Solving the multi-objective constrained optimization problem, calculate the priority of each traffic flow in the following way: Among them, H(i) represents the priority of traffic flow i, w1, w2, w3, w4 represent weight coefficients, and pri i represents the priority characteristic of traffic flow i, and respectively represent the maximum number of routing hops and the minimum number of routing hops that traffic flow i passes through from the source node to the destination node; Convert the priority into the probability of the traffic flow: where p i is the probability of service flow i; Select the traffic flow with the largest probability for scheduling.

8. A joint routing and scheduling method based on multi-queue cyclic queuing and forwarding according to claim 6, characterized in that, Determining the routing path of the traffic flow in the time-sensitive network includes: According to the source node and destination node of the traffic flow, based on the shortest path algorithm, plan a path with the fewest number of routed paths passed as the routing path of the traffic flow.

9. A combined routing and scheduling device based on multi-queue cyclic queuing and forwarding, characterized in that A device for implementing the joint routing and scheduling method based on multi-queue cyclic queuing and forwarding according to any one of claims 1 to 8, the device includes: A configuration module for designing a multi-queue cyclic queuing and forwarding mechanism for each switch in the time-sensitive network; the multi-queue cyclic queuing and forwarding mechanism means that M transmission queues are set on the switch to send and receive traffic flows. In each transmission time slot, there is exactly one transmission queue to send the traffic flow, and the remaining transmission queues are used to receive the traffic flow; the traffic flow includes time-triggered flows and bursty flows; A problem modeling module for modeling the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem on the basis of ensuring the network requirements for bursty flow transmission, with the goal of maximizing the number of time-triggered flow scheduling; An optimization scheduling module is used to determine the routing path of traffic flows in a time-sensitive network. After each hop switch successfully receives the traffic, the multi-objective constraint optimization problem is solved to determine the receive queue and transmission time slot of the next-hop switch of the traffic flow.

Images