Joint routing scheduling method and device based on multi-queue circular queuing forwarding

Abstract

The invention discloses a joint routing scheduling method and device based on multi-queue circular queuing forwarding, and belongs to the technical field of network flow scheduling. The method comprises the following steps: designing a multi-queue circular queuing forwarding mechanism for each switch in the time-sensitive network; on the premise of guaranteeing the network requirement of burst flow transmission, with the maximum time-triggered flow scheduling number as the target, a time-triggered flow and burst flow joint routing scheduling problem is modeled as a multi-target constraint optimization problem, and deep reinforcement learning is adopted for solving, so that the time-triggered flow and burst flow joint routing scheduling problem is solved. And obtaining a receiving queue and a transmission time slot of a next hop of a service stream in a transmission queue of each switch in the time-sensitive network. According to the multi-queue circular queuing forwarding mechanism provided by the invention, the problem of mixed flow scheduling of the burst flow and the time-triggered flow is well solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of network traffic scheduling, and in particular, to a joint routing and scheduling method and device based on multi-queue cyclic queuing and forwarding. Background Art

[0002] With the rapid development of industrial Internet of Things, intelligent manufacturing, and big data technologies, the demand for real-time and deterministic transmission of information has become increasingly prominent. Time Sensitive Networking (TSN) has emerged as a new industrial communication technology.

[0003] Based on traditional Ethernet, Time Sensitive Network supports the mixed transmission of periodic and aperiodic flows in the network through technologies such as clock synchronization mechanism, traffic shaping mechanism, gated array mechanism, and flow reservation, and can provide end-to-end deterministic, low-latency, and low-jitter transmission guarantees for data. Among them, traffic scheduling technology is a key technology in the current research of Time Sensitive Network. Existing research shows that the TSN traffic scheduling problem is a non-deterministic polynomial problem. Since there are often a large number of bursty flows with high latency requirements in actual industrial scenarios, and such bursty flows are highly dynamic and unpredictable, the mixed traffic scheduling mechanism needs to be focused on. At present, there is a lack of research on the mixed traffic scheduling method that includes bursty flows and time-triggered flows, and the impact of network topology has not been considered. Most studies only consider the case of a single Time Sensitive Network switch. Summary of the Invention

[0004] The purpose of the present invention is to provide a joint routing and scheduling method and device based on multi-queue cyclic queuing and forwarding, which realizes the joint routing and scheduling problem of mixed flows based on the multi-queue cyclic queuing and forwarding mechanism, and maximally guarantees the number of time-triggered flow schedules while ensuring the network requirements for bursty flow transmission.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] In the first aspect, the present invention provides a joint routing and scheduling method based on multi-queue cyclic queuing and forwarding, including:

[0007] Design a multi-queue cyclic queuing and forwarding mechanism for each switch in the time sensitive network; the multi-queue cyclic queuing and forwarding mechanism means that transmission queues are set on the switch to send and receive traffic flows. In each transmission time slot, there is one and only one transmission queue to send traffic flows, and the remaining transmission queues are used to receive traffic flows; the traffic flows include time-triggered flows and bursty flows.

[0008] To meet the network requirements for the transmission of bursty flows, with the goal of maximizing the number of time-triggered flow schedules, the joint routing and scheduling problem of time-triggered flows and bursty flows is modeled as a multi-objective constrained optimization problem;

[0009] Determine the routing path of the traffic flows in the time-sensitive network. After each hop switch successfully receives the traffic, solve the multi-objective constrained optimization problem to determine the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

[0010] Preferably, the multi-objective constrained optimization problem is expressed as:

[0011] ,

[0012] ,

[0013] And it needs to satisfy the following constraints:

[0014] ,

[0015] Among them, represents the total number of traffic flows, represents the set of traffic flows, represents the traffic flow , is the value function of the th transmission time slot, represents the number of transmission time slots within a scheduling period, is the weight coefficient, indicates whether the traffic flow is successfully scheduled, represents the network load balancing degree function, then represents the end-to-end transmission delay of the bursty flow from the source end to the destination end , represents the end-to-end delay of the traffic flow from the source node to the destination node, represents the traffic flow at the th node, the selected receiving queue offset value, represents the length of the transmission time slot, represents the traffic flow the total number of nodes passed through on the transmission path, represents the traffic flow selected receiving queue offset value, represents the link bandwidth, represents the traffic flow size, represents the largest data packet, represents the sending time of the time-triggered flow, Represents the period of the time-triggered flow, Represents the service flow The deadline of, For the service flow The End-to-end delay of the nth data packet of, Represents the service flow The number of data packets contained in, Represents the service flow The End-to-end delay of the nth data packet of.

[0016] Preferably, the Is expressed as:

[0017] ,

[0018] Wherein, Represents the service flow The Whether the nth data packet of has been assigned to the mth transmission time slot. If Then it means that the The The nth data packet of has been assigned to the mth transmission time slot.

[0019] Preferably, the network load balancing degree function is expressed as:

[0020] ,

[0021] ,

[0022] ,

[0023] Wherein, Represents the load ratio of the mth transmission time slot, Represents the average value of the load ratios of all transmission time slots within the scheduling period, Represents the service flow The nth data packet of, Represents the size of the service flow , Represents the service flow The number of data packets contained in, Represents the service flow The Whether the nth data packet of has been assigned to the mth transmission time slot, Represents the Capacity size of the mth transmission time slot.

[0024] Preferably, the service flow The selected receive queue offset value needs to satisfy the following constraints:

[0025] If the receive queue is before the send queue, then,

[0026] ,

[0027] If the receive queue is after the send queue, then,

[0028] ,

[0029] wherein, represents the offset value of the receive queue ; represents the offset value of the send queue, represents the send queue;

[0030] When the burst flow is received by the switch of the first hop, select a receive queue with a receive queue offset value greater than or equal to 2. After the burst flow is successfully received by the switch of the first hop, always select a receive queue with a receive queue offset value of 1.

[0031] Preferably, the scheduling period is set to the least common multiple of the periods of all time-triggered flows.

[0032] Preferably, solving the multi-objective constraint optimization problem includes:

[0033] Set the state parameters of the environment as: ; The includes six attributes of the service flow represented as , where is the source node of the service flow , is the destination node of the service flow , is the size of the service flow , is the period of the service flow , is the deadline of the service flow , is the routing path of the service flow ;

[0034] Set the action parameters as: ; wherein, represents the action taken by the switch and represents the switch at the current transmission time slot The receiving queue for the next-hop selection of the traffic flow in the transmission queue , is the number of switches in the time-sensitive network;

[0035] Set the reward function as: ;

[0036] wherein, represents the state , action when the reward function;

[0037] Based on the set state parameters, action parameters and reward function, use the deep reinforcement learning algorithm to solve, and obtain the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

[0038] Preferably, in the solving process, calculate the priority of each traffic flow in the following manner:

[0039] ,

[0040] wherein, represents the traffic flow priority of, represents the weight coefficient, represents the traffic flow period of, represents the traffic flow deadline of, represents the traffic flow priority characteristic of, and respectively represent the maximum number of routing hops and the minimum number of routing hops that the traffic flow passes through from the source node to the destination node;

[0041] Convert the priority to the probability of the traffic flow:

[0042] ,

[0043] wherein, is the probability of the traffic flow ;

[0044] Select the traffic flow with the largest selection probability for scheduling.

[0045] Preferably, the determining the routing path of the traffic flow in the time-sensitive network includes:

[0046] According to the source node and destination node of the traffic flow, based on the shortest path algorithm, plan a path with the fewest number of routing hops passed through as the routing path of the traffic flow.

[0047] In a second aspect, the present invention provides a joint routing and scheduling device based on multi-queue cyclic queuing and forwarding, which is used to implement the above-mentioned joint routing and scheduling method based on multi-queue cyclic queuing and forwarding. The device includes:

[0048] A configuration module, which is used to design a multi-queue cyclic queuing and forwarding mechanism for each switch in the time-sensitive network; the multi-queue cyclic queuing and forwarding mechanism means that a number of transmission queues are set on the switch to send and receive traffic flows. In each transmission time slot, there is exactly one transmission queue to send traffic flows, and the remaining transmission queues are used to receive traffic flows; the traffic flows include time-triggered flows and bursty flows.

[0049] A problem modeling module, which is used to model the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem with the goal of maximizing the scheduling quantity of time-triggered flows on the premise of ensuring the network requirements for bursty flow transmission.

[0050] An optimized scheduling module, which is used to determine the routing path of traffic flows in the time-sensitive network. After each switch in each hop successfully receives, it solves the multi-objective constrained optimization problem to determine the receiving queue and transmission time slot of the next-hop switch of the traffic flow.

[0051] The beneficial effects achieved by the present invention are as follows:

[0052] Aiming at the joint routing and scheduling problem of hybrid flows in the time-sensitive network, the present invention proposes a joint routing and scheduling method based on multi-queue cyclic queuing and forwarding, which uses a deep reinforcement learning network to select the optimal injection transmission time slot and receiving queue, and combines the shortest path planning algorithm to perform routing planning to complete traffic flow scheduling. The multi-queue cyclic queuing and forwarding mechanism proposed by the present invention can well handle the scheduling problem of hybrid flows of bursty flows and time-triggered flows. Among them, a highly efficient policy sorting network is trained from a large number of hybrid flow scheduling processes by using deep reinforcement learning, which significantly improves the final hybrid flow scheduling result. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the multi-queue cyclic queuing and forwarding mechanism proposed in the embodiment of the present invention;

[0054] Figure 2 It is a topology diagram of the time-sensitive network in the embodiment of the present invention;

[0055] Figure 3 It is a schematic diagram of the hybrid flow joint routing and scheduling process based on deep reinforcement learning in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] To make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the embodiments and the drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention.

[0057] Herein, it should also be noted that in order to avoid obscuring the present invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the present invention are shown in the drawings, while other details less related to the present invention are omitted.

[0058] It should be emphasized that the term "comprising / including" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0059] It should be emphasized here that the step marks mentioned hereinafter are not intended to limit the order of the steps. Instead, it should be understood that the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0060] The traditional Cyclic Queuing and Forwarding (CQF) mechanism maintains two queue cycles on each interface of each network device to receive and send traffic flows. Based on considering the hybrid flow scheduling, the embodiments of the present invention improve the traditional cyclic queuing and forwarding mechanism and design a multi-queue cyclic queuing and forwarding mechanism as Figure 1 shown. In order to support the hybrid traffic transmission of bursty flows and time-triggered flows, transmission queues are set on each time-sensitive network switch. The gating on these transmission queues periodically sends and receives traffic flows. In each transmission time slot, there is and only one transmission queue that can send traffic flows, and the remaining transmission queues can only receive traffic flows.

[0061] The time-sensitive network topology is as Figure 2 shown. The time-sensitive network topology is modeled as an undirected graph , where is the set of all time-sensitive network switches and terminals in the network, Figure 2 in which, SW 1 , SW 2 , SW 3 , SW 4 , SW 5 , SW 6 are switches, is the set of links in the network.

[0062] Specifically, the queue state is divided into two states. The queue state for sending traffic flows is denoted as , where represents a queue ; the state of the received traffic flow is recorded as , , , representing a queue . During the transmission time slot , the state of the queue is recorded as:

[0063] (1)

[0064] (2)

[0065] where represents the transmission time slot number, represents the only queue for sending traffic flow in the transmission time slot , while represents the remaining queues for receiving traffic flow.

[0066] When the switch port receives a traffic flow, in order to determine a queue for the traffic flow, a receiving queue offset value is defined as the value from the sending queue corresponding to the current transmission time slot to the receiving queue . The offset value of the sending queue is recorded as , and based on the queue offset value definition, it can be expressed as:

[0067] If the receiving queue is in front of the sending queue, that is, , then,

[0068] (3)

[0069] If the receiving queue is behind the sending queue, that is, , then,

[0070] (4)

[0071] In addition, when receiving a bursty flow, a queue with a queue offset value greater than or equal to 2 should be selected to receive the bursty flow, that is:

[0072] (5)

[0073] where represents the distance from the receiving queue to the sending queue. Setting the offset value greater than 2 is to prevent the phenomenon of packet loss due to the inability to fully receive the bursty flow within the current time slot, ensuring the reliability of bursty flow transmission.

[0074] After the burst flow is successfully received by the switch at the first hop, the fast-forwarding policy should be followed at the multi-hop nodes thereafter. The burst flow has the highest priority, so the queue with a queue offset value of 1 is always selected for the burst flow to receive, that is:

[0075] (6)

[0076] The purpose is to ensure that the burst flow can be forwarded in the next time slot of the current node.

[0077] While the time-triggered flow selects the queue to receive, so as to meet the requirements of time-triggered flow delay, jitter, etc. without affecting the transmission of the burst flow.

[0078] Based on this, before scheduling the mixed flow, it is necessary to explore the priority of the flow. When the flow arrives at a certain node, the optimal transmission time slot should also be selected for the flow. The embodiment of the present invention proposes to use the deep reinforcement learning method to explore the problem of the transmission time slot of the flow, and the probability distribution of the flow is based on the priority of the flow. The priority calculation formula integrates indicators such as the period, transmission time, deadline, and routing hop count of the traffic in the receiving queue to give different priority measures to different traffic, so as to optimize resource allocation. Specifically, the priority calculation is as follows:

[0079] (7)

[0080] Wherein, represents the priority of the traffic flow , represents the flow sequence number, and the traffic flow includes burst flow and time-triggered flow; represents the weight coefficient; represents the period of the traffic flow ; represents the deadline of the traffic flow ; represents the priority characteristic of the traffic flow , and this item is mainly used to distinguish between burst flow and time-triggered flow, and the specific values can be seen in Table 1; and respectively represent the maximum routing hop count and the minimum routing hop count that the traffic flow passes from the source node to the destination node. Therefore this item reflects the influence of the routing hop count on the priority level.

[0081] Table 1 Priority characteristic values of traffic flow

[0082]

[0083] Due to limited bandwidth resources in the network, it is necessary to arrange transmission time slot allocation and routing selection in combination with the service flow to be scheduled to ensure reliable traffic scheduling and strive for optimal network resource allocation. Therefore, the embodiments of the present invention are based on a multi-queue cyclic queuing and forwarding mechanism and use a deep reinforcement learning algorithm to solve the joint routing hybrid flow scheduling problem.

[0084] It should be noted that when solving the joint routing hybrid flow scheduling problem of the embodiments of the present invention, the following constraints need to be considered:

[0085] A. To achieve the periodic scheduling of time-triggered flows, the scheduling period is set to the least common multiple (LCM) of the periods of all time-triggered flows, that is:

[0086] (8)

[0087] where, represents the least common multiple, represents the set of time-triggered flows, represents the period of the time-triggered flow, represents taking the least common multiple of the periods of all time-triggered flows.

[0088] B. In the multi-queue cyclic queuing and forwarding mechanism, the end-to-end transmission delay of the service flow should satisfy the following constraints:

[0089] (9)

[0090] where, represents the service flow from the source node to the end-to-end delay of the destination node, represents the service flow at the th node, the selected receive queue offset value, represents the length of the transmission time slot, represents the service flow the total number of nodes passed on the transmission path.

[0091] C. The multi-queue cyclic queuing and forwarding mechanism takes each transmission time slot as the smallest unit of resource allocation. Each time-triggered flow should be divisible by the transmission time slot, and the length of the transmission time slot should be at most equal to the greatest common divisor of the periods of all time-triggered flows. Therefore:

[0092] (10)

[0093] (11)

[0094] where, represents the greatest common divisor, represents the remainder operation.

[0095] D. The length of the receiving time slot should be able to accommodate the hybrid flow, that is, both the bursty flow and the time-triggered flow can be received. Considering the worst case, when the bursty flow is sent by the upstream node exactly at the end of the transmission time slot, in order to receive all traffic flows, the receiving time slot shall not be less than the sum of a transmission time slot and the interface transmission delay. Therefore, the receiving time slot can be described as:

[0096] (12)

[0097] where, represents the largest data packet, represents the traffic flow size, represents the traffic flow selected receiving queue offset value, represents the link bandwidth.

[0098] E. The sending moment of the time-triggered flow shall not exceed the period size of the time-triggered flow to avoid conflicts of time-triggered flows with different periods at the same node. Therefore, it is necessary to satisfy:

[0099] (13)

[0100] where, represents the sending moment of the time-triggered flow.

[0101] F. The worst-case end-to-end transmission delay of the traffic flow shall not exceed its deterministic delay requirement,

[0102] (14)

[0103] where, represents the traffic flow deadline.

[0104] G. Let be the end-to-end delay of the th data packet of the traffic flow. The jitter of transmitting each data packet of the traffic flow shall not exceed the jitter upper limit requirement of the traffic flow. Therefore, the jitter constraint is described as:

[0105] (15)

[0106] where, represents the number of data packets contained in the traffic flow , represents the traffic flow th The end-to-end delay of a data packet.

[0107] Based on the above constraints, for the joint routing and hybrid flow scheduling problem of the multi-queue cyclic queuing and forwarding mechanism in the embodiments of the present invention, the solution process is as follows:

[0108] Model the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem, that is:

[0109] (16)

[0110]

[0111] ,

[0112] where, represents the number of transmission time slots within a scheduling period, represents the total number of traffic flows, represents the set of traffic flows, is the value function of the th transmission time slot, expressed as:

[0113]

[0114] where, is the weight coefficient, , indicates whether the traffic flow is successfully scheduled, represents the network load balancing degree function, represents the end-to-end transmission delay of the bursty flow from the source end to the destination end .

[0115] The set of traffic flows is expressed as: , and each traffic flow has six attributes, namely , where is the source node of the traffic flow , is the destination node of the traffic flow , is the size of the traffic flow , is the period of the traffic flow , is the deadline of the traffic flow , is the routing path of the traffic flow .

[0116] If all the data packets included in the traffic flow are successfully scheduled, then The value is 1, otherwise it is 0. Therefore, it can be expressed as:

[0117] (18)

[0118] Among them, , if it means that the th data packet of the service flow is allocated to the th transmission time slot.

[0119] It is represented by the mean square deviation of the time slot load value, that is:

[0120] (19)

[0121] Among them, represents the load ratio of the th transmission time slot, that is,

[0122] (20)

[0123] (21)

[0124] Among them, represents the average value of the load ratios of all transmission time slots within the scheduling period, represents the th data packet of the service flow, represents the size of the service flow , represents the number of data packets contained in the service flow , represents whether the th data packet of the service flow is allocated to the th transmission time slot, represents the capacity size of the th transmission time slot.

[0125] It can be seen from the optimization objective function in Equation (16) that when the number of successfully scheduled flows is higher, the time slot load is more balanced, and the burst flow delay is lower, a higher value benefit can be obtained.

[0126] For the above multi-objective constrained optimization problem, the embodiment of the present invention uses a deep reinforcement learning algorithm to solve it. The deep reinforcement learning network framework is as Figure 3 shown. In the embodiment of the present invention, the state space , the action space and the reward function are as follows:

[0127] State space : The state space mainly refers to the set of states in which the network environment is located. In this embodiment, the state information is mainly composed of two parts: service flow characteristic information and the load ratio of transmission time slots, which is expressed as:

[0128] (22)

[0129] Indicates the state parameters of the environment.

[0130] Action space : In the transmission time slot, according to the current state of each switch, calculate the receiving queue selected by the next hop of the service flow in the transmission queue of each switch in the current transmission time slot, which is expressed as:

[0131] (23)

[0132] Among them, Indicates the action, that is, the receiving queue selected by the next hop of the service flow in the transmission queue of each switch in the current transmission time slot, Indicates the switch The action taken, , Is the number of switches.

[0133] The reward function is as follows:

[0134] (24)

[0135] Indicates the state , action The reward function at time.

[0136] Use the deep reinforcement learning algorithm to solve the above multi-objective constrained optimization problem, as Figure 3 Shown, the specific steps are as follows:

[0137] Step 1: Initialize the state parameters of the environment , which includes network status and service flow characteristic information, and input the state parameters of the environment into the policy network. The policy network is responsible for directly outputting the distribution of actions, so as to determine the action strategy adopted in each state.

[0138] Step 2: The policy network directly selects actions based on the environmental information. According to the constraints in the above multi-objective constrained optimization problem, randomly select a queue that meets the constraints and inject a transmission time slot. In this process, it is necessary to calculate the priority of each service flow to be scheduled , and use the softmax function to convert the priority into the probability distribution of the service flow to be scheduled. The probability of each service flow to be scheduled is:

[0139] (25)

[0140] The policy network selects the service flow to be scheduled with the highest probability according to the probability distribution of the service flow to be scheduled, and schedules the service flow to be scheduled with the highest probability. According to the shortest path algorithm, that is, based on the source node and destination node of the service flow to be scheduled, a path with the fewest number of routed hops is planned as the routing path of the service flow to be scheduled.

[0141] Step 3: Execute the action. After completing the scheduling of the service flow to be scheduled, update the network state, store the experience of each interaction in the experience buffer, and obtain the reward and the next state .

[0142] Step 4: The evaluation network estimates the value to guide the improvement of the policy network. The evaluation network calculates the temporal difference error by estimating the value function , that is:

[0143] (26)

[0144] where is the discount factor, is the value function at state , is the value function at state , and is expressed using the Bellman equation, that is:

[0145] (27)

[0146] where represents the expectation operation, represents the reward at time , represents the value function at state .

[0147] Step 5: Update the evaluation network:

[0148] (28)

[0149] where is the weight of the updated evaluation network, is the weight of the evaluation network before update, is the learning rate of the evaluation network, represents the gradient.

[0150] Step 6: Update the policy network:

[0151] (29)

[0152] Wherein, and are the policy network parameters before and after update respectively, is the learning rate of the policy network, represents the probability of selecting action under state , represents the gradient.

[0153] Step 7: Repeat Step 2 to Step 6 until the maximum number of iterations is reached and convergence occurs, and finally output the scheduling policy of the service flow to be scheduled.

[0154] Based on the same inventive concept, the second embodiment of the present invention provides a joint routing and scheduling device based on multi-queue cyclic queuing and forwarding for implementing the joint routing and scheduling method based on multi-queue cyclic queuing and forwarding in the above embodiment. The device includes:[[]]

[0155] A configuration module for designing a multi-queue cyclic queuing and forwarding mechanism for each switch in the time-sensitive network; the multi-queue cyclic queuing and forwarding mechanism means that transmission queues are set on the switch to send and receive service flows. In each transmission time slot, there is exactly one transmission queue to send service flows, and the remaining transmission queues are used to receive service flows; the service flows include time-triggered flows and bursty flows;

[0156] A problem modeling module for modeling the joint routing and scheduling problem of time-triggered flows and bursty flows as a multi-objective constrained optimization problem with the goal of maximizing the number of scheduled time-triggered flows while ensuring the network requirements for bursty flow transmission;

[0157] An optimized scheduling module for determining the routing path of the service flow in the time-sensitive network. After each switch in each hop successfully receives, the multi-objective constrained optimization problem is solved to determine the receiving queue and transmission time slot of the next-hop switch of the service flow.

[0158] It should be noted that the device embodiment corresponds to the above method embodiment, and the implementation manners of the above method embodiment are all applicable to the device embodiment and can achieve the same or similar technical effects, so they will not be elaborated here.

[0159] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0161] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: still can modify the specific implementation manners of the present invention or make equivalent replacements, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A joint routing scheduling method based on multi-queue circular queuing forwarding, characterized in that: include: Design a multi-queue round-robin queuing forwarding mechanism for each switch in a time-sensitive network; The multi-queue circular queuing forwarding mechanism refers to the switch setting transmission queues for sending and receiving service flows. In each transmission time slot, there is only one transmission queue for sending service flows, and the remaining transmission queues are used to receive service flows. The service flows include time-triggered flows and burst flows. In order to ensure the network requirements of burst flow transmission, the time-triggered flow and burst flow joint routing scheduling problem is modeled as a multi-objective constrained optimization problem with the goal of maximizing the number of time-triggered flow scheduling. The routing path of the service flow in the time-sensitive network is determined. After each hop switch successfully receives the service flow, the multi-objective constraint optimization problem is solved to determine the receiving queue and transmission time slot of the switch of the next hop of the service flow.

2. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 1, characterized in that: The multi-objective constrained optimization problem is expressed as: ， ， And the following constraints must be met: ， in, Indicates the total number of business flows. Represents a set of business flows. Indicates business flow , For the The cost function of a transmission slot is Indicates the number of transmission time slots in a scheduling cycle, is the weight coefficient, Indicates whether the business flow is successfully scheduled. represents the network load balancing function, Indicates that the burst flow is from the source To the destination The end-to-end transmission delay is Indicates business flow The end-to-end delay from the source node to the destination node, Indicates business flow In the The receive queue offset value selected on each node, Indicates the length of the transmission time slot, Indicates business flow The total number of nodes passed through on the transmission path, Indicates business flow The selected receive queue offset value, Indicates the link bandwidth, Indicates business flow The size of Indicates the largest data packet. Indicates the sending time of the time-triggered stream. represents the period of the time-triggered flow, Indicates business flow The deadline, For business flow No. The end-to-end delay of a packet, Indicates business flow The number of packets contained, Indicates business flow No. The end-to-end delay of a packet.

3. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 2, characterized in that: Said It is expressed as: ， in, Indicates business flow No. Is the packet assigned to In a transmission time slot, if It means business flow No. The data packets are assigned to transmission time slot.

4. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 2, characterized in that: The network load balancing function is expressed as: ， ， ， in, Indicates The load ratio of the transmission time slots, It represents the average value of the load ratio of all transmission time slots in the scheduling period. Indicates business flow No. Data packets, Indicates business flow The size of Indicates business flow The number of packets contained, Indicates business flow No. Is the packet assigned to In the transmission time slot, Indicates The capacity of a transmission time slot.

5. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 2, characterized in that: The business flow The selected receive queue offset value must meet the following constraints: If the receiving queue Before sending the queue, ， If the receiving queue After sending the queue, then, ， in, Indicates the receiving queue The offset value of Indicates the offset value of the send queue. Indicates the sending queue; If the burst flow is received by the first-hop switch, a receive queue with a receive queue offset value greater than or equal to 2 is selected. When the burst flow is successfully received by the first-hop switch, a receive queue with a receive queue offset value of 1 is always selected.

6. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 2, characterized in that: The scheduling period is set to the least common multiple of the periods of all time-triggered flows.

7. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 4, characterized in that: The solving of the multi-objective constrained optimization problem comprises: Set the state parameters of the environment for: ; Including business flow The six attributes of ,in For business flow The source node, For business flow Destination node, For business flow The size of For business flow The cycle, For business flow The deadline, For business flow The routing path; Set action parameters for: ;in, Indicates a switch The action taken represents the current transmission time slot switch The receiving queue selected by the next hop of the service flow in the transmission queue, , is the number of switches in the time-sensitive network; Set the reward function to: ; in, Indicates status ,action The reward function when ; Based on the set state parameters, action parameters and reward function, a deep reinforcement learning algorithm is used to solve the receiving queue and transmission time slot of the switch at the next hop of the business flow.

8. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 7, characterized in that: In the solution process, the priority of each service flow is calculated as follows: ， in, Indicates business flow The priority of represents the weight coefficient, Indicates business flow The priority characteristics of and Respectively represent business flows The maximum and minimum number of routing hops from the source node to the destination node; The probability of converting the priority into a traffic flow: ， in, For business flow probability; The service flow with the highest probability is selected for scheduling.

9. A joint routing scheduling method based on multi-queue circular queuing forwarding according to claim 7, characterized in that: Determining a routing path of a service flow in a time-sensitive network includes: According to the source node and the destination node of the service flow, based on the shortest path algorithm, a path with the least number of routes is planned as the routing path of the service flow.

10. A joint routing scheduling device based on multi-queue circular queuing forwarding, characterized in that: The device is used to implement the joint routing scheduling method based on multi-queue circular queuing forwarding according to any one of claims 1 to 9, comprising: The configuration module is used to design a multi-queue circular queuing forwarding mechanism for each switch in the time-sensitive network; the multi-queue circular queuing forwarding mechanism refers to setting transmission queues for sending and receiving service flows. In each transmission time slot, there is only one transmission queue for sending service flows, and the remaining transmission queues are used to receive service flows. The service flows include time-triggered flows and burst flows. The problem modeling module is used to model the joint routing scheduling problem of time-triggered flow and burst flow as a multi-objective constrained optimization problem based on the network requirements for guaranteeing burst flow transmission and taking the maximum number of time-triggered flow scheduling as the goal; The optimization scheduling module is used to determine the routing path of the business flow in the time-sensitive network. After each hop switch successfully receives the business flow, the multi-objective constraint optimization problem is solved to determine the receiving queue and transmission time slot of the switch of the next hop of the business flow.

Images