Time-sensitive network communication flow scheduling method based on deep reinforcement learning

Abstract

The invention discloses a time-sensitive network communication flow scheduling method based on deep reinforcement learning, and the method comprises the following steps: achieving the overall construction through the construction of a foundation, system modeling, a system framework, time slot selection, state modeling, motion modeling, environment modeling, and a deep neural network; detecting the whole body through error recovery, experimental demonstration and experimental objects, so the normal use of the whole body is ensured. According to the time-sensitive network communication flow scheduling method based on deep reinforcement learning, a DRL-based modeling, training and application method suitable for TS flow scheduling is provided, the method can be applied to different network environments by using different data training, and some optimization methods are provided; network information is expressed by using a directed graph, so that the DRLS can dynamically schedule the TS flow and can be quickly recovered when the network topology changes, and the uncertainty of the DNN is relieved and the scheduling capability and reliability of the scheduling method are improved by using a control gate technology.

Description

technical field [0001] The invention relates to the technical field of communication flow scheduling methods, in particular to a time-sensitive network communication flow scheduling method based on deep reinforcement learning. Background technique [0002] Time Sensitive Networking (TSN) is a key technology in the field of industrial public office applications. This field usually has real-time network communication requirements. One of the most challenging tasks in TSN networks is to design appropriate routing configurations to achieve time isolation for time-sensitive communications. And end-to-end real-time guarantee, this task is also called scheduling, time-sensitive communication requirements (TS flow) may change due to changes in application communication requirements or changes in network topology, in this case, scheduling The method must calculate the new schedule as soon as possible. There are two main scheduling methods in academia and industry: the solver-based me...

AI Technical Summary

Problems solved by technology

[0007] Step 2: System modeling, network topology is modeled as a directed graph, where V represents a collection of nodes, each node is a switch in the network, E is a collection of links, each link contains two node, if there is a physical link between two nodes vm and vn, then (v m , v n ), (v n , v m )∈E, the previous node in a node pair represents the source node of the link, and the latter node represents the target node of the link, all nodes can be used as the source node and target node of the TS flow, and can forward messages, And TS communication needs to use the concept of flow to model, a flow is a periodic unicast message, there is only one source node and one destination node, use S to represent the collection of all TS flows, a TS flow s k ∈S uses a quintuple (X k , Y k , C k ,T k , L k ), where Xk and Yk represent the source node and destination node of the TS flow, Ck, Tk, and Lk represent the packet size in bytes, the TS flow cycle in milliseconds and the maximum end-to-end delay , since different TS streams have different periods, the macro period is greater than or equal to the period of all streams, and the macro period is represented by Ts, which is obtained by calculating the least common multiple of all stream periods, and the routing of a TS stream from the source node to the destination node Expressed as a set of links {e 0 , e 1 ,...,e n }, where the source node of link e0 is Xk, the destination node of en is Yk, and ei and ei+1 are two adjacent links, if ei.src and ei.dst are used to represent the source node of an edge respectively And the target node, then the following constraints will be obtained: a TS stream s k ∈S flows through e i The first message instance (frame) of ∈E is expressed as in is the set of all frames that flow sk flows through ei, in order to schedule TS flow, need to specify Through the time of ei, in order to simplify the problem, one millisecond is divided into a time slot, and the time slot is represented by t, and each time slot contains the time of equal length. In one grand cycle, it contains a total of T s ×α time slots, select one of the time slots to pass through ei, and express the state of the jth time slot tj of the link ei as in is the state of all slots of ei, Is an integer number, indicating the number of frames passing through ei using time slot tj. A time slot can only be occupied by one frame at most, so there are the following constraints: Set the period of all TS streams to a multiple of 2: The system model has two basic assumptions. First, all network nodes (switches) have distributed time synchronization capabilities. Second, all devices have the ability to forward packets in real time. In a static scheduling environment, the scheduling method can Know all TS flow requirements, and these requirements will not change (increase or decrease). Among the dynamic scheduling problems that DRLS needs to solve, the TS flow requirements will change with time. DRLS generates schedules for these TS flows one by one. The new The scheduling plan of the flow cannot conflict with the scheduling plan of the existing flow. The complexity of the scheduling problem can be reduced by dividing the macro cycle into time slots, which greatly reduces the running time of DRLS and can generate new TS flow requirements in time. Scheduling;

Images