Time-sensitive network communication flow scheduling method based on deep reinforcement learning

Abstract

The invention discloses a time-sensitive network communication flow scheduling method based on deep reinforcement learning, and the method comprises the following steps: achieving the overall construction through the construction of a foundation, system modeling, a system framework, time slot selection, state modeling, motion modeling, environment modeling, and a deep neural network; detecting the whole body through error recovery, experimental demonstration and experimental objects, so the normal use of the whole body is ensured. According to the time-sensitive network communication flow scheduling method based on deep reinforcement learning, a DRL-based modeling, training and application method suitable for TS flow scheduling is provided, the method can be applied to different network environments by using different data training, and some optimization methods are provided; network information is expressed by using a directed graph, so that the DRLS can dynamically schedule the TS flow and can be quickly recovered when the network topology changes, and the uncertainty of the DNN is relieved and the scheduling capability and reliability of the scheduling method are improved by using a control gate technology.

Description

technical field [0001] The invention relates to the technical field of communication flow scheduling methods, in particular to a time-sensitive network communication flow scheduling method based on deep reinforcement learning. Background technique [0002] Time Sensitive Networking (TSN) is a key technology in the field of industrial public office applications. This field usually has real-time network communication requirements. One of the most challenging tasks in TSN networks is to design appropriate routing configurations to achieve time isolation for time-sensitive communications. And end-to-end real-time guarantee, this task is also called scheduling, time-sensitive communication requirements (TS flow) may change due to changes in application communication requirements or changes in network topology, in this case, scheduling The method must calculate the new schedule as soon as possible. There are two main scheduling methods in academia and industry: the solver-based me...

AI Technical Summary

Problems solved by technology

[0007] Step 2: System modeling, network topology is modeled as a directed graph, where V represents a collection of nodes, each node is a switch in the network, E is a collection of links, each link contains two node, if there is a physical link between two nodes vm and vn, then (v m , v n ), (v n , v m )∈E, the previous node in a node pair represents the source node of the link, and the latter node represents the target node of the link, all nodes can be used as the source node and target node of the TS flow, and can forward messages, And TS communication needs to use the concept of flow to model, a flow is a periodic unicast message, there is only one source node and one destination node, use S to represent the collection of all TS flows, a TS flow s k ∈S uses a quintuple (X k , Y k , C k ,T k , L k ), where Xk and Yk represent the source node and destination node of the TS flow, Ck, Tk, and Lk represent the packet size in bytes, the TS flow cycle in milliseconds and the maximum end-to-end delay , since different TS streams have different periods, the macro period is greater than or equal to the period of all streams, and the macro period is represented by Ts, which is obtained by calculating the least common multiple of all stream periods, and the routing of a TS stream from the source node to the destination node Expressed as a set of links {e 0 , e 1 ,...,e n }, where the source node of link e0 is Xk, the destination node of en is Yk, and ei and ei+1 are two adjacent links, if ei.src and ei.dst are used to represent the source node of an edge respectively And the target node, then the following constraints will be obtained: a TS stream s

Images