Traffic scheduling method of time sensitive network and intelligent device

Abstract

The application relates to the technical field of flow scheduling, and particularly provides a flow scheduling method for a time-sensitive network and an intelligent device, and aims to solve the problem that an existing flow scheduling method cannot simultaneously consider robustness and efficiency. To this end, the flow scheduling method comprises the following steps: constructing input state data by taking network state data, link state data, current scheduling configuration data, flow attributes and rhythm descriptors as input, inputting the input state data into a time-random state encoder to obtain a state tensor; taking the state tensor as input, obtaining a scheduling strategy by using a policy network, combining the rhythm descriptors to construct input state data, and predicting the scheduling strategy by using the time-random state encoder and the policy network, so that the robustness and efficiency of flow scheduling can be simultaneously considered.

Description

Technical Field

[0001] This application relates to the field of traffic scheduling technology, specifically providing a traffic scheduling method and intelligent device for time-sensitive networks. Background Technology

[0002] Time-Sensitive Networking (TSN) provides deterministic Ethernet communication for industrial control, automotive, and other scenarios. However, in actual brownfield factories, two common phenomena exist: First, periodic messages from traditional equipment such as PLCs (Programmable Logic Controllers) and robots exhibit short-term jitter and long-term clock drift; second, frequent additions and deletions to the topology due to production process reconfiguration, AGV (Automated Guided Vehicle) path optimization, and maintenance cause existing GCLs (Gate Control Lists) to quickly become invalid.

[0003] There are two main types of methods: precise optimization, such as ILP (Integer Linear Programming) or SMT (Satisfiability Modulo Theories), can characterize uncertainty, but their runtime is too long at medium scales; while heuristic / deep reinforcement learning can improve timeliness, it often ignores jitter and drift in pursuit of speed, resulting in a significant decrease in robustness in real-world applications such as workshops. Neither of these methods can balance the robustness and efficiency of flow scheduling. Summary of the Invention

[0004] This application aims to solve the aforementioned technical problem, namely, to address the issue that existing traffic scheduling methods cannot simultaneously achieve both robustness and efficiency.

[0005] In a first aspect, this application provides a traffic scheduling method for time-sensitive networks, comprising:

[0006] Obtain network status data, link status data, current scheduling configuration data, and flow set of the time-sensitive network to be scheduled. The flow set includes the flow attributes of each flow, and the flow attributes include at least the release jitter range and the maximum clock drift limit of the flow.

[0007] A rhythmic descriptor for the stream is constructed based on the release jitter range and the maximum clock drift boundary;

[0008] Input state data is constructed using the network state data, the link state data, the current scheduling configuration data, the flow attributes, and the rhythm descriptor. The input state data is then input into a time-series-random state encoder to obtain a state tensor.

[0009] Using the state tensor as input, a scheduling policy is obtained using a policy network.

[0010] In some embodiments, the temporal-random state encoder includes a graph neural network, a flow path encoder, and a feature fusion network. The input state data is input into the temporal-random state encoder to obtain a state tensor including:

[0011] The graph neural network generates a graph coding representation based on the input state data, and the graph coding representation includes a link representation and a global topology representation.

[0012] The flow path encoder generates a path representation based on the link sequence traversed by each flow, the link representation of each link, the current scheduling configuration data, and the flow attributes.

[0013] The feature fusion network obtains a flow representation based on the path representation and the rhythm descriptor;

[0014] The state tensor is obtained by concatenating the link representation, the global topology representation, and the flow representation.

[0015] In some embodiments, the graph neural network generates a graph-coded representation based on the input state data, including:

[0016] A graph structure is constructed based on the network state data, and link features are constructed based on the link state data, the current scheduling configuration data, and the flow attributes.

[0017] The graph structure and the link features are input into the graph neural network to obtain the link messages corresponding to the link features;

[0018] The link messages are aggregated to obtain the graph coding representation.

[0019] In some embodiments, after constructing link features based on the link state data, the current scheduling configuration data, and the flow attributes, the method further includes:

[0020] For each link, the gating weight corresponding to the link feature is calculated, and the link message of the current link is adjusted based on the gating weight, so as to obtain the graph coding representation based on the adjusted link message.

[0021] In some embodiments, the flow attributes include flow period, the current scheduling configuration parameters include time slot length, and the flow path encoder generates path representations based on the link sequence traversed by each flow, the link representation of each link, the current scheduling configuration data, and the flow attributes, including:

[0022] For each link through which the flow passes, the phase code of the current link is obtained using a flow path encoder based on the link number, the flow period, and the time slot length. The phase code is used to characterize the relative time slot position of the current link within the flow period.

[0023] The path representation of the flow is obtained by weighted summation of the phase encoding and the link representation of each link.

[0024] In some embodiments, the policy network includes a Transformer backbone network, an offset head, and a guard strip head, wherein the offset head and the guard strip head share the Transformer backbone network; the step of obtaining the scheduling policy using the policy network with the state tensor as input includes:

[0025] The state tensor is input into the Transformer backbone network to obtain the hidden states of each link and each flow in the time-sensitive network to be scheduled.

[0026] Using the hidden state as input, the offset is obtained using the offset head and the protection band is obtained using the protection band head based on the factorization strategy.

[0027] In some embodiments, after obtaining the scheduling policy using the policy network with the state tensor as input, the method further includes:

[0028] Determine whether the scheduling strategy satisfies the hard constraints;

[0029] If not, the current scheduling policy is masked, and a new scheduling policy is generated using the policy network.

[0030] In some embodiments, after regenerating the scheduling policy using the policy network, the method further includes:

[0031] Identify links with local conflicts in the regenerated scheduling policy;

[0032] A local optimization algorithm is used to repair links with local conflicts, resulting in a repaired scheduling strategy.

[0033] Determine whether the repaired scheduling strategy satisfies the hard constraint condition;

[0034] If yes, generate a candidate gating list based on the repaired scheduling policy; if no, disable the repaired scheduling policy and regenerate the scheduling policy using the policy network.

[0035] In some embodiments, after obtaining the scheduling policy using the policy network with the state tensor as input, the method further includes:

[0036] The scheduling strategy is input into the hybrid digital twin module;

[0037] In the hybrid digital twin module, jitter-drift injection and topology perturbation are combined to simulate and evaluate the scheduling strategy, and obtain the evaluation results and the input state data for the next round of scheduling.

[0038] In some embodiments, before performing simulation evaluation on the scheduling policy, the method further includes:

[0039] Determine whether the scheduling strategy satisfies the hard constraint conditions under jitter-drift injection and topology perturbation.

[0040] If not, the scheduling policy is masked and a new scheduling policy is generated using the policy network; if yes, the scheduling policy is then evaluated through simulation.

[0041] In some embodiments, the evaluation results include scheduling rate, protection band waste, and average repair depth, and the method further includes:

[0042] A reward value is obtained based on the scheduling rate, protection band waste, and average repair depth, and the parameters of the policy network are updated based on the reward value.

[0043] In some embodiments, after obtaining the evaluation results and the input state data for the next round of scheduling, the method further includes:

[0044] The parameters of the policy network are updated based on the reward value, and the update magnitude is constrained;

[0045] The updated scheduling policy is obtained based on the input state data of the next round of scheduling and the updated policy network.

[0046] In a second aspect, this application provides a smart device comprising:

[0047] At least one processor;

[0048] And, a memory communicatively connected to the at least one processor;

[0049] The memory stores a computer program that, when executed by the at least one processor, implements the method described above.

[0050] In a third aspect, this application provides a computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the method described in any of the preceding claims.

[0051] Solution 1. A traffic scheduling method for time-sensitive networks, characterized by comprising:

[0052] Obtain network status data, link status data, current scheduling configuration data, and flow set of the time-sensitive network to be scheduled. The flow set includes the flow attributes of each flow, and the flow attributes include at least the release jitter range and the maximum clock drift limit of the flow.

[0053] A rhythmic descriptor for the stream is constructed based on the release jitter range and the maximum clock drift boundary;

[0054] Input state data is constructed using the network state data, the link state data, the current scheduling configuration data, the flow attributes, and the rhythm descriptor. The input state data is then input into a time-series-random state encoder to obtain a state tensor.

[0055] Using the state tensor as input, a scheduling policy is obtained using a policy network.

[0056] Solution 2. The method according to Solution 1, characterized in that the temporal-random state encoder includes a graph neural network, a flow path encoder, and a feature fusion network, and the input state data is input into the temporal-random state encoder to obtain a state tensor including:

[0057] The graph neural network generates a graph coding representation based on the input state data, and the graph coding representation includes a link representation and a global topology representation.

[0058] The flow path encoder generates a path representation based on the link sequence traversed by each flow, the link representation of each link, the current scheduling configuration data, and the flow attributes.

[0059] The feature fusion network obtains a flow representation based on the path representation and the rhythm descriptor;

[0060] The state tensor is obtained by concatenating the link representation, the global topology representation, and the flow representation.

[0061] Solution 3. The method according to Solution 2, characterized in that the graph neural network generates a graph encoding representation based on the input state data, including:

[0062] A graph structure is constructed based on the network state data, and link features are constructed based on the link state data, the current scheduling configuration data, and the flow attributes.

[0063] The graph structure and the link features are input into the graph neural network to obtain the link messages corresponding to the link features;

[0064] The link messages are aggregated to obtain the graph coding representation.

[0065] Solution 4. The method according to Solution 3, characterized in that, after constructing link features based on the link status data, the current scheduling configuration data, and the flow attributes, the method further includes:

[0066] For each link, the gating weight corresponding to the link feature is calculated, and the link message of the current link is adjusted based on the gating weight, so as to obtain the graph coding representation based on the adjusted link message.

[0067] Solution 5. The method according to Solution 2, characterized in that the flow attribute includes flow period, the current scheduling configuration parameter includes time slot length, and the flow path encoder generates path representation based on the link sequence traversed by each flow, the link representation of each link, the current scheduling configuration data, and the flow attribute, including:

[0068] For each link through which the flow passes, the phase code of the current link is obtained using a flow path encoder based on the link number, the flow period, and the time slot length. The phase code is used to characterize the relative time slot position of the current link within the flow period.

[0069] The path representation of the flow is obtained by weighted summation of the phase encoding and the link representation of each link.

[0070] Solution 6. The method according to any one of Solutions 1 to 5, characterized in that the policy network includes a Transformer backbone network, an offset head, and a guard strip head, wherein the offset head and the guard strip head share the Transformer backbone network; the step of obtaining the scheduling policy using the policy network with the state tensor as input includes:

[0071] The state tensor is input into the Transformer backbone network to obtain the hidden states of each link and each flow in the time-sensitive network to be scheduled.

[0072] Using the hidden state as input, the offset is obtained using the offset head and the protection band is obtained using the protection band head based on the factorization strategy.

[0073] Solution 7. The method according to Solution 6, characterized in that, after obtaining the scheduling policy using the policy network with the state tensor as input, the method further includes:

[0074] Determine whether the scheduling strategy satisfies the hard constraints;

[0075] If not, the current scheduling policy is masked, and a new scheduling policy is generated using the policy network.

[0076] Solution 8. The method according to Solution 7, characterized in that, after regenerating the scheduling policy using the policy network, the method further includes:

[0077] Identify links with local conflicts in the regenerated scheduling policy;

[0078] A local optimization algorithm is used to repair links with local conflicts, resulting in a repaired scheduling strategy.

[0079] Determine whether the repaired scheduling strategy satisfies the hard constraint condition;

[0080] If yes, generate a candidate gating list based on the repaired scheduling policy; if no, disable the repaired scheduling policy and regenerate the scheduling policy using the policy network.

[0081] Solution 9. The method according to Solution 6, characterized in that, after obtaining the scheduling policy using the policy network with the state tensor as input, the method further includes:

[0082] The scheduling strategy is input into the hybrid digital twin module;

[0083] In the hybrid digital twin module, jitter-drift injection and topology perturbation are combined to simulate and evaluate the scheduling strategy, and obtain the evaluation results and the input state data for the next round of scheduling.

[0084] Solution 10. The method according to Solution 9, characterized in that, before performing simulation evaluation on the scheduling strategy, the method further includes:

[0085] Determine whether the scheduling strategy satisfies the hard constraint conditions under jitter-drift injection and topology perturbation.

[0086] If not, the scheduling policy is masked and a new scheduling policy is generated using the policy network; if yes, the scheduling policy is then evaluated through simulation.

[0087] Solution 11. The method according to Solution 10, characterized in that the evaluation results include scheduling rate, protection band waste, and average repair depth, and the method further includes:

[0088] A reward value is obtained based on the scheduling rate, protection band waste, and average repair depth, and the parameters of the policy network are updated based on the reward value.

[0089] Solution 12. The method according to Solution 11, characterized in that, after obtaining the evaluation result and the input state data for the next round of scheduling, the method further includes:

[0090] The parameters of the policy network are updated based on the reward value, and the update magnitude is constrained;

[0091] The updated scheduling policy is obtained based on the input state data of the next round of scheduling and the updated policy network.

[0092] Option 13. A smart device, characterized in that it comprises:

[0093] At least one processor;

[0094] And, a memory communicatively connected to the at least one processor;

[0095] The memory stores a computer program that, when executed by the at least one processor, implements the method of any one of schemes 1 to 12.

[0096] Scheme 14. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the method of any one of Schemes 1 to 12.

[0097] When adopting the above technical solution, the traffic scheduling method of this application includes: constructing input state data with network state data, link state data, current scheduling configuration data, flow attributes and rhythm descriptors, and inputting the input state data into a time-series-random state encoder to obtain a state tensor; using the state tensor as input, using a policy network to obtain a scheduling policy; by combining the rhythm descriptor to construct the input state data, and using the time-series-random state encoder and policy network to predict the scheduling policy, the robustness and efficiency of traffic scheduling can be taken into account at the same time. Attached Figure Description

[0098] The preferred embodiments of this application are described below with reference to the accompanying drawings, in which:

[0099] Figure 1 This is a schematic diagram of the overall traffic scheduling process for time-sensitive networks provided in the embodiments of this application;

[0100] Figure 2 This is a flowchart illustrating a traffic scheduling method for a time-sensitive network provided in an embodiment of this application;

[0101] Figure 3 This is a flowchart illustrating the method for obtaining a state tensor using a time-random state encoder, as provided in an embodiment of this application.

[0102] Figure 4 This is a schematic flowchart of a method for obtaining a scheduling policy using a policy network, provided in an embodiment of this application.

[0103] Figure 5 This is a schematic flowchart of the method for checking the feasibility of scheduling strategies provided in the embodiments of this application;

[0104] Figure 6 This is a schematic diagram of the method for simulating and evaluating scheduling strategies provided in an embodiment of this application. Detailed Implementation

[0105] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0106] In the description of this application, "module" and "processor" can include hardware, software, or a combination of both. A module may include hardware circuitry, various suitable sensors, communication ports, and memory, and may also include software components, such as program code, or a combination of software and hardware. A processor may be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor may be implemented in software, in hardware, or a combination of both. Computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc.

[0107] Figure 1 This is a schematic diagram of the overall traffic scheduling process for time-sensitive networks provided in the embodiments of this application. Figure 2 This is a flowchart illustrating a traffic scheduling method for a time-sensitive network provided in an embodiment of this application, which includes steps S1 to S4.

[0108] Step S1: Obtain the network status data, link status data, and current scheduling configuration data stream set of the time-sensitive network to be scheduled. The stream set includes the stream attributes of each stream, and the stream attributes include at least the release jitter range and the maximum clock drift limit of the stream.

[0109] In some embodiments, the time-sensitive network to be scheduled can be a real-world network scenario, such as a brownfield factory workshop network scenario. This time-sensitive network can have multiple network nodes, including at least one of forwarding nodes, terminal nodes, and control nodes. Forwarding nodes are used to forward time-sensitive traffic and perform gating scheduling, and can include at least one of TSN switches and TSN gateways. Terminal nodes are used to generate or receive critical flows and can include at least one of PLCs, robot controllers, and sensors. Control nodes are used to acquire network status and generate or distribute scheduling configurations, and can include at least one of SDN (Software-Defined Networking) controllers and centralized scheduling servers. The scheduling server deploys a time-series-random state encoder, a policy network, a hybrid digital twin module, and a runtime distribution module. The time-series-random state encoder and policy network can be integrated within an agent.

[0110] The time-sensitive network to be scheduled can be represented by a directed graph G=(V,L), where G represents the topology of the time-sensitive network to be scheduled, and V represents the set of network nodes in the time-sensitive network to be scheduled (e.g., ...). Figure 1 In the network, nodes A through F are represented by L, which represents the set of directed links connecting network nodes. A flow refers to a set of periodic messages sent from a source node to a destination node in a network to be scheduled. It usually corresponds to a type of critical service that needs to meet latency, jitter, or reliability requirements. The flow set can be represented as F = {f1, ..., fN}, where N is a positive integer greater than or equal to 1.

[0111] In some embodiments, network state data may include a set of network nodes and a set of links. Link state data may include link capacity, remaining bandwidth, propagation delay, link rate, link utilization, and whether a link has been added, deleted, or failed. Current scheduling configuration data may include the current gating list GCL1 for each link, where GCL1 represents the queue opening and closing time window of the output port corresponding to the directed link l within the scheduling period. In some embodiments, current scheduling configuration data may also include timing, offset, and guard band. Flow attributes may include the source node si, destination node di, flow period Ti, packet length Li, deadline Di, release jitter range [Jmin_i, Jmax_i], and maximum clock drift limit Δmax_i for the current flow fi, where Jmin_i represents the minimum release jitter and Jmax_i represents the maximum release jitter, and i is a positive integer greater than or equal to 1 and less than or equal to N.

[0112] In some embodiments, step S1 may specifically involve the scheduling server obtaining network status data and link status data through the SDN controller, LLDP (Link Layer Discovery Protocol), NETCONF (Network Configuration Protocol), or the switch telemetry interface, and the scheduling server sampling the periodic control messages sent by the terminal nodes to obtain multiple flows and the flow attributes of each flow.

[0113] Step S2: Construct a rhythm descriptor for the stream based on the release jitter range and the maximum clock drift boundary.

[0114] The rhythm descriptor can be represented as Ωi=(Jmin_i,Jmax_i,Δmax_i) to reflect the temporal uncertainty of the current flow fi. As an example, ... Figure 1 As shown, the source node si of the current flow fi is node A, the destination node di is node E, the flow period Ti=8ms, the message length Li=2, the cutoff period Di=256μs, the maximum clock drift limit Δmax_i=1μs, and the rhythm descriptor Ωi=(2μs, 5μs, 1μs).

[0115] Step S3: Construct input state data using network state data, link state data, current scheduling configuration data, flow attributes, and rhythm descriptors, and input the input state data into the time-series-random state encoder to obtain the state tensor.

[0116] In this embodiment, the input state data is constructed using network state data, link state data, current scheduling configuration data, flow attributes, and rhythm descriptors. It also includes spatial topology information such as network state data, link state data, flow attributes, temporal scheduling information such as current scheduling configuration data, and random uncertainty information such as rhythm descriptors. This approach is more flexible than considering only static topology.

[0117] The input state data can be represented as C=({GCLl}l∈L,{Ωi}fi∈F,G,link_attr,flow_attr), where link_attr represents the link state data and flow_attr represents the flow attribute.

[0118] In some embodiments, see Figure 1As shown, a temporal-random state encoder may include a graph neural network, a flow path encoder, and a feature fusion network (not shown in the figure). In some embodiments, the graph neural network may be a GNN (Graph Neural Network). In other embodiments, the graph neural network may be a temporal graph convolutional network, a graph attention network, a message-passing neural network, or a Transformer graph encoder with edge features, the network structure of which includes a node input layer, an edge input layer, a message-passing layer, a temporal / phase embedding layer, and a pooling output layer.

[0119] See Figure 1 and Figure 3 As shown, step S3 involves inputting the input state data into a time-random state encoder to obtain the state tensor, including steps S31 to S34.

[0120] Step S31: The graph neural network generates a graph coding representation based on the input state data. The graph coding representation includes link representation and global topology representation.

[0121] Step S32: The flow path encoder generates a path representation based on the link sequence through which each flow passes, the link representation of each link, the current scheduling configuration data, and the flow attributes.

[0122] Step S33: The feature fusion network obtains the flow representation based on the path representation and rhythm descriptor.

[0123] Step S34: Concatenate the link representation, global topology representation, and flow representation to obtain the state tensor.

[0124] In some embodiments, step S31 includes:

[0125] A graph structure is constructed based on network state data, and link features are built based on link state data, current scheduling configuration data, and flow attributes. The graph structure and link features are input into a graph neural network to obtain link messages corresponding to the link features. These link messages are then aggregated to obtain a graph encoding representation. This graph neural network possesses rhythm-aware capabilities and can perform multi-round link message passing between adjacent links. As an example, [example follows]. Figure 1 As shown, three rounds of message passing can be performed.

[0126] Each directed link *l* can be modeled as an edge with attributes. Link characteristics can include link utilization *ρl*, remaining bandwidth *bres_l*, propagation delay *φl*, maximum queuing or transmission deviation *σmax_l*, current gated duty cycle *g%_l*, guard band ratio *γl / P*, where *P* represents the overcycle length, and the aggregated value of the relative uncertainties *ui* of each flow passing through the link. The relative uncertainty of each flow can be expressed as *ui* = ((Jmax_i - Jmin_i), Δmax_i) / Ti, representing the normalized uncertainty of the jitter range and maximum clock drift limit of the *i*-th flow relative to its period *Ti*. By constructing link characteristics *xl* based on link state data, the link capacity, guard band ratio, release jitter range, and maximum clock drift limit are comprehensively considered, which is beneficial for subsequent models to simultaneously perceive link resource load and timing uncertainties.

[0127] The link message corresponding to the link feature can be represented as ml = σ(Wm[hu||hv||xl]), where hu and hv are the hidden representations of the nodes at both ends of the link, Wm is a trainable weight matrix, || denotes vector concatenation, and σ denotes a non-linear activation function. The global topology representation can be the global network state representation obtained by aggregating the link representations of the entire graph structure.

[0128] In other embodiments, after constructing link features based on link state data, current scheduling configuration data, and flow attributes, the method may further include:

[0129] For each link, the gating weight corresponding to the link feature is calculated, and the link message of the current link is adjusted based on the gating weight to obtain the graph coding representation based on the adjusted link message.

[0130] In some embodiments, the gating weights can be obtained by the following expression: gl = sigmoid(Wg x l), where Wg is the trainable weight matrix and sigmoid represents the gating function. gl represents the gating weight used to adjust the link messages of link l. By adjusting the link messages of the link through the gating weights, the graph neural network can distinguish between low-uncertainty links and high-uncertainty links during encoding, assigning lower gating weights to high-uncertainty links, thereby avoiding excessive influence of high-jitter or high-drift links on the global state.

[0131] In the embodiments of this application, as described above, the flow attribute may include the flow period, and the current scheduling configuration parameter may include the time slot length.

[0132] In some embodiments, step S32 may specifically be:

[0133] For each link through which the flow passes, the phase code of the current link is obtained using the flow path encoder based on the link number, flow period, and time slot length. The phase code is used to characterize the relative time slot position of the current link within the flow period. The phase code and link characterization of each link are weighted and summed to obtain the path characterization of the flow.

[0134] The path of each flow fi can be represented as path(fi)={l1,...,lKi}, and the flow path encoder can encode each link in the path of each flow fi. For the k-th link lk on the path, the phase code obtained based on the link number k, flow period Ti and time slot length τ can be expressed as: pTi_k=(sin(2πk / (Ti / τ)),cos(2πk / (Ti / τ))).

[0135] In some embodiments, the phase encoding and link representation of each link can be weighted and summed using the following expression: h˜i=Σkαi,k(hlk+pTi_k),

[0136] Where hlk is the link representation of link lk, αi,k is the attention weight of the k-th link of the i-th flow, used to highlight bottleneck links or links with high uncertainty on the current flow path, and h˜i represents the path representation of the i-th flow.

[0137] In some embodiments, step S33 may specifically involve concatenating the path representation and rhythm descriptor and inputting them into a feature fusion network to obtain the flow representation ei.

[0138] In some embodiments, the feature fusion network may employ a multilayer perceptron.

[0139] In some embodiments, step S34 may specifically involve concatenating the link representation, global topology representation, and flow representation to obtain the state tensor H using the following expression: H = PadCat(Hnet, Hflow), where Hnet represents the network-side fusion representation, which can be obtained by fusing the link representation and the global topology representation; Hflow represents the flow-side fusion representation, which can be obtained by fusing the various flow representations; PadCat represents padding multiple representation vectors of different lengths with zeros to make the lengths of each representation vector consistent along a preset dimension; and then concatenating the multiple zero-padded representation vectors along the feature dimension to obtain the fusion representation.

[0140] Step S4: Using the state tensor as input, obtain the scheduling policy through the policy network.

[0141] In the embodiments of this application, see Figure 1As shown, the policy network may include a Transformer backbone network, an offset head, and a guard strip, with the offset head and guard strip sharing the Transformer backbone network. As an example, the Transformer backbone network may consist of four 4-layer Transformers as the backbone network, and process the state tensors input using Rotary Position Embedding (RoPE). In other embodiments, the Transformer backbone network may also be replaced with a deeper Transformer, a sparse attention Transformer, a lightweight MLP-Mixer (Multilayer Perceptron Mixer), or a recurrent neural network structure.

[0142] Figure 4 This is a schematic flowchart of a method for obtaining a scheduling policy using a policy network, provided in an embodiment of this application, which includes steps S41 and S42.

[0143] Step S41: Input the state tensor into the Transformer backbone network to obtain the hidden states of each link and each flow in the time-sensitive network to be scheduled.

[0144] Step S42: Using the hidden state as input, obtain the offset using the offset head and the guard band using the guard band head based on the factorization strategy.

[0145] The offset header is used to predict the starting transmission time slot for each flow on each link, i.e., the offset ol,i. The guard band header is used to predict the number of idle guard time slots reserved for each link within a supercycle, i.e., the guard band γl. Here, the supercycle is the repetition period jointly scheduled by multiple flows, which can be the least common multiple of the cycles of multiple flows.

[0146] Based on the factorization strategy, the offset can be obtained by using the offset head and the guard band can be obtained by using the guard band head. First, the offset ol,i can be selected according to the state tensor H, and then the guard band γl can be selected under the condition that the offset is determined.

[0147] The factorization strategy can be expressed as: π(a|H)=πoff(ol,i|H)·πgb(γl|H,ol,i), where a represents the complete scheduling action, πoff represents the offset selection strategy, and πgb represents the guard band selection strategy. By adopting the factorization strategy, the offset and guard band can be predicted separately, decomposing the joint action space into multiple sub-action spaces, achieving action space decoupling, which can significantly improve convergence and robustness, and avoid combining the offset and guard band as a single joint scheduling action for search, thus reducing the size of the action space.

[0148] The scheduling strategy, also known as the candidate scheduling action A, can include the offset and guard band obtained above, and can be expressed as: A={ol,i,γl|l∈L,fi∈F}.

[0149] In some embodiments, Gumbel-Softmax can be used to perform differentiable sampling on the outputs of the offset head and the guard band head, respectively. The resulting sampling can be used for subsequent reward or loss calculations, and the policy network can achieve end-to-end updates through backpropagation. In other embodiments, the scheduling policy can also be directly predicted by the Transformer backbone network.

[0150] In some embodiments, the offset ol,i ranges from [0, P / τ-1], and the guard band γl can be set according to the link rate, maximum release jitter, maximum drift, and engineering safety margin, for example, from 0 to γmax time slots.

[0151] In some embodiments, a feasibility check can be performed on the generated scheduling policy after step S4. See [link to relevant documentation]. Figure 5 As shown, Figure 5 This is a flowchart illustrating the method for feasibility checking of scheduling strategies provided in this application embodiment. The method further includes steps S51 to S57.

[0152] Step S51: Determine whether the scheduling strategy meets the hard constraints.

[0153] If not, proceed to step S52: mask the current scheduling policy and regenerate the scheduling policy using the policy network. If yes, subsequent steps can be performed directly based on the current scheduling policy.

[0154] It should be noted that steps S51 and S52 can be executed repeatedly until a scheduling strategy that satisfies the hard constraints is obtained.

[0155] Among them, hard constraints can be set based on the basic feasibility conditions that candidate scheduling strategies must meet in time-sensitive networks. In some embodiments, hard constraints include at least one of gating consistency constraints, link mutual exclusion constraints, first-in-first-out isolation constraints, deadline constraints, causal constraints, as well as static gating constraints, buffer capacity constraints, and maximum transmission unit constraints.

[0156] Gating consistency constraints are used to limit the transmission of packets only when the gating window of the corresponding link is open; link mutual exclusion constraints are used to limit the transmission of at most one key frame in the same time slot of the same link; first-in-first-out isolation constraints are used to limit the out-of-order transmission of packets within the same flow and to prevent illegal interleaving of different flows on the same link; deadline constraints are used to limit the end-to-end delay of a flow from exceeding the corresponding deadline; causality constraints are used to limit the transmission time slot of a flow on a downstream link to be later than the time after the flow has completed forwarding on the upstream link and passed through the switch for delay processing; static gating constraints are used to limit the candidate scheduling action from violating the pre-configured static gating rules; buffer capacity constraints are used to limit the number of packets to be forwarded in the buffer of a node or port from not exceeding the buffer capacity; maximum transmission unit constraints are used to limit the packet length from not exceeding the maximum transmission unit supported by the corresponding link or port.

[0157] When there are multiple hard constraints, if all scheduling strategies meet multiple hard constraints, it can be determined as a feasible scheduling strategy, and subsequent steps can be executed directly based on the current scheduling strategy; if the scheduling strategy does not meet at least one of the multiple hard constraints, step S52 is executed.

[0158] In some embodiments, after step S52, the method further includes:

[0159] Step S53: Identify links with local conflicts in the regenerated scheduling policy.

[0160] Step S54: Use a local optimization algorithm to repair links with local conflicts and obtain a repaired scheduling strategy.

[0161] Step S55: Determine whether the repaired scheduling strategy meets the hard constraints.

[0162] If yes, proceed to step S56: generate a candidate gating list based on the repaired scheduling policy; if no, proceed to step S57: mask the repaired scheduling policy and regenerate the scheduling policy using the policy network.

[0163] It should be noted that steps S51 to S55 can be executed repeatedly until a repaired scheduling strategy that satisfies the hard constraints is obtained.

[0164] In some embodiments, step S54 may specifically be as follows: based on the regenerated scheduling policy, locate the conflicting links in the current time-series network to be scheduled, the flow and time slot to which the conflicting links belong; adopt the k-opt local optimization algorithm, that is, without changing most of the determined scheduling results, select the offsets of no more than k flows from the affected local window as the variables to be adjusted, perform time slot forward shift, time slot backward shift or variable exchange operations on these selected variables to be adjusted, and recalculate the guard band and end-to-end transmission delay of the links involved in these operations to obtain the repaired scheduling policy.

[0165] In some embodiments, step S55 can be implemented by adaptively replacing the repaired scheduling policy with the scheduling policy in the same manner as step S51, in order to verify the feasibility of the repaired scheduling.

[0166] In some embodiments, the repaired scheduling policy can be represented in the same way as the above-described scheduling policy. In step S56, a candidate gating list is generated based on the repaired scheduling policy. The candidate gating list may include the gate opening or closing time of each link within one supercycle, the offset of each flow transmission, the guard band position, and the guard band length.

[0167] In some embodiments, step S57 can be implemented in the same manner as step S52. In other embodiments, the repaired scheduling policy can also be marked as infeasible and a negative reward returned to update the parameters of the policy network based on the negative reward.

[0168] This application embodiment detects scheduling policies generated by the policy network by combining them with hard constraints, and uses k-opt to achieve local small-scale repair. This avoids global reconfiguration of the scheduling policy, which is conducive to providing efficient scheduling policies and solving the problem of excessively long rescheduling time after topology changes.

[0169] In some embodiments, see Figure 1 As shown, to adapt the scheduling strategy to the timing uncertainties and topology changes in real workshop network scenarios, after step S4 or after steps S5 and S6, a hybrid digital twin module can be used to simulate and evaluate the scheduling strategy. Figure 6 This is a schematic flowchart of a method for simulating and evaluating a scheduling strategy provided in an embodiment of this application, which may include steps S61 and S62.

[0170] Step S61: Input the scheduling strategy into the hybrid digital twin module.

[0171] Step S62: In the hybrid digital twin module, the scheduling strategy is simulated and evaluated by combining jitter-drift injection and topology perturbation to obtain the evaluation results and the input state data for the next round of scheduling.

[0172] In some embodiments, step S62 may specifically involve, in the hybrid digital twin module, combining jitter-drift injection, simulating the actual release time of the k-th packet instance based on the rhythm descriptor Ωi of each flow using the following expression: a(k)_i=k·Ti+ε(k)_i+δ(k)_i, where a(k)_i represents the actual release time of the k-th packet instance of the i-th flow, ε(k)_i represents release jitter, whose value is within the release jitter range [Jmin_i,Jmax_i]; δ(k)_i represents clock drift, whose absolute value does not exceed the maximum clock drift boundary Δmax_i. Combining jitter-drift injection can simulate the non-strict periodic packet sending behavior of traditional terminal nodes in actual production environments; and, combined with topology disturbances, simulating the addition, deletion, or state change of links caused by production line reconstruction, equipment maintenance, AGV path adjustment, or energy-saving strategies; and evaluating the generated scheduling strategy in an environment where the topology of the time-sensitive network to be scheduled changes and non-strict periodic packet sending behavior exists.

[0173] State transition can be achieved by predicting the input state data for the next round of scheduling. By maintaining the input state data for the next round of scheduling, and using it as the initial input state data for the next iteration, state updates can be achieved.

[0174] It should be noted that when performed after step S56, step S61 can specifically involve inputting the repaired scheduling strategy into the hybrid digital twin module. Step S62 can specifically involve performing simulation evaluation on the repaired scheduling strategy.

[0175] In some embodiments, before performing simulation evaluation of the scheduling policy, the method further includes:

[0176] Determine whether the scheduling policy satisfies the hard constraints under jitter-drift injection and topology perturbation. If not, mask the scheduling policy and regenerate it using the policy network. If yes, perform simulation evaluation on the scheduling policy. The hard constraints can be set in the same way as in step S51 above.

[0177] In the embodiments of this application, the evaluation results may include at least one of scheduling rate, guard band waste and average repair depth, wherein the average repair depth can be characterized by how many hops of links are traced back or affected along the transmission path of the data flow, and guard band waste can represent the situation of guard band being idle or ineffectively occupied.

[0178] In some embodiments, the evaluation results may also include the schedule generation time.

[0179] In some embodiments, see Figure 1 As shown, the method also includes:

[0180] Reward values ​​are obtained based on scheduling rate, protection band waste, and average repair depth, and the parameters of the policy network are updated based on these reward values.

[0181] The reward objective can be to maximize the reward value, and it can be represented by the following expression:

[0182] , where 1sched(fi) is an indicator function, which is 1 when the i-th flow meets the deadline constraint, and 0 otherwise; kavg represents the average k-opt repair depth; λγ and λk are adjustable weights used to balance the scheduling rate, the bandwidth waste represented by the second term in the above formula and the repair cost represented by the third term. The scheduling rate is a positive reward, while the bandwidth waste and repair cost are negative rewards.

[0183] This method simulates and evaluates scheduling strategies by combining jitter-drift injection and topology perturbation in a hybrid digital twin module, and updates the parameters of the policy network based on reward values. This helps to make the policy network have robust offset and guard band allocation methods, avoiding the situation where it is only applicable to ideal periodic traffic, and thus has a wider range of applications.

[0184] In some embodiments, see Figure 1 As shown, after obtaining the evaluation results and the input state data for the next round of scheduling, the method further includes:

[0185] The parameters of the policy network are updated based on the reward value, and the update magnitude is constrained; the updated scheduling policy is obtained based on the input state data of the next round of scheduling and the updated policy network.

[0186] In some embodiments, the online reconfiguration condition is determined to be met, i.e., the scheduling policy is regenerated, when the evaluation result indicates a degraded scheduling performance, or when at least one of the following events is detected: link addition / deletion, traffic parameter changes, increased release jitter range, or increased clock drift boundary. Specifically, the input state data of the next round of scheduling can be used as a new task. Several reinforcement learning interaction rounds are collected in the hybrid digital twin simulation module based on the new task. The policy network parameters are updated according to the reward values ​​corresponding to the reinforcement learning interaction rounds, and the updated policy network parameters are obtained by constraining the update magnitude of the policy network parameters based on the Bayesian posterior approximation. Based on the updated policy network parameters and the current input state data, an updated scheduling policy is generated. By using the Bayesian posterior approximation to constrain the update magnitude of the policy network parameters, the update speed of the scheduling policy can be effectively improved.

[0187] In some embodiments, updating the policy network parameters based on the reward value corresponding to the reinforcement learning interaction round can specifically be achieved by using a proximal policy optimization algorithm to calculate the policy gradient based on the reward value and updating the policy network parameters based on the policy gradient.

[0188] In this embodiment of the application, when the network topology or traffic changes, step S3 and several other steps before the scheduling policy is issued can be repeated.

[0189] In other embodiments, the hybrid digital twin module can be expanded with different simulators according to the application scenario. Specifically, it can also include a burst traffic injector, a switch buffer model, and a multi-service priority model. The burst traffic injector simulates non-periodic traffic generated by cameras, diagnostic equipment, or temporary maintenance terminals; the switch buffer model evaluates queue overflow and queuing latency; and the multi-service priority model simulates the priorities of different links or flows. This allows for pre-verification of the robustness of scheduling policies under different disturbance conditions before the policies are issued, reducing on-site trial-and-error costs and improving the effectiveness of generated scheduling policies.

[0190] In some embodiments, the method provided in this application may further include a scheduling server obtaining the final gating list GCL by passing the final verified scheduling policy, and converting the final gating list GCL into configuration data conforming to the data model format (e.g., YANG model) defined by the IEEE 802.1Qcc centralized configuration standard, and distributing it to the forwarding nodes in the network through a standard network configuration protocol (e.g., NETCONF protocol).

[0191] In some embodiments, for cases where only some links or some flows change, an incremental approach can be used to update only the affected GCL lines and perform a hot switch at the next supercycle boundary synchronized by PTP (Precision Time Protocol). This is to avoid configuration inconsistencies when packets cross supercycles, to ensure that forwarding nodes write the new GCL to the inactive configuration area, and to avoid frame loss.

[0192] Once the configuration is distributed, it can include configuring the gate opening or closing time, flow offset, guard band length, and effective expiration period for each port, so that the actual network scenario can achieve traffic scheduling that balances robustness and efficiency based on the obtained scheduling policy.

[0193] In an optional implementation, the method provided in this application can also be applied to the IEEE 60802 industrial automation TSN configuration file scenario. In this scenario, the input data may further include industrial equipment type, service category, priority, redundant paths, time synchronization status, and engineering configuration constraints. Scheduling objectives, in addition to schedulability and protection band waste, may also include service priority satisfaction rate, redundant path consistency, and fault recovery time. Accordingly, before executing step S3, control flow, motion control flow, diagnostic flow, and general data flow may be grouped according to service category; then, the flow scheduling method of this application is executed for critical control flows; non-critical flows may be scheduled using remaining bandwidth or low-priority queues, and finally, a gated list (GCL) or queue configuration compatible with the industrial TSN configuration file is generated uniformly. This adapts to industrial automation and standardized deployment, improving portability across different industrial sites.

[0194] In some embodiments, if edge computing nodes, visual inspection servers, or other devices exist in the current scenario, computational power-network collaborative scheduling can also be performed. In addition to the input state data constructed above, the input data may also include at least one of the following: execution latency of the computation task, computational resource consumption, task deadline, and task dependencies. Based on the resource requirements and deadline of the computation task, an edge computing node or server is selected for the computation task, and the start time of the computation task is determined; based on the completion time of the computation task, the release time and deadline of the corresponding network flow are determined; the updated network flow parameters are input into a time-series-random state encoder, and computation scheduling actions and network scheduling actions are generated through a policy network. The computation scheduling action includes the computation node selection result and the computation task start time, and the network scheduling action includes the offset of the network flow transmission and the guard band length. Furthermore, a gating list (GCL) can be generated or updated based on the network scheduling action to simultaneously satisfy the end-to-end deadline constraints of both the computation task and the network transmission task.

[0195] In one implementation, the engine running the scheduling method provided in this application can be deployed in a central server, edge server, industrial control computer, or switch control plane. For large-scale networks, multi-GPU (Graphics Processing Unit) or multi-process parallel inference can be used; for small-to-medium-scale networks, CPU (Central Processing Unit) or edge accelerator inference can be used; for ports with extremely high real-time requirements, multiple sets of candidate GCLs can be pre-cached and quickly switched when a corresponding topology event is detected. By adjusting the deployment form according to the on-site hardware conditions, fast GCL generation, incremental distribution, and hot switching can be achieved under different computing resource conditions.

[0196] The above describes a traffic scheduling method for time-sensitive networks provided in this application. By combining rhythmic descriptors to construct input state data, the policy network can reserve time margins that match actual uncertainties when selecting offsets and guard bands, rather than assuming all flows arrive strictly periodically, thereby improving schedulability. By employing a factorization strategy to decouple the offset header and guard band prediction, a fixed guard band configuration can be avoided, reducing guard band waste. By using a policy network, combining hard constraints for feasibility checks, and employing local repair, the policy generation speed and effectiveness can be effectively improved, and the reconfiguration time can be significantly shortened. Through simulation evaluation in a hybrid digital twin module and a small number of rounds of Bayesian meta-adaptation in the hybrid digital twin, updating only the offsets and guard bands of affected links and flows improves the robustness and online configuration capability of the scheduling policy. This meets the online reconfiguration requirements in scenarios with rapid topology changes, is applicable to a wider range of traffic scheduling scenarios, and simultaneously considers scheduling robustness and efficiency.

[0197] Those skilled in the art will understand that all or part of the processes in the method of the above-described embodiment can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0198] Another aspect of this application provides a computer-readable storage medium.

[0199] In one embodiment of a computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program that performs the traffic scheduling method of the above-described method embodiments. This program can be loaded and run by a processor to implement the traffic scheduling method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0200] Another aspect of this application provides a smart device.

[0201] In one embodiment of a smart device according to this application, the smart device may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program that, when executed by the at least one processor, implements the method described in any of the above embodiments.

[0202] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A traffic scheduling method for time-sensitive networks, characterized in that, include: Obtain network status data, link status data, current scheduling configuration data, and flow set of the time-sensitive network to be scheduled. The flow set includes the flow attributes of each flow, and the flow attributes include at least the release jitter range and the maximum clock drift limit of the flow. A rhythmic descriptor for the stream is constructed based on the release jitter range and the maximum clock drift boundary; Input state data is constructed using the network state data, the link state data, the current scheduling configuration data, the flow attributes, and the rhythm descriptor. The input state data is then input into a time-series-random state encoder to obtain a state tensor. Using the state tensor as input, a scheduling policy is obtained using a policy network.

2. The method according to claim 1, characterized in that, The temporal-random state encoder includes a graph neural network, a flow path encoder, and a feature fusion network. The input state data is input into the temporal-random state encoder to obtain a state tensor including: The graph neural network generates a graph coding representation based on the input state data, and the graph coding representation includes a link representation and a global topology representation. The flow path encoder generates a path representation based on the link sequence traversed by each flow, the link representation of each link, the current scheduling configuration data, and the flow attributes. The feature fusion network obtains a flow representation based on the path representation and the rhythm descriptor; The state tensor is obtained by concatenating the link representation, the global topology representation, and the flow representation.

3. The method according to claim 2, characterized in that, The graph neural network generates a graph-coded representation based on the input state data, including: A graph structure is constructed based on the network state data, and link features are constructed based on the link state data, the current scheduling configuration data, and the flow attributes. The graph structure and the link features are input into the graph neural network to obtain the link messages corresponding to the link features; The link messages are aggregated to obtain the graph coding representation.

4. The method according to claim 3, characterized in that, After constructing link features based on the link status data, the current scheduling configuration data, and the flow attributes, the method further includes: For each link, the gating weight corresponding to the link feature is calculated, and the link message of the current link is adjusted based on the gating weight, so as to obtain the graph coding representation based on the adjusted link message.

5. The method according to claim 2, characterized in that, The flow attributes include the flow period, the current scheduling configuration parameters include the time slot length, and the flow path encoder generates path representations based on the link sequence traversed by each flow, the link representation of each link, the current scheduling configuration data, and the flow attributes, including: For each link through which the flow passes, the phase code of the current link is obtained using a flow path encoder based on the link number, the flow period, and the time slot length. The phase code is used to characterize the relative time slot position of the current link within the flow period. The path representation of the flow is obtained by weighted summation of the phase encoding and the link representation of each link.

6. The method according to any one of claims 1 to 5, characterized in that, The policy network includes a Transformer backbone network, an offset head, and a guard strip head, wherein the offset head and the guard strip head share the Transformer backbone network; the step of obtaining the scheduling policy using the policy network with the state tensor as input includes: The state tensor is input into the Transformer backbone network to obtain the hidden states of each link and each flow in the time-sensitive network to be scheduled. Using the hidden state as input, the offset is obtained using the offset head and the protection band is obtained using the protection band head based on the factorization strategy.

7. The method according to claim 6, characterized in that, After obtaining the scheduling policy using the policy network with the state tensor as input, the method further includes: Determine whether the scheduling strategy satisfies the hard constraints; If not, mask the current scheduling policy and regenerate the scheduling policy using the policy network.

8. The method according to claim 7, characterized in that, After regenerating the scheduling policy using the policy network, the method further includes: Identify links with local conflicts in the regenerated scheduling policy; A local optimization algorithm is used to repair links with local conflicts, resulting in a repaired scheduling strategy. Determine whether the repaired scheduling strategy satisfies the hard constraint condition; If yes, generate a candidate gating list based on the repaired scheduling policy; if no, disable the repaired scheduling policy and regenerate the scheduling policy using the policy network.

9. The method according to claim 6, characterized in that, After obtaining the scheduling policy using the policy network with the state tensor as input, the method further includes: The scheduling strategy is input into the hybrid digital twin module; In the hybrid digital twin module, jitter-drift injection and topology perturbation are combined to simulate and evaluate the scheduling strategy, and obtain the evaluation results and the input state data for the next round of scheduling.

10. The method according to claim 9, characterized in that, Before performing simulation evaluation on the scheduling strategy, the method further includes: Determine whether the scheduling strategy satisfies the hard constraint conditions under jitter-drift injection and topology perturbation. If not, the scheduling policy is masked and a new scheduling policy is generated using the policy network; if yes, the scheduling policy is then evaluated through simulation.

Images