TSN-based intelligent substation data flow scheduling model training method, scheduling method and device

Abstract

The invention provides an intelligent substation data flow scheduling model training method, scheduling method and device based on a TSN, and the method comprises the steps: equivalently reconstructing a new graph through a link topological structure and a link state of an intelligent substation, and constructing a state space in combination with parameters of newly-added periodic time sensitive flow data flow and newly-added non-periodic time sensitive flow data flow; an actor network pair in the actor commentator network is introduced to calculate and select each candidate node of a next link, and selection probabilities are constructed into an affinity set; searching, simulating and complementing subsequent walk link nodes according to a Monte Carlo tree, calculating an actual profit value, and updating an actor network and a commentator network; the next walk link node is selected in the application based on the trained actor network, the time slot update state is allocated, and the operation is repeated until the target node position is reached, so that the deterministic transmission requirement on the heterogeneous time sensitive data in the TSN network can be ensured, and the overall network stability is improved.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a TSN-based smart substation data stream scheduling model training method, a scheduling method and a device. Background Art

[0002] The bearer network of smart substations needs to meet the requirements of bearing and interconnection of multiple services. At present, the substation network still has various buses coexisting. How to ensure that the requirements of different types of traffic for end-to-end transmission delay, jitter, packet loss rate, etc. are met under mixed flow conditions has been an important issue faced by smart substations for a long time. At this stage, by introducing TSN (Time Sensitive Network) to improve the best-effort forwarding characteristics of traditional Ethernet, according to the different priorities of data traffic, provide different degrees of end-to-end bounded delay guarantee and smaller jitter, etc., so as to meet the application requirements of smart substations.

[0003] The data of smart substations are highly time-sensitive. At the same time, with the development and evolution of smart substations in terms of data, intelligence, and networking, the types of power grid data are also showing a diversified development trend, resulting in an urgent need for the coordinated transmission of heterogeneous time-sensitive data on the power network. In order to achieve the coordinated transmission of heterogeneous time-sensitive data streams in TSN networks, it is proposed to model heterogeneous time-sensitive data streams into two categories: periodic time-sensitive (PTS) streams and non-periodic time-sensitive (STS) streams. PTS streams usually have very strict constraints on periodicity and deterministic delays (such as cyclic control instructions, synchronization information, etc.) in order to generate accurate control decisions and ensure the safe and reliable operation of the control system. STS streams are usually non-periodic sporadic streams (such as fault alarm information) or periodic time-sensitive data streams with large jitter (such as data collected by wireless sensors). The present application aims to provide a solution for the deterministic transmission requirements of smart substation data streams to ensure transmission window restrictions. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a TSN-based smart substation data stream scheduling model training method, scheduling method and device to eliminate or improve one or more defects in the prior art and solve the problem of deterministic transmission requirements for heterogeneous time-sensitive data streams in smart substations. .

[0005] One aspect of the present invention provides a TSN-based smart substation data stream scheduling model training method, the method comprising the following steps:

[0006] Acquire the link topology and link state parameters of the smart substation, wherein the link state parameters include the bandwidth and utilization of each link, and the first data flow parameters of the existing periodic time-sensitive flows and the existing non-periodic time-sensitive flows occupied in each link; the first data flow parameters include the period, link occupation time, data volume, and occupied time slot range;

[0007] Obtaining second data flow parameters of the newly added periodic time-sensitive flow, the second data flow parameters including the period of each of the newly added periodic time-sensitive flows, the generation time within each period, the transmission deadline, the data volume, the source address, and the destination address;

[0008] Obtaining third data flow parameters of the newly added non-periodic time-sensitive flow, the third data flow parameters including the generation time, transmission deadline, data volume, source address and destination address of each of the newly added non-periodic time-sensitive flows;

[0009] The link topology structure and the link state parameters are equivalently reconstructed, the edges in the original link topology structure are used as nodes of the new graph, and the connection relationship between the edges in the original link topology structure is used as the edge of the new graph;

[0010] Taking the second data flow parameter of each newly added periodic time-sensitive flow in the current node, the third data flow parameter of each newly added non-periodic time-sensitive flow and the new graph as state parameters, and taking a set of next link candidate nodes connected to the current node as action options;

[0011] Obtaining an initial actor network and an initial critic network, wherein the actor network includes a first neural network module for mining the feature information of the new graph and a second neural network module for mining the feature information of the current data stream, wherein the first neural network module extracts the feature vector of the previous transmission link node, the feature vector of the current node and the global significant features of the new graph, and the second neural network module extracts the feature vector of the current data stream, and concatenates them to obtain a query vector; passing the query vector through a third neural network module, passing the feature vector of the current node through a fourth neural network module, unifying the dimensions, performing correlation calculation and normalization to obtain the selection probability as the affinity between the current data stream and the candidate node, and constructing an affinity set; based on the affinity set between the current data stream and each candidate node, sampling and selecting the next wandering link node according to probability and updating the state parameter; the initial critic network is used to calculate the prediction value when selecting the corresponding candidate node;

[0012] Based on the selected next walking link node, simulate and complete the subsequent walking link nodes according to the Monte Carlo tree search simulation, and calculate the actual benefit value;

[0013] A loss function is constructed based on the predicted value and the actual benefit value, and the parameters of the initial actor network and the initial critic network are updated to complete one round of iteration; multiple rounds of iterations are performed according to preset conditions, and the updated initial critic network is used as a smart substation data flow scheduling model.

[0014] In some embodiments, before calculating the actual benefit value, the method further includes:

[0015] If the current traffic is the newly added periodic time-sensitive flow, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d k represents the latest completion time of the current data flow k on the current node, l k represents the time slot length occupied by the current data stream k on the current node, P k represents the period of the current data stream k; the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ];

[0016] Calculate k In the infeasible region of the solution space, the calculation process includes:

[0017] For any data flow y that has been deployed on the current node, if the data flow y is a periodic time-sensitive flow, calculate the period P of the current data flow k. k With the period P of the arbitrary data stream y y The least common multiple S of y , A=[0, a); if the arbitrary data stream y is a non-periodic time-sensitive stream, A={0};

[0018] For any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up;

[0019] After obtaining the infeasible region, combined with o kThe value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

[0020] In some embodiments, before calculating the actual benefit value, the method further includes:

[0021] If the current traffic is the newly added non-periodic time-sensitive traffic, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d k represents the latest completion time of the current data flow k on the current node, l k Indicates the time slot length occupied by the current data stream k on the current node, and the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ];

[0022] Calculate k In the infeasible region of the solution space, the calculation process includes:

[0023] For any data flow y deployed on the current node, if the data flow y is a periodic time-sensitive flow, define a=floor(u k / P y ), b = floor(d k / P y ), where floor represents the rounding down operation, A = [a, b]; for any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up;

[0024] If y is non-periodic, then [max(u k ,u y ), min(dk -l k , d y -l y )] is the infeasible region;

[0025] After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

[0026] In some embodiments, the first neural network module and the second neural network module adopt a Transformer model, and the third neural network module and the fourth neural network module adopt a multilayer perceptron.

[0027] In some embodiments, the query vector is passed through a third neural network module, and the current node feature vector is passed through a fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalization is performed to obtain the selection probability as the affinity between the current data stream and the candidate node, and the expression is:

[0028] The feature vector f of the current data stream k is k , the feature vector h of the previous transmission link node l l , the feature vector h of the current node p p , the global salient features g of the new graph are concatenated as the query vector q k , that is, q k =[f k ;h l ;h p ; g];

[0029] definition represents the candidate node set, which is a set of nodes connected to the current node p but does not include nodes that the current data stream k has traversed;

[0030] The affinity d between the current data stream k and the candidate node u ku The calculation formula is:

[0031]

[0032] Wherein, ψ represents the operation of the third neural network, φ represents the operation of the fourth neural network, and is used to convert q k and h u Unify to the same dimension and mine node features again; Indicates the affinity between the current data stream k and the candidate node u after it travels to the current node p.

[0033] In some embodiments, the subsequent walking link nodes are simulated and completed according to the Monte Carlo tree search simulation, and the actual benefit value is calculated, including:

[0034] For the current data stream at the current node z, according to the Monte Carlo tree search, candidate nodes are randomly selected based on affinity and the walking path after the candidate node z′ is selected is simulated. The methods used to construct the walking path during the simulation process include random Rollout, maximum affinity sampling, polynomial sampling and greedy algorithm sampling. The average benefit of the benefits of various methods is calculated as the actual benefit value and then returned.

[0035] In some embodiments, constructing a loss function based on the predicted value and the actual benefit value, and updating parameters of the initial actor network and the initial critic network include:

[0036] Suppose the probability that the wandering path of the current data stream k extends from the current node z to the candidate node z′ is P ZZ′ , when the current data flow k travels to z and z′, the service state of the current node is S Z , the service status of the candidate node z is S Z′ ; Let V(·) denote the value function; γ denote the discount factor; represents the benefit of extending the current data flow k from z to z′ in the walking path;

[0037] Then the loss function expression of the actor network is:

[0038]

[0039] The update rules are:

[0040]

[0041] Wherein, θ represents the set of learnable parameters of the actor network, α is the learning rate of the actor network, τ is the walking path ending at z′, and N represents the length of the path τ;

[0042] The fifth neural network is used as the critic network to predict the value function of the actor network, and the loss function of the critic network is:

[0043]

[0044] The update rules are:

[0045]

[0046] Wherein, ω represents the set of learnable parameters of the critic network, and β is the learning rate of the critic network.

[0047] On the other hand, the present invention also includes a TSN-based smart substation data flow scheduling method, comprising the following steps:

[0048] Acquire the link topology and link state parameters of the smart substation, wherein the link state parameters include the bandwidth and utilization of each link, and the first data flow parameters of the existing periodic time-sensitive flows and the existing non-periodic time-sensitive flows occupied in each link; the first data flow parameters include the period, link occupation time, data volume, and occupied time slot range;

[0049] Obtaining second data flow parameters of the newly added periodic time-sensitive flow, the second data flow parameters including the period of each of the newly added periodic time-sensitive flows, the generation time within each period, the transmission deadline, the data volume, the source address, and the destination address;

[0050] Obtaining third data flow parameters of the newly added non-periodic time-sensitive flow, the third data flow parameters including the generation time, transmission deadline, data volume, source address and destination address of each of the newly added non-periodic time-sensitive flows;

[0051] The link topology structure and the link state parameters are equivalently reconstructed, the edges in the original link topology structure are used as nodes of the new graph, and the connection relationship between the edges in the original link topology structure is used as the edge of the new graph;

[0052] Using the second data flow parameter of each newly added periodic time-sensitive flow in the current node, the third data flow parameter of each newly added non-periodic time-sensitive flow and the new graph as state parameters;

[0053] The state parameters are input into the smart substation data flow scheduling model obtained in the above-mentioned TSN-based smart substation data flow scheduling model training method to select the output of the next wandering link node, allocate time slots for the current data flow according to preset rules, and update the state parameters; repeat the process until the target node position is reached.

[0054] In some embodiments, allocating a time slot for the current data stream according to a preset rule includes:

[0055] If the current traffic is the newly added periodic time-sensitive flow, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d k represents the latest completion time of the current data flow k on the current node, l krepresents the time slot length occupied by the current data stream k on the current node, P k represents the period of the current data stream k; the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ];

[0056] Calculate k In the infeasible region of the solution space, the calculation process includes:

[0057] For any data flow y that has been deployed on the current node, if the data flow y is a periodic time-sensitive flow, calculate the period P of the current data flow k. k With the period P of the arbitrary data stream y y The least common multiple S of y , A=[0, a); if the arbitrary data stream y is a non-periodic time-sensitive stream, A={0};

[0058] For any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up;

[0059] After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

[0060] In some embodiments, allocating a time slot for the current data stream according to a preset rule includes:

[0061] If the current traffic is the newly added non-periodic time-sensitive traffic, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d krepresents the latest completion time of the current data flow k on the current node, l k Indicates the time slot length occupied by the current data stream k on the current node, and the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ];

[0062] Calculate k In the infeasible region of the solution space, the calculation process includes:

[0063] For any data flow y deployed on the current node, if the data flow y is a periodic time-sensitive flow, define a=floor(u k / P y ), b = floor(d k / P y ), where floor represents the rounding down operation, A = [a, b]; for any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up;

[0064] If y is non-periodic, then [max(u k ,u y ), min(d k -l k , d y -l y )] is the infeasible region;

[0065] After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

[0066] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction implements the steps of the above method when executed by a processor.

[0067] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0068] The beneficial effects of the present invention are at least:

[0069] The data flow scheduling model training method, scheduling method and device of the smart substation based on TSN described in the present invention reconstruct a new graph with the link topology structure and link state of the smart substation, and construct a state space in combination with the parameters of the newly added periodic time-sensitive flow and the newly added non-periodic time-sensitive flow data flow, introduce the actor network in the actor critic network to calculate and select the candidate nodes of the next link, and construct the selection probability as an affinity set; simulate and complete the subsequent wandering link nodes according to the Monte Carlo tree search simulation, and calculate the actual benefit value, update the actor network and the critic network; select the next wandering link node in the application based on the trained actor network, and allocate time slots to update the state, and repeat the operation until the target node position is reached. Equivalent reconstruction of the new graph can more effectively utilize network resources, avoid congestion and bottlenecks, and improve the overall performance of the network. The introduction of the actor critic network quickly calculates and updates the scheduling strategy to adapt to the real-time changing network environment. According to the Monte Carlo tree search to calculate the actual benefit value, the search strategy can be dynamically adjusted according to the real-time feedback, and the adaptability is strong, and a better solution can be found within a limited time.

[0070] Furthermore, time slot allocation schemes are constructed for newly added periodic time-sensitive flows and newly added non-periodic time-sensitive flows respectively, which can ensure the deterministic transmission requirements of heterogeneous time-sensitive data in the TSN network and improve the overall network stability.

[0071] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0072] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0074] Figure 1 The present invention is a flowchart of a method for training a data stream scheduling model for a smart substation based on TSN according to an embodiment of the present invention.

[0075] Figure 2 This is a logical diagram of the actor network operation in the TSN-based smart substation data flow scheduling model training method described in an embodiment of the present invention.

[0076] Figure 3 This is a logical schematic diagram of calculating the actual benefit value based on Monte Carlo tree search in the TSN-based smart substation data stream scheduling model training method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0078] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0079] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0080] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection with an intermediate.

[0081] In view of the existing technical problems, in the processing of PTS streams in this application, it is assumed that all cycles are positive integer multiples of the minimum time slot granularity, and the time-aware shaping mechanism (TAS) mechanism defined by IEEE 802.1Qbv is used for fine-grained wait-free scheduling to achieve the optimal end-to-end deterministic delay; however, since STS streams do not have strict sampling period constraints, TAS cannot be directly applied for fine-grained scheduling. For the processing of such streams, the cyclic queue forwarding (CQF) mechanism defined by the IEEE 802.1Qch standard will be used for transmission to achieve end-to-end deterministic upper bound constraints, and to collaborate with the TAS mechanism in the design of queues and gated lists.

[0082] Specifically, the present invention provides a TSN-based smart substation data flow scheduling model training method, such as Figure 1 As shown, the method includes the following steps S101 to S108:

[0083] Step S101: Obtain the link topology and link status parameters of the smart substation. The link status parameters include the bandwidth and utilization of each link, and the first data flow parameters of the existing periodic time-sensitive flows and existing non-periodic time-sensitive flows occupied in each link; the first data flow parameters include the period, link occupancy time, data volume and occupied time slot range.

[0084] Step S102: Obtain second data flow parameters of the newly added periodic time-sensitive flow, where the second data flow parameters include the period of each newly added periodic time-sensitive flow, the generation time within each period, the transmission deadline, the data volume, the source address and the destination address.

[0085] Step S103: Acquire the third data flow parameters of the newly added non-periodic time-sensitive flow, where the third data flow parameters include the generation time, transmission deadline, data volume, source address and destination address of each newly added non-periodic time-sensitive flow.

[0086] Step S104: equivalently reconstruct the link topology structure and link state parameters, use the edges in the original link topology structure as nodes of the new graph, and use the connection relationship between the edges in the original link topology structure as edges of the new graph.

[0087] Step S105: using the second data flow parameters of each newly added periodic time-sensitive flow in the current node, the third data flow parameters of each newly added non-periodic time-sensitive flow and the new graph as state parameters, and the next link candidate node set connected to the current node as action options.

[0088] Step S106: Figure 2 As shown, an initial actor network and an initial critic network are obtained. The actor network includes a first neural network module for mining feature information of the new graph and a second neural network module for mining feature information of the current data stream. The first neural network module extracts the feature vector of the previous transmission link node, the feature vector of the current node and the global significant features of the new graph. The second neural network module extracts the feature vector of the current data stream and concatenates them to obtain a query vector. The query vector passes through the third neural network module, and the feature vector of the current node passes through the fourth neural network module. After unifying the dimensions, the correlation is calculated and normalized to obtain the selection probability as the affinity between the current data stream and the candidate node, and an affinity set is constructed. Based on the affinity set of the current data stream and each candidate node, the next wandering link node is selected by sampling according to probability and the state parameters are updated. The initial critic network is used to calculate the prediction value when selecting the corresponding candidate node.

[0089] Step S107: Based on the selected next wandering link node, simulate and complete the subsequent wandering link nodes according to the Monte Carlo tree search simulation, and calculate the actual benefit value.

[0090] Step S108: construct a loss function based on the predicted value and the actual benefit value, update the parameters of the initial actor network and the initial critic network, and complete one round of iteration; perform multiple rounds of iterations according to preset conditions, and use the updated initial critic network as the smart substation data flow scheduling model.

[0091] In steps S101 to S103, in order to construct the state space and action space required for executing the reinforcement learning algorithm for the operating state of the smart substation, two aspects need to be combined: the existing topological structure and existing data flow of the network on the one hand, and the newly added data flow in the network on the other hand. This part meets the data requirements required for the reinforcement learning model to execute decisions.

[0092] In step S104, by equivalently reconstructing the link topology structure and link state parameters in step S101, network resources are more effectively utilized, congestion and bottlenecks are avoided, and the overall network performance is improved. In the original graph data structure in step S101, each edge of the link connection topology structure of the smart substation network contains information about the corresponding link, including bandwidth, utilization, STS flow and PTS flow information that have occupied the link time slot. The topology structure is equivalently reconstructed into a new graph, and the reconstruction idea is: all edges in the original topology graph are represented by nodes in the new graph, and the connection relationship between edges in the original topology graph is represented by edges in the new graph.

[0093] In steps S105 to S106, the Actor-Critic Network is a reinforcement learning method that combines the Policy Gradient method and the Value Function method to improve learning efficiency and stability. This method uses two main network components (actor network and critic network) to make decisions and evaluations.

[0094] The actor network is responsible for selecting actions based on the current state. It is a policy network that outputs the probability distribution of each possible action, or directly outputs a specific action. Input the current state and output the action or the probability distribution of the action. In this embodiment, the actor network includes a first neural network module for mining the feature information of the new graph and a second neural network module for mining the feature information of the current data stream. The first neural network module extracts the feature vector of the previous transmission link node, the feature vector of the current node, and the global significant features of the new graph, wherein the global significant features are based on the output of the max pooling layer in the module. The second neural network module extracts the feature vector of the current data stream and splices it to obtain the question vector; the question vector is passed through the third neural network module, and the current node feature vector is passed through the fourth neural network module. After unifying the dimensions, the correlation is calculated and normalized to obtain the selection probability as the affinity between the current data stream and the candidate node. In some embodiments, the first neural network module and the second neural network module adopt the Transformer model, and the third neural network module and the fourth neural network module adopt a multi-layer perceptron.

[0095] In some embodiments, the query vector is passed through the third neural network module, and the current node feature vector is passed through the fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalization is performed to obtain the selection probability as the affinity between the current data stream and the candidate node, and the expression is:

[0096] The feature vector f of the current data stream k is k , the feature vector h of the previous transmission link node l l , the feature vector h of the current node p p , the global salient features g of the new image are concatenated as the query vector q k , that is, q k =[f k ;h l ;h p ; g].

[0097] definition Represents a candidate node set, which is a set of nodes connected to the current node p but does not include nodes that the current data stream k has already traveled through.

[0098] The affinity d between the current data stream k and the candidate node u ku The calculation formula is:

[0099]

[0100] Among them, ψ represents the operation of the third neural network, and φ represents the operation of the fourth neural network, which is used to convert q k and h u Unify to the same dimension and mine node features again; Indicates the affinity between the current data stream k and the candidate node u after it travels to the current node p.

[0101] The critic network evaluates the performance of the current strategy. It is a value network that predicts the value of an action and outputs a state value (V-value) or an action-state value (Q-value). It inputs the current state and (possible) action and outputs the value of the current state (or the value of the action-state). In this application, a convolutional neural network can be used.

[0102] In steps S107-S108, after completing one selection, the Monte Carlo tree search simulation is used to simulate and complete the subsequent wandering link nodes, and the actual benefit value is calculated, and the loss function is constructed for iteration. The training logic can be expressed as: state 1 → affinity 1 → sampling and selecting the next wandering node 1 → Monte Carlo completion path → calculation of actual benefit value → back propagation update model → then use sampling 1 to update the state to state 2 → affinity 2 → sampling and selecting the next wandering node 2 → Monte Carlo completion path → calculation of actual benefit value → back propagation update model; repeat the operation.

[0103] In some embodiments, before calculating the actual benefit value, steps S201 to S203 are also included:

[0104] Step S201: If the current traffic is a newly added periodic time-sensitive flow, define k to represent the current data flow, u k Indicates the time when the current data stream arrives at the current node, d k Indicates the latest completion time of the current data stream k on the current node, l k represents the time slot length occupied by the current data stream k on the current node, P k Indicates the period of the current data stream k; the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ].

[0105] Step S202: Calculate o k In the infeasible region of the solution space, the calculation process includes steps S2021 to S2022:

[0106] Step S2021: For any data flow y deployed on the current node, if any data flow y is a periodic time-sensitive flow, calculate the period P of the current data flow k. k With the period P of any data stream y y The least common multiple S of y , A=[0,a); if any data stream y is a non-periodic time-sensitive stream, A={0}.

[0107] Step S2022: For any integer c in A, calculate r=c·P y +u y , e=r+l y , z = r / P k , w=e / P y ;

[0108] If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is an infeasible region.

[0109] If ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is an infeasible region; where ceil means rounding up.

[0110] Step S203: After obtaining the infeasible region, combine o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k idle time slots and allocate them to the current data flow.

[0111] In some embodiments, before calculating the actual benefit value, steps S301 to S303 are also included:

[0112] Step S301: If the current traffic is a newly added non-periodic time-sensitive flow, define k to represent the current data flow, u k Indicates the time when the current data stream arrives at the current node, d k Indicates the latest completion time of the current data stream k on the current node, l k Indicates the length of the time slot occupied by the current data stream k on the current node, and the offset of the time slot starting to occupy is o k , the value range is [u k , d k -l k ];

[0113] Step S302: Calculate o k In the infeasible region of the solution space, the calculation process includes steps S3021 to S3022:

[0114] Step S3021: For any data flow y deployed on the current node, if any data flow y is a periodic time-sensitive flow, define a=floor(u k / P y ), b = florr(dk / P y ), where floor represents the rounding down operation, A = [a, b]; for any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ;

[0115] If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is an infeasible region.

[0116] If ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is an infeasible region; where ceil means rounding up.

[0117] Step S3022: If y is non-periodic, then [max(u k ,u y ), min(d k -l k , d y -l y )] is the infeasible region;

[0118] Step S303: After obtaining the infeasible region, combine o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k idle time slots and allocate them to the current data flow.

[0119] In some embodiments, in step S107, the Monte Carlo tree search simulation is used to simulate and complete the subsequent walking link nodes, and the actual benefit value is calculated, such as Figure 3 As shown, it includes: for the current data stream at the current node z, based on the Monte Carlo tree search, the candidate node is randomly selected based on affinity and the walking path after the candidate node z′ is simulated and selected. The methods used to construct the walking path during the simulation process include random Rollout, maximum affinity sampling, polynomial sampling and greedy algorithm sampling. The average benefit of the benefits of various methods is calculated as the actual benefit value and then returned.

[0120] In some embodiments, in step S108, constructing a loss function based on the predicted value and the actual benefit value, and updating the parameters of the initial actor network and the initial critic network include:

[0121] Suppose the probability that the current data stream k's wandering path extends from the current node z to the candidate node z' is P ZZ′ , when the current data flow k travels to z and z′, the service state of the current node is S Z , the service status of candidate node z′ is S Z′ ; Let V(·) denote the value function; γ denote the discount factor; represents the benefit of expanding the current data flow k of the walking path from z to z′;

[0122] Then the loss function expression of the actor network is:

[0123]

[0124] The update rules are:

[0125]

[0126] Among them, θ represents the set of learnable parameters of the actor network, α is the learning rate of the actor network, τ is the walking path ending at z′, and N represents the length of the path τ.

[0127] The fifth neural network is used as the value function of the critic network to predict the actor network, and the loss function of the critic network is:

[0128]

[0129] The update rules are:

[0130]

[0131] Among them, ω represents the set of learnable parameters of the critic network, and β is the learning rate of the critic network.

[0132] The actor network obtained based on the above training content is directly migrated to the application scenario of this application, calculates the probability of selecting each candidate node as the next link node for the current data traffic in the current node, and samples to determine the next wandering link node.

[0133] Specifically, the present invention also includes a TSN-based smart substation data flow scheduling method, including the following steps S401 to S406:

[0134] Step S401: Obtain the link topology and link status parameters of the smart substation. The link status parameters include the bandwidth and utilization of each link, and the first data flow parameters of the existing periodic time-sensitive flows and existing non-periodic time-sensitive flows occupied in each link; the first data flow parameters include the period, link occupancy time, data volume and occupied time slot range.

[0135] Step S402: Obtain the second data flow parameters of the newly added periodic time-sensitive flow, where the second data flow parameters include the period of each newly added periodic time-sensitive flow, the generation time within each period, the transmission deadline, the data volume, the source address and the destination address.

[0136] Step S403: Obtain third data flow parameters of the newly added non-periodic time-sensitive flow, where the third data flow parameters include generation time, transmission deadline, data volume, source address, and destination address of each newly added non-periodic time-sensitive flow;

[0137] Step S404: equivalently reconstruct the link topology structure and link state parameters, use the edges in the original link topology structure as nodes of the new graph, and use the connection relationship between the edges in the original link topology structure as edges of the new graph.

[0138] Step S405: taking the second data flow parameter of each newly added periodic time-sensitive flow in the current node, the third data flow parameter of each newly added non-periodic time-sensitive flow and the new graph as state parameters.

[0139] Step S406: Input the state parameters into the smart substation data flow scheduling model obtained in the TSN-based smart substation data flow scheduling model training method described in the above steps S101 to S108 to select the output of the next wandering link node, allocate time slots for the current data flow according to preset rules, and update the state parameters; repeat the process until the target node position is reached.

[0140] In some embodiments, allocating a time slot for the current data stream according to a preset rule includes steps S501 to S503:

[0141] Step S501: If the current traffic is a newly added periodic time-sensitive flow, define k to represent the current data flow, u k Indicates the time when the current data stream arrives at the current node, d k Indicates the latest completion time of the current data stream k on the current node, l k represents the time slot length occupied by the current data stream k on the current node, P k Indicates the period of the current data stream k; the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ].

[0142] Step S502: Calculate o k In the infeasible region of the solution space, the calculation process includes steps S5021 to S5022:

[0143] Step S5021: For any data flow y deployed on the current node, if any data flow y is a periodic time-sensitive flow, calculate the period P of the current data flow k. k With the period P of any data stream y y The least common multiple S of y , A=[0,a); if any data stream y is a non-periodic time-sensitive stream, A={0}.

[0144] Step S5022: For any integer c in A, calculate r=c·P y +u y , e=r+l y , z = r / P k , w=e / P y ;

[0145] If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is an infeasible region.

[0146] If ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is an infeasible region; where ceil means rounding up.

[0147] Step S503: After obtaining the infeasible region, combine o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k idle time slots and allocate them to the current data flow.

[0148] In some embodiments, allocating time slots for the current data stream according to a preset rule includes steps S601 to S603:

[0149] Step S601: If the current traffic is a newly added non-periodic time-sensitive flow, define k to represent the current data flow, and u k Indicates the time when the current data flow arrives at the current node, d k Indicates the latest completion time of the current data stream k on the current node, l kIndicates the length of the time slot occupied by the current data stream k on the current node, and the offset of the time slot starting to occupy is o k , the value range is [u k , d k -l k ];

[0150] Step S602: Calculate o k In the infeasible region of the solution space, the calculation process includes steps S6021 to S6022:

[0151] Step S6021: For any data flow y deployed on the current node, if any data flow y is a periodic time-sensitive flow, define a=floor(u k / P y ), b = floor(d k / P y ), where floor represents the rounding down operation, A = [a, b]; for any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ;

[0152] If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is an infeasible region.

[0153] If ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is an infeasible region; where ceil means rounding up.

[0154] Step S6022: If y is non-periodic, then [max(u k ,u y ), min(d k -l k , d y -l y )] is an infeasible region;

[0155] Step S603: After obtaining the infeasible region, combine o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k idle time slots and allocate them to the current data flow.

[0156] Therefore, the scheduling logic can be expressed as: state 1→affinity 1→sample the next node→update state 2→affinity 2→sample the next node→update state 3→affinity 3→…→arrive at the target node position and terminate.

[0157] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction implements the steps of the above method when executed by a processor.

[0158] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0159] The present invention is described below in conjunction with a specific embodiment:

[0160] This embodiment discloses a flow dispatching mechanism for a smart substation, which includes:

[0161] Step S1: Obtain the link topology of the smart substation and the bandwidth, utilization rate, and PTS and STS information of each link that have occupied the link. The PTS and STS information includes the cycle, link occupation time, data volume, and occupied time slot range.

[0162] Step S2: Obtain the newly added PTS flow data related information of the smart substation, including the cycle of each PTS flow, the generation time within each cycle, the transmission deadline (ddl), the data volume, the source address and the destination address (the starting transmission node and the destination receiving node of the flow).

[0163] Step S3: Obtain the newly added STS flow related information of the smart substation, including the generation time of the STS flow, the transmission deadline (ddl), the data volume, the source address and the destination address (the starting transmission node and the destination receiving node of the flow).

[0164] Step S4: Based on the traffic scheduling method of the invention, and the acquired network information and traffic information, a network transmission time slot on each link is allocated to each data stream.

[0165] Step S5: The smart substation network transmits corresponding data to respective receiving nodes in strict accordance with the time slots allocated for each flow by the scheduling method, ensuring that the transmission delay and jitter of each data flow are controllable.

[0166] Furthermore, this embodiment provides a smart substation flow scheduling method, which includes the following steps S11 to S18:

[0167] Step S11: In the link connection topology of the smart substation network, each edge contains the information of the corresponding link, including bandwidth, utilization, STS flow and PTS flow information that have occupied the link time slot. The topology is equivalently reconstructed into a new graph. The reconstruction idea is: all edges in the original topology are represented by nodes in the new graph, and the connection relationship between edges in the original topology is represented by edges in the new graph.

[0168] Step S12: Construct a neural network model based on reinforcement learning to mine important feature information of the graph and feature information of the data stream, and serialize the transmission time slots allocated to each stream on each link. This embodiment uses the actor network in the actor-critic model to perform action decisions. The neural network model includes two parts, namely neural network module A and neural network module B. AB can be implemented using but not limited to the existing Transformer. A is used to mine feature information on a new graph that is equivalently reconstructed, and B is used to mine feature information of each data volume. After being processed by A, the feature information of each node on the new graph can be represented as a feature tensor H (H is an N×d-dimensional tensor, N represents the number of nodes in the node graph, and d is the node feature h u After processing by B, the representation of the data flow is expressed as a tensor F (F is an M×d-dimensional tensor, M represents the total number of data flows in the system, and d is the data flow feature vector h v dimensions).

[0169] Step S13: After obtaining the link feature H and the data stream feature F, calculate the affinity between the link and the data stream. k , the feature vector h of the previous transmission link node l l , the feature vector h of the current link node p p , the global salient features g of the new image are concatenated as the query vector q k , that is, q k =[f k ;h l ;h p ; g]. Represents the set of nodes connected to node p but not including the nodes that data flow k has traveled through. Data flow k and link node The affinity d ku It can be calculated as follows:

[0170]

[0171] Among them, ψ and φ represent neural network operations, which can be implemented by but not limited to multi-layer perceptrons. The purpose is to convert q k and h u Unify to the same dimension and mine node features again; It represents the affinity between data flow k and the connected link node u after it travels to link node p.

[0172] Step S14: When data flow k travels to link node p, the affinity set with other associated link nodes is expressed as based on Based on Monte Carlo tree search, data stream k can select the next wandering link node q through multiple methods such as multinomial sampling and maximum probability sampling. When wandering to link node q, calculate Based on The next wandering link node is selected. This process is repeated until the data stream k wanders to a specific target node and the wandering path is constructed.

[0173] Step S15: If the data stream k is a periodic data stream, execute step S16; if it is a non-periodic data stream, execute step S17.

[0174] Step S16: Allocate transmission time slots on each link node on the wandering path for the periodic data stream k based on the planned wandering path. Specifically,

[0175] Let k be the data flow to be deployed on the current link node, and use u k represents the time when data stream k arrives at the link node, d k represents the latest completion time of data flow k on the link node, l k represents the time slot length occupied by data stream k on the link node, P k Indicates the period of data stream k, and the time slot starts to occupy the offset o k The value range is [u k , d k -l k ], then calculate the infeasible solutions within this range, which includes two steps:

[0176] Step 1: For any data flow y that has been deployed on the link node, if y is a periodic data flow, calculate the period P of data flow k k and the period P of the data stream y y The lowest common multiple (i.e., superperiod) of is S, and a=S / P is defined y , A=[0,a); if y is a non-periodic data stream, A={0}.

[0177] Step 2: For any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / Py ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is an infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is an infeasible region; where ceil means rounding up.

[0178] After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k A free time slot.

[0179] Step 17: Based on the planned wandering path, the transmission time slots on each link node on the wandering path are allocated to the non-periodic data flow k. Specifically, k represents the non-periodic data flow to be deployed at the current node, u k , d k , l k They represent the time when data stream k arrives at the link node, the latest completion time of data stream k on the link node, and the length of the time slot occupied by data stream k on the link node. The offset of the start of the time slot is o k The value range is [u k , d k -l k ]. k The infeasible region in the solution space is calculated as follows:

[0180] For the data flow y that has been deployed on the link node, if y is periodic, define a = floor (u k / P y ), b = floor(d k / P y ), where floor represents the rounding down operation, A = [a, b]; for any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ;

[0181] If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is an infeasible region;

[0182] If ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is an infeasible region; where ceil means rounding up.

[0183] If y is non-periodic, then [max(u k ,u y ), min(d k -l k , d y -l y )] is an infeasible region.

[0184] After calculating the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within this search range until a continuous l k There are idle time slots, that is, o k A feasible solution of .

[0185] Step S18: o of each link node on the wandering path of data flow k k The transmission time slots of each link occupied by data stream k are jointly clarified.

[0186] In steps S12 and S13, the actor network is combined with the critic network for pre-training, and the probability of the walking path extending from Z to Z' is set to P. ZZ′ , when data flow k travels to Z and Z', the link node service states are S Z and S Z′ V(·) represents the value function; γ represents the discount factor; Represents the benefit of expanding the walking path from Z to Z'.

[0187] Then the loss function expression of the actor network is:

[0188]

[0189] The update rules are:

[0190]

[0191] Among them, θ represents the set of learnable parameters of the actor network, α is the learning rate of the actor network, τ is the walking path ending at z, and N represents the length of the path τ.

[0192] The loss function of the critic network is:

[0193]

[0194] The update rules are:

[0195]

[0196] Among them, ω represents the set of learnable parameters of the critic network, and β is the learning rate of the critic network.

[0197] In step S14, Figure 3 As shown in the figure, when the data flow k walks to the link node Z, the extended link node Z' is selected. At this time, the walk path after Z' is simulated and sampled according to the Monte Carlo tree search. In the simulation process, the optional methods for constructing the walk path include random Rollout, maximum affinity sampling, polynomial sampling, greedy algorithm sampling, etc. Each simulation will generate a corresponding walk path and get timely benefits. Then, the benefits of each path are averaged and sent back to the expansion node Z' as the benefit from Z to Z'. It refers to the transmission delay caused by data stream k using the current wandering path, which is generated by the substation network system evaluation.

[0198] Corresponding to the above method, the present invention also provides an apparatus / system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system implements the steps of the method described above.

[0199] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0200] In summary, the training method, scheduling method and device of the data flow scheduling model of the smart substation based on TSN described in the present invention reconstructs a new graph with the link topology structure and link state of the smart substation, and constructs a state space in combination with the parameters of the newly added periodic time-sensitive flow and the newly added non-periodic time-sensitive flow data flow, introduces the actor network in the actor-critic network to calculate and select each candidate node of the next link, and constructs the selection probability as an affinity set; simulates and completes the subsequent wandering link nodes according to the Monte Carlo tree search simulation, and calculates the actual benefit value, updates the actor network and the critic network; selects the next wandering link node in the application based on the trained actor network, and allocates the time slot to update the state, and repeats the operation until the target node position is reached. Equivalent reconstruction of the new graph can more effectively utilize network resources, avoid congestion and bottlenecks, and improve the overall performance of the network. The introduction of the actor-critic network quickly calculates and updates the scheduling strategy to adapt to the real-time changing network environment. The actual benefit value is calculated according to the Monte Carlo tree search, and the search strategy can be dynamically adjusted according to the real-time feedback. It has strong adaptability and can find a better solution within a limited time.

[0201] Furthermore, time slot allocation schemes are constructed for newly added periodic time-sensitive flows and newly added non-periodic time-sensitive flows respectively, which can ensure the deterministic transmission requirements of heterogeneous time-sensitive data in the TSN network and improve the overall network stability.

[0202] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0203] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0204] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0205] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A TSN-based smart substation data stream scheduling model training method, characterized in that: The method comprises the following steps: Acquire the link topology and link state parameters of the smart substation, wherein the link state parameters include the bandwidth and utilization of each link, and the first data flow parameters of the existing periodic time-sensitive flows and the existing non-periodic time-sensitive flows occupied in each link; the first data flow parameters include the period, link occupation time, data volume, and occupied time slot range; Obtaining second data flow parameters of the newly added periodic time-sensitive flow, the second data flow parameters including the period of each of the newly added periodic time-sensitive flows, the generation time within each period, the transmission deadline, the data volume, the source address, and the destination address; Obtaining third data flow parameters of the newly added non-periodic time-sensitive flow, the third data flow parameters including the generation time, transmission deadline, data volume, source address and destination address of each of the newly added non-periodic time-sensitive flows; The link topology structure and the link state parameters are equivalently reconstructed, the edges in the original link topology structure are used as nodes of the new graph, and the connection relationship between the edges in the original link topology structure is used as the edge of the new graph; Taking the second data flow parameter of each newly added periodic time-sensitive flow in the current node, the third data flow parameter of each newly added non-periodic time-sensitive flow and the new graph as state parameters, and taking a set of next link candidate nodes connected to the current node as action options; Obtaining an initial actor network and an initial critic network, wherein the actor network includes a first neural network module for mining the feature information of the new graph and a second neural network module for mining the feature information of the current data stream, wherein the first neural network module extracts the feature vector of the previous transmission link node, the feature vector of the current node and the global significant features of the new graph, and the second neural network module extracts the feature vector of the current data stream, and concatenates them to obtain a query vector; passing the query vector through a third neural network module, passing the feature vector of the current node through a fourth neural network module, unifying the dimensions, performing correlation calculation and normalization to obtain the selection probability as the affinity between the current data stream and the candidate node, and constructing an affinity set; based on the affinity set between the current data stream and each candidate node, sampling and selecting the next wandering link node according to probability and updating the state parameter; the initial critic network is used to calculate the prediction value when selecting the corresponding candidate node; Based on the selected next walking link node, simulate and complete the subsequent walking link nodes according to the Monte Carlo tree search simulation, and calculate the actual benefit value; A loss function is constructed based on the predicted value and the actual benefit value to update the parameters of the initial actor network and the initial critic network, completing one round of iteration; multiple rounds of iterations are performed according to preset conditions, and the updated initial critic network is used as a smart substation data flow scheduling model.

2. The TSN-based smart substation data stream scheduling model training method according to claim 1 is characterized in that: Before calculating the actual return value, it also includes: If the current traffic is the newly added periodic time-sensitive flow, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d k represents the latest completion time of the current data flow k on the current node, l k represents the time slot length occupied by the current data stream k on the current node, P k represents the period of the current data stream k; the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ]; Calculate k In the infeasible region of the solution space, the calculation process includes: For any data flow y that has been deployed on the current node, if the data flow y is a periodic time-sensitive flow, calculate the period P of the current data flow k. k With the period P of the arbitrary data stream y y The least common multiple S of y , A=[0, a); if the arbitrary data stream y is a non-periodic time-sensitive stream, A={0}; For any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up; After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

3. The TSN-based smart substation data stream scheduling model training method according to claim 2 is characterized in that: Before calculating the actual return value, it also includes: If the current traffic is the newly added non-periodic time-sensitive traffic, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d k represents the latest completion time of the current data flow k on the current node, l k Indicates the time slot length occupied by the current data stream k on the current node, and the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ]; Calculate k In the infeasible region of the solution space, the calculation process includes: For any data flow y deployed on the current node, if the data flow y is a periodic time-sensitive flow, define a=floor(u k / P y ), b = floor(d k / P y ), where floor represents the rounding down operation, A = [a, b]; for any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up; If y is non-periodic, then [max(u k ,u y ), min(d k -l k , d y -l y )] is the infeasible region; After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

4. The TSN-based smart substation data stream scheduling model training method according to claim 1 is characterized in that: The first neural network module and the second neural network module adopt Transformer model, and the third neural network module and the fourth neural network module adopt multi-layer perceptron.

5. The TSN-based smart substation data stream scheduling model training method according to claim 1 is characterized in that: The query vector is passed through the third neural network module, and the current node feature vector is passed through the fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalization is performed to obtain the selection probability as the affinity between the current data stream and the candidate node. The expression is: The feature vector f of the current data stream k is k , the feature vector h of the previous transmission link node l l , the feature vector h of the current node p p , the global salient features g of the new graph are concatenated as the query vector q k , that is, q k =[f k ;h l ;h p ; g]; definition represents the candidate node set, which is a set of nodes connected to the current node p but does not include nodes that the current data stream k has traversed; The affinity d between the current data stream k and the candidate node u ku The calculation formula is: Wherein, ψ represents the operation of the third neural network, φ represents the operation of the fourth neural network, and is used to convert q k and h u Unify to the same dimension and mine node features again; Indicates the affinity between the current data stream k and the candidate node u after it travels to the current node p.

6. The TSN-based smart substation data stream scheduling model training method according to claim 1 is characterized in that: Based on the Monte Carlo tree search simulation, the subsequent walking link nodes are simulated and completed, and the actual benefit value is calculated, including: For the current data stream at the current node z, according to the Monte Carlo tree search, candidate nodes are randomly selected based on affinity and the walking path after the candidate node z′ is selected is simulated. The methods used to construct the walking path during the simulation process include random Rollout, maximum affinity sampling, polynomial sampling and greedy algorithm sampling. The average benefit of the benefits of various methods is calculated as the actual benefit value and then returned.

7. The TSN-based smart substation data stream scheduling model training method according to claim 6 is characterized in that: Constructing a loss function based on the predicted value and the actual benefit value, and updating parameters of the initial actor network and the initial critic network, including: Suppose the probability that the wandering path of the current data stream k extends from the current node z to the candidate node z′ is P ZZ′ , when the current data flow k travels to z and z′, the service state of the current node is S Z , the service status of the candidate node z′ is S Z′ ; Let V(·) denote the value function; γ denote the discount factor; represents the benefit of extending the current data flow k from z to z′ in the walking path; Then the loss function expression of the actor network is: The update rules are: Wherein, θ represents the set of learnable parameters of the actor network, α is the learning rate of the actor network, τ is the walking path ending at z′, and N represents the length of the path τ; The fifth neural network is used as the critic network to predict the value function of the actor network, and the loss function of the critic network is: The update rules are: Wherein, ω represents the set of learnable parameters of the critic network, and β is the learning rate of the critic network.

8. A data flow scheduling method for smart substation based on TSN, characterized in that: The steps include: Acquire the link topology and link state parameters of the smart substation, wherein the link state parameters include the bandwidth and utilization of each link, and the first data flow parameters of the existing periodic time-sensitive flows and the existing non-periodic time-sensitive flows occupied in each link; the first data flow parameters include the period, link occupation time, data volume, and occupied time slot range; Obtaining second data flow parameters of the newly added periodic time-sensitive flow, the second data flow parameters including the period of each of the newly added periodic time-sensitive flows, the generation time within each period, the transmission deadline, the data volume, the source address, and the destination address; Obtaining third data flow parameters of the newly added non-periodic time-sensitive flow, the third data flow parameters including the generation time, transmission deadline, data volume, source address and destination address of each of the newly added non-periodic time-sensitive flows; The link topology structure and the link state parameters are equivalently reconstructed, the edges in the original link topology structure are used as nodes of the new graph, and the connection relationship between the edges in the original link topology structure is used as the edge of the new graph; Using the second data flow parameter of each newly added periodic time-sensitive flow in the current node, the third data flow parameter of each newly added non-periodic time-sensitive flow and the new graph as state parameters; The state parameters are input into the smart substation data flow scheduling model obtained in the TSN-based smart substation data flow scheduling model training method described in any one of claims 1 to 7 to select the output of the next wandering link node, allocate time slots for the current data flow according to preset rules, and update the state parameters; repeat the process until the target node position is reached.

9. The TSN-based smart substation data flow scheduling method according to claim 8 is characterized in that: Allocating a time slot for the current data stream according to a preset rule includes: If the current traffic is the newly added periodic time-sensitive flow, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d k represents the latest completion time of the current data flow k on the current node, l k represents the time slot length occupied by the current data stream k on the current node, P k represents the period of the current data stream k; the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ]; Calculate k In the infeasible region of the solution space, the calculation process includes: For any data flow y that has been deployed on the current node, if the data flow y is a periodic time-sensitive flow, calculate the period P of the current data flow k. k With the period P of the arbitrary data stream y y The least common multiple S of y , A=[0, a); if the arbitrary data stream y is a non-periodic time-sensitive stream, A={0}; For any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up; After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

10. The method for data flow scheduling of a smart substation based on TSN according to claim 8, characterized in that: Allocating a time slot for the current data stream according to a preset rule includes: If the current traffic is the newly added non-periodic time-sensitive traffic, define k to represent the current data flow, u k Indicates the time when the current data flow arrives at the current node, d k represents the latest completion time of the current data flow k on the current node, l k Indicates the time slot length occupied by the current data stream k on the current node, and the time slot starts to occupy the offset o k , the value range is [u k , d k -l k ]; Calculate k In the infeasible region of the solution space, the calculation process includes: For any data flow y deployed on the current node, if the data flow y is a periodic time-sensitive flow, define a=floor(u k / P y ), b = floor(d k / P y ), where floor represents the rounding down operation, A = [a, b]; for any integer c in A, calculate r = c·P y +u y , e=r+l y , z = r / P k , w=e / P y ; If ceil(z)-ceil(w)==0, then [r%P y , e%P y ] is the infeasible region; if ceil(w)-ceil(z)>0, [r%P y , P y ] and [0, e%P y ] is the infeasible region; wherein ceil means rounding up; If y is non-periodic, then [max(u k ,u y ), min(d k -l k , d y -l y )] is the infeasible region; After obtaining the infeasible region, combined with o k The value range of [u k , d k -l k ]Shrink k The search range is then exhausted within the feasible search range until a continuous l k An idle time slot is allocated to the current data flow.

11. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images