Training method, scheduling method and device for data flow scheduling model of smart substation based on TSN

Abstract

This invention provides a training method, scheduling method, and apparatus for a data flow scheduling model in a smart substation based on Time-Sensitive Network (TSN). It reconstructs a new graph using the link topology and link states of the smart substation, and constructs a state space by combining parameters of newly added periodic and aperiodic time-sensitive flows. An actor network from the actor-critic network is introduced to calculate and select candidate nodes for the next link, constructing the selection probability as an affinity set. Monte Carlo tree search simulation is used to complete the subsequent wandering link nodes, and the actual benefit value is calculated to update the actor and critic networks. Based on the trained actor network, the next wandering link node is selected in the application, and time slots are allocated to update the state. This process is repeated until the target node position is reached. This ensures deterministic transmission of heterogeneous time-sensitive data in the TSN network, improving overall network stability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network communication, and in particular to a TSN-based intelligent substation data flow scheduling model training method and device. BACKGROUND

[0002] The bearing network of an intelligent substation needs to meet the bearing and interconnection requirements of multiple services. At present, the networks of substations still coexist with various buses. How to ensure that the requirements of various types of traffic for end-to-end transmission delay, jitter, and packet loss rate are met under mixed flow conditions is an important problem that intelligent substations have faced for a long time. At present, TSN (Time-Sensitive Network) is introduced to improve the forwarding characteristics of traditional Ethernet best effort, and according to the different priorities of data traffic, different degrees of end-to-end bounded delay protection and smaller jitter are provided, so as to meet the application requirements of intelligent substations.

[0003] The data of an intelligent substation has a high degree of time-sensitive characteristics. At the same time, with the development and evolution of data, intelligence, and networking of intelligent substations, the types of power grid data also show a diversified development trend, and there is an urgent need for the collaborative transmission of heterogeneous time-sensitive data on the power network. In order to realize the collaborative transmission of heterogeneous time-sensitive data flows in a TSN network, the heterogeneous time-sensitive data flows are modeled into two categories: periodic time-sensitive (PTS) flows and sporadic time-sensitive (STS) flows. PTS flows usually have very strict periodicity and deterministic delay constraints (such as cyclic control instructions, synchronization information, etc.) in order to produce accurate control decisions and ensure the safe and reliable operation of control systems. STS flows are usually non-periodic sporadic flows (such as fault alarm information) or periodic time-sensitive data flows with large jitter (such as data collected by wireless sensors), etc. The present application aims to provide a scheme for the deterministic transmission requirements of intelligent substation data flows to guarantee transmission window restrictions. SUMMARY

[0004] In view of this, the embodiments of the present application provide a TSN-based intelligent substation data flow scheduling model training method and device to eliminate or improve one or more defects in the prior art and solve the problem of deterministic transmission requirements of heterogeneous time-sensitive data flows in intelligent substations.

[0005] One aspect of the present application provides a TSN-based intelligent substation data flow scheduling model training method, which includes the following steps:

[0006] The link topology and link status parameters of the smart substation are obtained. The link status parameters include the bandwidth and utilization of each link, and the first data stream parameters of the existing periodic time-sensitive flow and the existing non-periodic time-sensitive flow that are occupied in each link. The first data stream parameters include the period, the link occupation duration, the data volume, and the occupation time slot range.

[0007] Obtain the second data stream parameters of the newly added periodic time-sensitive stream. The second data stream parameters include the period of each newly added periodic time-sensitive stream, the generation time within each period, the transmission deadline, the data size, the source address, and the destination address.

[0008] Obtain the third data stream parameters of the newly added aperiodic time-sensitive stream. The third data stream parameters include the generation time, transmission deadline, data size, source address, and destination address of each newly added aperiodic time-sensitive stream.

[0009] The link topology and link state parameters are reconstructed equivalently, and the edges in the original link topology are used as nodes in the new graph, and the connection relationships between edges in the original link topology are used as edges in the new graph.

[0010] The second data stream parameters of each newly added periodic time-sensitive stream in the current node, the third data stream parameters of each newly added non-periodic time-sensitive stream, and the new graph are used as state parameters, and the set of candidate nodes for the next link connected to the current node is used as action options.

[0011] An initial actor network and an initial critic network are obtained. The actor network includes a first neural network module for mining feature information of the new graph and a second neural network module for mining feature information of the current data stream. The first neural network module extracts the feature vector of the previous transmission link node, the feature vector of the current node, and the global salient features of the new graph. The second neural network module extracts the feature vector of the current data stream and concatenates them to obtain a challenge vector. The challenge vector is passed through a third neural network module, and the feature vector of the current node is passed through a fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalized to obtain the selection probability as the affinity between the current data stream and the candidate nodes, and an affinity set is constructed. Based on the affinity set between the current data stream and each candidate node, the next walking link node is selected according to the probability, and the state parameters are updated. The initial critic network is used to calculate the prediction value when selecting the corresponding candidate node.

[0012] Based on the selected next traversal node, the Monte Carlo tree search is used to simulate and complete the subsequent traversal nodes, and the actual revenue value is calculated.

[0013] Based on the predicted value and the actual revenue value, a loss function is constructed, and the parameters of the initial actor network and the initial critic network are updated to complete one round of iteration. Multiple rounds of iteration are performed according to preset conditions, and the updated initial critic network is used as the data flow scheduling model for the smart substation.

[0014] In some embodiments, before calculating the actual revenue value, the method further includes:

[0015] If the current data stream is the newly added periodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node. This indicates the period of the current data stream k; the time slot start offset is... The range of values ​​is ;

[0016] calculate In the infeasible region of the solution space, the computation process includes:

[0017] For any data stream y already deployed on the current node, if the data stream y is a periodic time-sensitive stream, calculate the period of the current data stream k. With the period of the arbitrary data stream y The least common multiple S is defined as follows: , If the arbitrary data stream y is an aperiodic time-sensitive stream, ;

[0018] For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up;

[0019] After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0020] In some embodiments, before calculating the actual revenue value, the method further includes:

[0021] If the current data stream is the newly added aperiodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node, with the time slot starting offset being... The range of values ​​is ;

[0022] calculate In the infeasible region of the solution space, the computation process includes:

[0023] For any data stream y already deployed on the current node, if the arbitrary data stream y is a periodic time-sensitive stream, define... , ,in This indicates the floor function. For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up;

[0024] If y is non-periodic, then This refers to the infeasible region;

[0025] After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0026] In some embodiments, the first neural network module and the second neural network module adopt the Transformer model, and the third neural network module and the fourth neural network module adopt the multilayer perceptron.

[0027] In some embodiments, the challenge vector is processed through a third neural network module, and the current node feature vector is processed through a fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalized to obtain the selection probability as the affinity between the current data stream and the candidate node. The calculation steps are as follows:

[0028] The feature vector of the current data stream k Previous transmission link node eigenvectors The current node eigenvectors The global salient features of the new graph Concatenation as a question vector ,Right now ;

[0029] definition This represents the set of candidate nodes, which is related to the current node. The set of nodes that are connected, but not included, that the current data stream k has already traversed;

[0030] The current data stream k and the candidate nodes affinity The calculation formula is:

[0031] ;

[0032] ;

[0033] in, This represents the function corresponding to the third neural network. The function corresponding to the fourth neural network is used to... and Unify to the same dimension and further mine node features; This indicates that the current data stream k has traveled to the current node. The affinity with the candidate node u.

[0034] In some embodiments, the Monte Carlo tree search simulation is used to complete the subsequent traversal link nodes and calculate the actual revenue value, including:

[0035] For the current data stream at the current node z, candidate nodes are randomly selected based on affinity using the Monte Carlo tree search, and the candidate node selection is simulated. The subsequent walking path, during the simulation process, is constructed using methods including random Rollout, combined with maximum affinity sampling, combined with multinomial sampling, and combined with greedy algorithm sampling. The average benefit of each method is calculated and used as the actual benefit value before being transmitted back.

[0036] In some embodiments, a loss function is constructed based on the predicted value and the actual revenue value to update the parameters of the initial actor network and the initial critic network, including:

[0037] Suppose that the current data stream k's traversal path extends from the current node z to the candidate node. The probability is The current data stream k travels to z and At that time, the service status of the current node is The candidate nodes The service status is ;make Represents a value function; Indicates the discount factor; The current data stream k in the traversal path extends from z to... The benefits;

[0038] The loss function expression for the actor network is:

[0039] ;

[0040] The update rules are as follows:

[0041] ;

[0042] in, This represents the set of learnable parameters of the actor network. The learning rate of the actor network. As of the end The walking path, where N represents the path Length;

[0043] A fifth neural network is used as the value function for the critic network to predict the actor network, and the loss function of the critic network is:

[0044] ;

[0045] The update rules are as follows:

[0046] ;

[0047] in, This represents the set of learnable parameters of the critic network. is the learning rate of the critic network.

[0048] On the other hand, the present invention also includes a data flow scheduling method for smart substations based on TSN, comprising the following steps:

[0049] The link topology and link status parameters of the smart substation are obtained. The link status parameters include the bandwidth and utilization of each link, and the first data stream parameters of the existing periodic time-sensitive flow and the existing non-periodic time-sensitive flow that are occupied in each link. The first data stream parameters include the period, the link occupation duration, the data volume, and the occupation time slot range.

[0050] Obtain the second data stream parameters of the newly added periodic time-sensitive stream. The second data stream parameters include the period of each newly added periodic time-sensitive stream, the generation time within each period, the transmission deadline, the data size, the source address, and the destination address.

[0051] Obtain the third data stream parameters of the newly added aperiodic time-sensitive stream. The third data stream parameters include the generation time, transmission deadline, data size, source address, and destination address of each newly added aperiodic time-sensitive stream.

[0052] The link topology and link state parameters are reconstructed equivalently, and the edges in the original link topology are used as nodes in the new graph, and the connection relationships between edges in the original link topology are used as edges in the new graph.

[0053] The second data stream parameters of each newly added periodic time-sensitive stream in the current node, the third data stream parameters of each newly added non-periodic time-sensitive stream, and the new graph are used as state parameters;

[0054] The state parameters are input into the intelligent substation data flow scheduling model obtained in the above-mentioned TSN-based intelligent substation data flow scheduling model training method to select the next roaming link node, allocate time slots to the current data flow according to preset rules, and update the state parameters; repeat the process until the target node position is reached and terminated.

[0055] In some embodiments, allocating time slots to the current data stream according to preset rules includes:

[0056] If the current data stream is the newly added periodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node. This indicates the period of the current data stream k; the time slot start offset is... The range of values ​​is ;

[0057] calculate In the infeasible region of the solution space, the computation process includes:

[0058] For any data stream y already deployed on the current node, if the data stream y is a periodic time-sensitive stream, calculate the period of the current data stream k. With the period of the arbitrary data stream y The least common multiple S is defined as follows: , If the arbitrary data stream y is an aperiodic time-sensitive stream, ;

[0059] For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up;

[0060] After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0061] In some embodiments, allocating time slots to the current data stream according to preset rules includes:

[0062] If the current data stream is the newly added aperiodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node, with the time slot starting offset being... The range of values ​​is ;

[0063] calculate In the infeasible region of the solution space, the computation process includes:

[0064] For any data stream y already deployed on the current node, if the arbitrary data stream y is a periodic time-sensitive stream, define... , ,in This indicates the floor function. For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up;

[0065] If y is non-periodic, then This refers to the infeasible region;

[0066] After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0067] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the above-described method.

[0068] On the other hand, the present invention also provides a computer program product, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the above-described method.

[0069] The beneficial effects of the present invention are at least as follows:

[0070] The present invention describes a training method, scheduling method, and apparatus for a data flow scheduling model in a smart substation based on TSN. It reconstructs a new graph using the link topology and link states of the smart substation, and constructs a state space by combining parameters of newly added periodic and aperiodic time-sensitive flows. An actor network from an actor-critic network is introduced to calculate and select candidate nodes for the next link, constructing an affinity set based on the selection probabilities. The subsequent traversal link nodes are simulated and completed using Monte Carlo tree search, and the actual reward value is calculated, updating the actor and critic networks. Based on the trained actor network, the next traversal link node is selected in the application, and time slots are allocated to update the state. This process is repeated until the target node position is reached. The reconstructed graph utilizes network resources more effectively, avoids congestion and bottlenecks, and improves overall network performance. The introduction of the actor-critic network allows for rapid calculation and updating of scheduling strategies, adapting to real-time changes in the network environment. The calculation of actual reward values ​​using Monte Carlo tree search allows for dynamic adjustment of the search strategy based on real-time feedback, demonstrating strong adaptability and the ability to find a better solution within a limited time.

[0071] Furthermore, time slot allocation schemes are constructed for newly added periodic time-sensitive flows and newly added aperiodic time-sensitive flows respectively, which can ensure the deterministic transmission requirements of heterogeneous time-sensitive data in the TSN network and improve the overall network stability.

[0072] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0073] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0074] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0075] Figure 1 This is a flowchart illustrating the training method for a data flow scheduling model of a smart substation based on TSN according to an embodiment of the present invention.

[0076] Figure 2 This is a schematic diagram of the actor network operation logic in the training method of the intelligent substation data flow scheduling model based on TSN according to an embodiment of the present invention.

[0077] Figure 3 This is a schematic diagram illustrating the logic of calculating the actual benefit value based on Monte Carlo tree search in the training method of the intelligent substation data flow scheduling model based on TSN according to an embodiment of the present invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0079] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0080] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0081] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0082] To address the existing technical problems, this application processes PTS streams by assuming all periods are positive integer multiples of the smallest timeslot granularity. It employs the Time Aware Shaping (TAS) mechanism defined by IEEE 802.1Qbv for fine-grained, wait-free scheduling to achieve optimal end-to-end deterministic latency. However, since STS streams do not have strict sampling period constraints, TAS cannot be directly applied for fine-grained scheduling. For this type of stream, the Circular Queuing Forwarding (CQF) mechanism defined by the IEEE 802.1Qch standard is used for transmission to implement end-to-end deterministic upper bound constraints, and it coordinates with the TAS mechanism in the design of queues and gating lists.

[0083] Specifically, this invention provides a training method for a data flow scheduling model of a smart substation based on TSN, such as... Figure 1 As shown, the method includes the following steps S101~S108:

[0084] Step S101: Obtain the link topology and link status parameters of the smart substation. The link status parameters include the bandwidth and utilization of each link, the first data stream parameters of the existing periodic time-sensitive flow and the existing non-periodic time-sensitive flow that are occupied in each link; the first data stream parameters include the period, the link occupation duration, the data volume, and the occupation time slot range.

[0085] Step S102: Obtain the second data stream parameters of the newly added periodic time-sensitive stream. The second data stream parameters include the period of each newly added periodic time-sensitive stream, the generation time within each period, the transmission deadline, the data size, the source address, and the destination address.

[0086] Step S103: Obtain the third data stream parameters of the newly added aperiodic time-sensitive stream. The third data stream parameters include the generation time, transmission deadline, data size, source address, and destination address of each newly added aperiodic time-sensitive stream.

[0087] Step S104: Perform equivalent reconstruction of the link topology and link state parameters, using the edges in the original link topology as nodes in the new graph, and the connection relationships between edges in the original link topology as edges in the new graph.

[0088] Step S105: Using the second data stream parameters of each newly added periodic time-sensitive stream in the current node, the third data stream parameters of each newly added non-periodic time-sensitive stream, and the new graph as state parameters, and the set of candidate nodes for the next link connected to the current node as action options.

[0089] Step S106: As Figure 2 As shown, an initial actor network and an initial critic network are obtained. The actor network includes a first neural network module for mining new graph feature information and a second neural network module for mining current data stream feature information. The first neural network module extracts the feature vectors of the previous transmission link node, the current node feature vector, and the global salient features of the new graph. The second neural network module extracts the current data stream feature vector and concatenates them to obtain a challenge vector. The challenge vector is passed through a third neural network module, and the current node feature vector is passed through a fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalized to obtain the selection probability as the affinity between the current data stream and the candidate node, and an affinity set is constructed. Based on the affinity set between the current data stream and each candidate node, the next walking link node is selected according to the probability and the state parameters are updated. The initial critic network is used to calculate the prediction value when selecting the corresponding candidate node.

[0090] Step S107: Based on the selected next traversal node, simulate and complete the subsequent traversal nodes according to the Monte Carlo tree search, and calculate the actual revenue value.

[0091] Step S108: Construct a loss function based on the predicted value and the actual revenue value, update the parameters of the initial actor network and the initial critic network, and complete one round of iteration; perform multiple rounds of iteration according to preset conditions, and use the updated initial critic network as the data flow scheduling model for the smart substation.

[0092] In steps S101-S103, to construct the state space and action space required for executing the reinforcement learning algorithm based on the operating state of the smart substation, two aspects need to be combined: the existing network topology and existing data flow, and newly added data flow in the network. This part satisfies the data requirements for the reinforcement learning model to execute decisions.

[0093] In step S104, by performing an equivalent reconstruction of the link topology and link state parameters from step S101, network resources are utilized more effectively, congestion and bottlenecks are avoided, and overall network performance is improved. In the original graph data structure of step S101, each edge in the link connection topology of the smart substation network contains information about the corresponding link, including bandwidth, utilization rate, and STS and PTS traffic information that has already occupied the link's time slot. This topology is equivalently reconstructed into a new graph. The reconstruction idea is that all edges in the original topology graph are represented by nodes in the new graph, and the connection relationships between edges in the original topology graph are represented by edges in the new graph.

[0094] In steps S105-S106, the Actor-Critic Network is a reinforcement learning method that combines policy gradient and value function methods to improve learning efficiency and stability. This method uses two main network components (the actor network and the critic network) for decision-making and evaluation.

[0095] The actor network is responsible for selecting actions based on the current state. It is a policy network that outputs the probability distribution of each possible action or directly outputs a specific action. The input is the current state, and the output is an action or the probability distribution of that action. In this embodiment, the actor network includes a first neural network module for mining new graph feature information and a second neural network module for mining current data stream feature information. The first neural network module extracts the feature vectors of the previous transmission link node, the current node feature vector, and the global salient features of the new graph. The global salient features are based on the output of the max pooling layer in the module. The second neural network module extracts the current data stream feature vector and concatenates them to obtain a challenge vector. The challenge vector is passed through a third neural network module, and the current node feature vector is passed through a fourth neural network module. After unifying the dimensions, correlation calculations are performed and normalized to obtain the selection probability as the affinity between the current data stream and the candidate node. In some embodiments, the first and second neural network modules use a Transformer model, and the third and fourth neural network modules use a multilayer perceptron.

[0096] In some embodiments, the challenge vector is passed through a third neural network module, and the current node feature vector is passed through a fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalized to obtain the selection probability as the affinity between the current data stream and the candidate node. The calculation steps are as follows:

[0097] The feature vector of the current data stream k Previous transmission link node eigenvectors Current node eigenvectors Global salient features of the new graph Concatenation as a question vector ,Right now .

[0098] definition This represents the set of candidate nodes, which is related to the current node. The set of nodes that are connected but not included in the current data stream k has already traversed.

[0099] Current data stream k and the candidate node affinity The calculation formula is:

[0100] ;

[0101] ;

[0102] in, This represents the function corresponding to the third neural network. The function corresponding to the fourth neural network is used to... and Unify to the same dimension and further mine node features; This indicates that the current data stream k has moved to the current node. The affinity between the node and the candidate node u.

[0103] The critic network evaluates the performance of the current policy. It is a value network used to predict the value of actions, outputting a state value (V-value) or an action-state value (Q-value). The input is the current state and (possible) actions, and the output is the value of the current state (or the action-state value). In this application, a convolutional neural network may be employed.

[0104] In steps S107-S108, after one selection is completed, the Monte Carlo tree search simulation is used to complete the subsequent walking link nodes, and the actual reward value is calculated. The loss function is then constructed for iteration. The training logic can be represented as: State 1 → Affinity 1 → Sampling and selecting the next walking node 1 → Monte Carlo path completion → Calculating the actual reward value → Backpropagation to update the model → Update the state to State 2 using sampling 1 → Affinity 2 → Sampling and selecting the next walking node 2 → Monte Carlo path completion → Calculating the actual reward value → Backpropagation to update the model; repeat the operation.

[0105] In some embodiments, before calculating the actual revenue value, steps S201 to S203 are further included:

[0106] Step S201: If the current data stream is a newly added periodic time-sensitive stream, define k as the current data stream. This indicates the time when the current data stream arrives at the current node. This represents the latest completion time of the current data stream k on the current node. This represents the length of the time slot occupied by the current data stream k on the current node. This indicates the period of the current data stream k; the offset at the start of the time slot is... The range of values ​​is .

[0107] Step S202: Calculation In the infeasible region of the solution space, the computation process includes steps S2021~S2022:

[0108] Step S2021: For any data stream y already deployed on the current node, if any data stream y is a periodic time-sensitive stream, calculate the period of the current data stream k. With the period of any data stream y The least common multiple S is defined as follows: , If any data stream y is an aperiodic time-sensitive stream, .

[0109] Step S2022: For any integer c in A, calculate , , , ;

[0110] if ,but This is an infeasible region.

[0111] like , and The region is infeasible; where ceil represents rounding up.

[0112] Step S203: After obtaining the infeasible region, combine... The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0113] In some embodiments, before calculating the actual revenue value, steps S301 to S303 are further included:

[0114] Step S301: If the current data stream is a newly added aperiodic time-sensitive stream, define k as the current data stream. This indicates the time when the current data stream arrives at the current node. This represents the latest completion time of the current data stream k on the current node. This represents the length of the time slot occupied by the current data stream k on the current node, with the time slot starting offset as . The range of values ​​is ;

[0115] Step S302: Calculation In the infeasible region of the solution space, the computation process includes steps S3021~S3022:

[0116] Step S3021: For any data stream y already deployed on the current node, if any data stream y is a periodic time-sensitive stream, define... , ,in This indicates the floor function. For any integer c in A, calculate , , , ;

[0117] if ,but This is an infeasible region.

[0118] like , and The region is infeasible; where ceil represents rounding up.

[0119] Step S3022: If y is non-periodic, then This refers to the infeasible region;

[0120] Step S303: After obtaining the infeasible region, combine... The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0121] In some embodiments, in step S107, the subsequent traversal link nodes are simulated and completed based on the Monte Carlo tree search simulation, and the actual revenue value is calculated, such as... Figure 3 As shown, this includes: for the current data stream at the current node z, using Monte Carlo tree search, randomly selecting candidate nodes based on affinity, and simulating the selection of candidate nodes. The subsequent walking path is constructed using methods such as random Rollout, combined with maximum affinity sampling, combined with multinomial sampling, and combined with greedy algorithm sampling. The average benefit of each method is calculated and used as the actual benefit value before being transmitted back.

[0122] In some embodiments, in step S108, constructing a loss function based on the predicted value and the actual revenue value, and updating the parameters of the initial actor network and the initial critic network, includes:

[0123] Suppose that the current data stream k's traversal path expands from the current node z to the candidate node. The probability is The current data stream k travels to z and At that time, the service status of the current node is Candidate nodes The service status is ;make Represents a value function; Indicates the discount factor; This indicates that the current data stream k in the traversal path expands from z to... The benefits;

[0124] The loss function expression for the actor network is:

[0125] ;

[0126] The update rules are as follows:

[0127] ;

[0128] in, This represents the set of learnable parameters of the actor network. For the learning rate of the actor network, As of the end The walking path, where N represents the path The length.

[0129] The fifth neural network is used as the value function for the critic network to predict the actor network. The loss function of the critic network is:

[0130] ;

[0131] The update rules are as follows:

[0132] ;

[0133] in, This represents the set of learnable parameters of the critic network. The learning rate of the commentator network.

[0134] The actor network obtained based on the above training content is directly transferred to the application scenario of this application to calculate the probability of selecting each candidate node as the next link node in the current data traffic of the current node, and sample to determine the next roaming link node.

[0135] Specifically, the present invention also includes a data flow scheduling method for smart substations based on TSN, comprising the following steps S401~S406:

[0136] Step S401: Obtain the link topology and link status parameters of the smart substation. The link status parameters include the bandwidth and utilization of each link, the first data stream parameters of the existing periodic time-sensitive flow and the existing non-periodic time-sensitive flow that are occupied in each link; the first data stream parameters include the period, the link occupation duration, the data volume, and the occupation time slot range.

[0137] Step S402: Obtain the second data stream parameters of the newly added periodic time-sensitive stream. The second data stream parameters include the period of each newly added periodic time-sensitive stream, the generation time within each period, the transmission deadline, the data size, the source address, and the destination address.

[0138] Step S403: Obtain the third data stream parameters of the newly added aperiodic time-sensitive stream. The third data stream parameters include the generation time, transmission deadline, data size, source address, and destination address of each newly added aperiodic time-sensitive stream.

[0139] Step S404: Perform equivalent reconstruction of the link topology and link state parameters, using the edges in the original link topology as nodes in the new graph, and the connection relationships between edges in the original link topology as edges in the new graph.

[0140] Step S405: Use the second data stream parameters of each newly added periodic time-sensitive stream, the third data stream parameters of each newly added non-periodic time-sensitive stream, and the new graph as state parameters in the current node.

[0141] Step S406: Input the state parameters into the intelligent substation data flow scheduling model obtained in the training method of the intelligent substation data flow scheduling model based on TSN described in steps S101 to S108 above to select the next roaming link node, allocate time slots for the current data flow according to preset rules, and update the state parameters; repeat the process until the target node position is reached and terminated.

[0142] In some embodiments, time slots are allocated to the current data stream according to preset rules, including steps S501 to S503:

[0143] Step S501: If the current data stream is a newly added periodic time-sensitive stream, define k as the current data stream. This indicates the time when the current data stream arrives at the current node. This represents the latest completion time of the current data stream k on the current node. This represents the length of the time slot occupied by the current data stream k on the current node. This indicates the period of the current data stream k; the offset at the start of the time slot is... The range of values ​​is .

[0144] Step S502: Calculation In the infeasible region of the solution space, the computation process includes steps S5021~S5022:

[0145] Step S5021: For any data stream y already deployed on the current node, if any data stream y is a periodic time-sensitive stream, calculate the period of the current data stream k. With the period of any data stream y The least common multiple S is defined as follows: , If any data stream y is an aperiodic time-sensitive stream, .

[0146] Step S5022: For any integer c in A, calculate , , , ;

[0147] if ,but This is an infeasible region.

[0148] like , and The region is infeasible; where ceil represents rounding up.

[0149] Step S503: After obtaining the infeasible region, combine... The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0150] In some embodiments, time slots are allocated to the current data stream according to preset rules, including steps S601 to S603:

[0151] Step S601: If the current data stream is a newly added aperiodic time-sensitive stream, define k as the current data stream. This indicates the time when the current data stream arrives at the current node. This represents the latest completion time of the current data stream k on the current node. This represents the length of the time slot occupied by the current data stream k on the current node, with the time slot starting offset as . The range of values ​​is ;

[0152] Step S602: Calculation In the infeasible region of the solution space, the computation process includes steps S6021~S6022:

[0153] Step S6021: For any data stream y already deployed on the current node, if any data stream y is a periodic time-sensitive stream, define... , ,in This indicates the floor function. For any integer c in A, calculate , , , ;

[0154] if ,but This is an infeasible region.

[0155] like , and The region is infeasible; where ceil represents rounding up.

[0156] Step S6022: If y is non-periodic, then This is an infeasible region;

[0157] Step S603: After obtaining the infeasible region, combine... The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

[0158] Therefore, the scheduling logic can be represented as: State 1 → Affinity 1 → Sample the next node → Update State 2 → Affinity 2 → Sample the next node → Update State 3 → Affinity 3 → ... → Terminate upon reaching the target node position.

[0159] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the above-described method.

[0160] On the other hand, the present invention also provides a computer program product, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the above-described method.

[0161] The present invention will now be described with reference to a specific embodiment:

[0162] This embodiment discloses a smart substation flow scheduling mechanism, which includes:

[0163] Step S1: Obtain the link topology of the smart substation, as well as the bandwidth, utilization rate, and relevant information of the PTS and STS already occupied on each link. The relevant information for PTS and STS includes the cycle time, link occupancy duration, data volume, and occupied time slot range.

[0164] Step S2: Obtain information related to newly added PTS traffic data in the smart substation, including the period of each PTS traffic, the generation time within each period, the transmission deadline (ddl), the data size, the source address, and the destination address (the starting transmission node and the destination receiving node of the traffic).

[0165] Step S3: Obtain information related to newly added STS traffic in the smart substation, including the generation time of the STS traffic, transmission deadline (ddl), data size, source address, and destination address (the starting transmission node and the destination receiving node of the traffic).

[0166] Step S4: Based on the invented traffic scheduling method and the obtained network information and traffic information, allocate network transmission time slots on each link for each data stream.

[0167] Step S5: The intelligent substation network strictly follows the scheduling method to transmit the corresponding data to the respective receiving nodes according to the time slots allocated to each flow, ensuring that the transmission delay and jitter of each data flow are controllable.

[0168] Furthermore, this embodiment provides a smart substation flow scheduling method, which includes the following steps S11~S18:

[0169] Step S11: In the link connection topology of the smart substation network, each edge contains information about the corresponding link, including bandwidth, utilization, and the STS and PTS traffic information that has already occupied the link's time slot. This topology is then equivalently reconstructed into a new graph. The reconstruction idea is that all edges in the original topology graph are represented by nodes in the new graph, and the connection relationships between edges in the original topology graph are represented by edges in the new graph.

[0170] Step S12: Construct a reinforcement learning-based neural network model to mine important feature information of the graph and feature information of the data stream, and serialize and allocate transmission time slots on each link for each stream. This embodiment uses the actor network in the actor critic model to perform action decisions. This neural network model consists of two parts: neural network module A and neural network module B. A and B can be implemented using, but are not limited to, existing Transformers. A is used to mine feature information on the new graph that has undergone equivalent reconstruction, and B is used to mine feature information for each data volume. After processing by A, the feature information of each node on the new graph can be represented as a feature tensor H (H is...). A 3D tensor, where N represents the number of nodes in the node graph, and d represents the node features. (Dimension of B). After processing by B, the data stream is represented as a tensor F (F is the dimension of B). A dimensional tensor, where M represents the total number of data streams in the system, and d is the feature vector of the data stream. (dimensions).

[0171] Step S13: After obtaining the link feature H and data flow feature F, calculate the affinity between the link and the data flow. Then, use the feature vector of data flow k... The previous transmission link node eigenvectors Current link node eigenvectors Global salient features of the new graph Concatenation as a question vector ,Right now .use Represents nodes Connected but not including data stream The set of nodes that have already been visited. Data flow. With link node affinity The following can be calculated:

[0172] ;

[0173] ;

[0174] in, and Representing neural network operations, which can be implemented using, but not limited to, multilayer perceptrons, with the aim of... and Unify to the same dimension and further mine node features; Represents data stream Wander to link node The link node that is connected later Affinity.

[0175] Step S14: Data stream k traverses to the link node At that time, the set of affinity with other associated link nodes is represented as , .based on Based on Monte Carlo tree search, the data flow k can select the next traversal node through various methods such as multinomial sampling and maximum probability sampling. Travel to the link node Then calculate and based on Select the next traversal node. This process is repeated until the data stream k traverses to the specific target node, and the traversal path is completed.

[0176] Step S15: Data Flow For periodic data streams, execute step S16; for non-periodic data streams, execute step S17.

[0177] Step S16: Based on the planned traversal path, allocate transmission time slots on each link node along the traversal path for the periodic data stream k. Specifically, this includes...

[0178] Let k be the data stream to be deployed on the current link node, using This indicates the time when data stream k arrives at this link node. This indicates the latest completion time of data stream k on this link node. This indicates the length of the time slot occupied by data stream k on this link node. This represents the period of data stream k, with the time slot starting offset being... The range of values ​​is Next, the infeasible solutions within this range are calculated, which involves two steps:

[0179] Step 1: For any data stream y already deployed on this link node, if y is a periodic data stream, calculate the period of data stream k. Period of data stream y The least common multiple (i.e., superperiod) is S, and is defined as follows: , If y is a non-periodic data stream, .

[0180] Step 2: For any integer c in A, calculate... , , , ;if ,but It is an infeasible region; if , and The region is infeasible; where ceil represents rounding up.

[0181] After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot.

[0182] Step 17: Based on the planned traversal path, allocate transmission time slots to each link node along the traversal path for the aperiodic data stream k. Specifically, let k represent the aperiodic data stream to be deployed at the current node. Representing data streams respectively The arrival time of the link node, the latest completion time of data stream k on that link node, the time slot length occupied by data stream k on that link node, and the time slot start offset are all considered. The range of values ​​is . The infeasible regions in the solution space are calculated according to the following steps:

[0183] For a data stream y already deployed on this link node, if y is periodic, define... , ,in This indicates the floor function. For any integer c in A, calculate , , , ;

[0184] if ,but This is an infeasible region;

[0185] like , and The region is infeasible; where ceil represents rounding up.

[0186] If y is non-periodic, then This is an infeasible region.

[0187] After calculating the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within this search range until a continuous sequence is found. One idle time slot, that is A feasible solution.

[0188] Step S18: On the traversal path of data flow k, the links of each node... Together, they determined the transmission time slots occupied by data stream k on each link.

[0189] In steps S12 and S13, the actor network and the critic network are pre-trained together, and the Actor-Critic policy gradient learning method is used. The probability of the walk path expanding from Z to Z' is set as follows: When data stream k travels to Z and Z', the service states of the link nodes are respectively and . Represents a value function; Indicates the discount factor; This represents the gain from extending the walking path from Z to Z'.

[0190] The loss function expression for the actor network is:

[0191] ;

[0192] The update rules are as follows:

[0193] ;

[0194] in, This represents the set of learnable parameters of the actor network. For the learning rate of the actor network, As of the end The walking path, where N represents the path The length.

[0195] The loss function of the commentator network is:

[0196] ;

[0197] The update rules are as follows:

[0198] ;

[0199] in, This represents the set of learnable parameters of the critic network. The learning rate of the commentator network.

[0200] In step S14, as Figure 3 As shown, when the data stream When the traversal path is planned to link node Z, link node Z' is selected for expansion. At this point, Monte Carlo tree search is used to simulate and sample the traversal path after Z'. During the simulation, there are several possible methods for constructing the traversal path, including random Rollout, combined with maximum affinity sampling, combined with multinomial sampling, and combined with greedy algorithm sampling. Each simulation generates a corresponding traversal path and provides timely benefits. Then, the average reward of each path is calculated and sent back to the expansion node Z' as the reward from Z to Z'. Real-time reward. Refers to data stream The transmission delay generated by the current traversal path is determined by the substation network system assessment.

[0201] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0202] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0203] In summary, the TSN-based intelligent substation data flow scheduling model training method, scheduling method, and apparatus of this invention reconstruct a new graph equivalent to the link topology and link states of the intelligent substation. It constructs a state space by combining parameters of newly added periodic time-sensitive flows and newly added aperiodic time-sensitive flows. An actor network from the actor-critic network is introduced to calculate and select candidate nodes for the next link, constructing the selection probability as an affinity set. The subsequent wandering link nodes are simulated and completed using Monte Carlo tree search, and the actual benefit value is calculated, updating the actor network and the critic network. Based on the trained actor network, the next wandering link node is selected in the application, and time slots are allocated to update the state. This process is repeated until the target node position is reached. The equivalent reconstruction of the new graph can more effectively utilize network resources, avoid congestion and bottlenecks, and improve overall network performance. The introduction of the actor-critic network allows for rapid calculation and updating of scheduling strategies, adapting to real-time changing network environments. The calculation of actual benefit values ​​based on Monte Carlo tree search allows for dynamic adjustment of the search strategy according to real-time feedback, demonstrating strong adaptability and the ability to find a better solution within a limited time.

[0204] Furthermore, time slot allocation schemes are constructed for newly added periodic time-sensitive flows and newly added aperiodic time-sensitive flows respectively, which can ensure the deterministic transmission requirements of heterogeneous time-sensitive data in the TSN network and improve the overall network stability.

[0205] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0206] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0207] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0208] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a data flow scheduling model of a smart substation based on TSN, characterized in that, The method includes the following steps: The link topology and link status parameters of the smart substation are obtained. The link status parameters include the bandwidth and utilization of each link, and the first data stream parameters of the existing periodic time-sensitive flow and the existing non-periodic time-sensitive flow that are occupied in each link. The first data stream parameters include the period, the link occupation duration, the data volume, and the occupation time slot range. Obtain the second data stream parameters of the newly added periodic time-sensitive stream. The second data stream parameters include the period of each newly added periodic time-sensitive stream, the generation time within each period, the transmission deadline, the data size, the source address, and the destination address. Obtain the third data stream parameters of the newly added aperiodic time-sensitive stream. The third data stream parameters include the generation time, transmission deadline, data size, source address, and destination address of each newly added aperiodic time-sensitive stream. The link topology and link state parameters are reconstructed equivalently, and the edges in the original link topology are used as nodes in the new graph, and the connection relationships between edges in the original link topology are used as edges in the new graph. The second data stream parameters of each newly added periodic time-sensitive stream in the current node, the third data stream parameters of each newly added non-periodic time-sensitive stream, and the new graph are used as state parameters, and the set of candidate nodes for the next link connected to the current node is used as action options. An initial actor network and an initial critic network are obtained. The actor network includes a first neural network module for mining feature information of the new graph and a second neural network module for mining feature information of the current data stream. The first neural network module extracts the feature vector of the previous transmission link node, the feature vector of the current node, and the global salient features of the new graph. The second neural network module extracts the feature vector of the current data stream and concatenates them to obtain a challenge vector. The challenge vector is passed through a third neural network module, and the feature vector of the current node is passed through a fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalized to obtain the selection probability as the affinity between the current data stream and the candidate nodes, and an affinity set is constructed. Based on the affinity set between the current data stream and each candidate node, the next walking link node is selected according to the probability, and the state parameters are updated. The initial critic network is used to calculate the prediction value when selecting the corresponding candidate node. Based on the selected next traversal node, the Monte Carlo tree search is used to simulate and complete the subsequent traversal nodes, and the actual revenue value is calculated. Based on the predicted value and the actual revenue value, a loss function is constructed to update the parameters of the initial actor network and the initial critic network, completing one round of iteration; multiple rounds of iteration are performed according to preset conditions, and the updated initial critic network is used as the data flow scheduling model for the smart substation.

2. The training method for a data flow scheduling model of a smart substation based on TSN according to claim 1, characterized in that, Before calculating the actual profit value, the following is also included: If the current data stream is the newly added periodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node. This indicates the period of the current data stream k; the time slot start offset is... The range of values ​​is ; calculate In the infeasible region of the solution space, the computation process includes: For any data stream y already deployed on the current node, if the data stream y is a periodic time-sensitive stream, calculate the period of the current data stream k. With the period of the arbitrary data stream y The least common multiple S is defined as follows: , If the arbitrary data stream y is an aperiodic time-sensitive stream, ; For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up; After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

3. The training method for a data flow scheduling model of a smart substation based on TSN according to claim 2, characterized in that, Before calculating the actual profit value, the following is also included: If the current data stream is the newly added aperiodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node, with the time slot starting offset being... The range of values ​​is ; calculate In the infeasible region of the solution space, the computation process includes: For any data stream y already deployed on the current node, if the arbitrary data stream y is a periodic time-sensitive stream, define... , ,in This indicates the floor function. For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up; If y is non-periodic, then This refers to the infeasible region; After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

4. The training method for a data flow scheduling model of a smart substation based on TSN according to claim 1, characterized in that, The first and second neural network modules adopt the Transformer model, while the third and fourth neural network modules adopt the multilayer perceptron model.

5. The training method for a data flow scheduling model of a smart substation based on TSN according to claim 1, characterized in that, The challenge vector is passed through the third neural network module, and the current node feature vector is passed through the fourth neural network module. After unifying the dimensions, correlation calculation is performed and normalized to obtain the selection probability as the affinity between the current data stream and the candidate node. The calculation steps are as follows: The feature vector of the current data stream k Previous transmission link node eigenvectors The current node eigenvectors The global salient features of the new graph Concatenation as a question vector ,Right now ; definition This represents the set of candidate nodes, which is related to the current node. The set of nodes that are connected, but not included, that the current data stream k has already traversed; The current data stream k and the candidate nodes affinity The calculation formula is: ； ； in, This represents the function corresponding to the third neural network. The function corresponding to the fourth neural network is used to... and Unify to the same dimension and further mine node features; This indicates that the current data stream k has traveled to the current node. The affinity with the candidate node u.

6. The training method for a data flow scheduling model of a smart substation based on TSN according to claim 1, characterized in that, Based on Monte Carlo tree search simulation, subsequent traversal links are completed and the actual revenue is calculated, including: For the current data stream at the current node z, candidate nodes are randomly selected based on affinity using the Monte Carlo tree search, and the candidate node selection is simulated. The subsequent walking path, during the simulation process, is constructed using methods including random Rollout, combined with maximum affinity sampling, combined with multinomial sampling, and combined with greedy algorithm sampling. The average benefit of each method is calculated and used as the actual benefit value before being transmitted back.

7. The training method for a data flow scheduling model of a smart substation based on TSN according to claim 6, characterized in that, Based on the predicted value and the actual revenue value, a loss function is constructed to update the parameters of the initial actor network and the initial critic network, including: Suppose that the current data stream k's traversal path extends from the current node z to the candidate node. The probability is The current data stream k travels to z and At that time, the service status of the current node is The candidate nodes The service status is ;make Represents a value function; Indicates the discount factor; The current data stream k in the traversal path extends from z to... The benefits; The loss function expression for the actor network is then: ； The update rules are as follows: ； in, This represents the set of learnable parameters of the actor network. The learning rate of the actor network. As of the end The walking path, where N represents the path Length; A fifth neural network is used as the value function for the critic network to predict the actor network, and the loss function of the critic network is: ； The update rules are as follows: ； in, This represents the set of learnable parameters of the critic network. is the learning rate of the critic network.

8. A data stream scheduling method for intelligent substations based on TSN, characterized in that, Includes the following steps: The link topology and link status parameters of the smart substation are obtained. The link status parameters include the bandwidth and utilization of each link, and the first data stream parameters of the existing periodic time-sensitive flow and the existing non-periodic time-sensitive flow that are occupied in each link. The first data stream parameters include the period, the link occupation duration, the data volume, and the occupation time slot range. Obtain the second data stream parameters of the newly added periodic time-sensitive stream. The second data stream parameters include the period of each newly added periodic time-sensitive stream, the generation time within each period, the transmission deadline, the data size, the source address, and the destination address. Obtain the third data stream parameters of the newly added aperiodic time-sensitive stream. The third data stream parameters include the generation time, transmission deadline, data size, source address, and destination address of each newly added aperiodic time-sensitive stream. The link topology and link state parameters are reconstructed equivalently, and the edges in the original link topology are used as nodes in the new graph, and the connection relationships between edges in the original link topology are used as edges in the new graph. The second data stream parameters of each newly added periodic time-sensitive stream in the current node, the third data stream parameters of each newly added non-periodic time-sensitive stream, and the new graph are used as state parameters; The state parameters are input into the intelligent substation data flow scheduling model obtained in the training method of the intelligent substation data flow scheduling model based on TSN according to any one of claims 1 to 7 to select the next roaming link node, allocate time slots for the current data flow according to preset rules, and update the state parameters; repeat the process until the target node position is reached and terminated.

9. The data stream scheduling method for intelligent substations based on TSN according to claim 8, characterized in that, Allocating time slots to the current data stream according to preset rules includes: If the current data stream is the newly added periodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node. This indicates the period of the current data stream k; the time slot start offset is... The range of values ​​is ; calculate In the infeasible region of the solution space, the computation process includes: For any data stream y already deployed on the current node, if the data stream y is a periodic time-sensitive stream, calculate the period of the current data stream k. With the period of the arbitrary data stream y The least common multiple S is defined as follows: , If the arbitrary data stream y is an aperiodic time-sensitive stream, ; For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up; After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

10. The data stream scheduling method for intelligent substations based on TSN according to claim 8, characterized in that, Allocating time slots to the current data stream according to preset rules includes: If the current data stream is the newly added aperiodic time-sensitive stream, then k is defined as the current data stream. This indicates the time when the current data stream arrives at the current node. This indicates the latest completion time of the current data stream k at the current node. This indicates the length of the time slot occupied by the current data stream k on the current node, with the time slot starting offset being... The range of values ​​is ; calculate In the infeasible region of the solution space, the computation process includes: For any data stream y already deployed on the current node, if the arbitrary data stream y is a periodic time-sensitive stream, define... , ,in This indicates the floor function. For any integer c in A, calculate , , , ;if ,but Let this be the infeasible region; if , and Let be the infeasible region; where ceil represents rounding up; If y is non-periodic, then This refers to the infeasible region; After obtaining the infeasible region, combine The range of values Shrink The search range is defined, and then the search is exhaustively performed within the feasible search range until a continuous sequence is found. A free time slot is allocated to the current data stream.

11. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 7.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images