Ring main unit bus switching path optimization method based on deep reinforcement learning

Abstract

This invention discloses a method for optimizing bus switching paths in ring network boxes based on deep reinforcement learning, comprising the following steps: S1, collecting real-time operating data of all ring network boxes in the power distribution system; S2, constructing a graph structure model of the power distribution system; S3, defining the reinforcement learning state space; S4, defining the action space as the set of all executable bus switching operations; S5, constructing a graph attention-guided dual-strategy deep reinforcement learning model; S6, constructing a multi-objective reinforcement learning reward function, and forming a path reward value through weighted combination; S7, training the model based on an Actor-Critic structure, updating the value assessment using time difference error, and introducing an experience replay mechanism; S8, inputting the current power distribution state into the trained model to generate the optimal bus switching path that satisfies the constraints; S9, executing the path action sequence sequentially. This invention integrates graph modeling and dual-strategy learning to achieve intelligent optimization and safe control of bus switching paths in ring network boxes.

Description

Technical Field

[0001] This invention relates to the field of intelligent dispatching and electrical automation control technology for power distribution systems, and in particular to a method for optimizing the switching path of ring network busbars based on deep reinforcement learning. Background Technology

[0002] In modern power distribution systems, ring main units (RMS) are widely deployed in urban power distribution networks and industrial park distributed power supply architectures as key equipment for achieving power supply reliability and fault isolation capabilities. RMS structures are characterized by strong inter-node connectivity and high power supply path redundancy, enabling rapid switching and recovery in the event of a fault. However, in actual operation, the planning and execution of bus switching paths still heavily rely on manual experience, rule templates, or shortest path algorithms based on static models, making it difficult to meet the real-time, adaptive, and multi-objective control requirements in dynamic environments.

[0003] Existing methods for generating bus switching paths mostly employ heuristic search, topology reconstruction rule bases, or linear optimization methods to screen and evaluate switching paths. While these methods are feasible in some stable scenarios, they have significant limitations: First, traditional rule bases cannot handle complex conditions such as multi-node failures, equipment malfunctions, or large-scale load disturbances, and lack generalization ability; second, heuristic path search methods typically only optimize path length or number of operation steps, making it difficult to comprehensively consider multi-dimensional indicators such as voltage stability, load balancing, and connectivity restoration efficiency, leading to local optima or even infeasible solutions in practical applications; third, lacking online learning and historical experience feedback mechanisms, current methods cannot update strategies based on grid structure evolution and load distribution changes, resulting in delayed system control response and increased operational risks.

[0004] To address the aforementioned issues, this invention proposes a deep reinforcement learning-based method for optimizing bus switching paths in ring network enclosures. By constructing a graph structure model and introducing a dual-strategy deep reinforcement learning architecture guided by graph attention, intelligent generation and dynamic optimization of bus switching paths are achieved. This method not only models the complex dependencies between electrical topology and state information but also adaptively learns optimal path strategies for different scenarios in a high-dimensional state space, significantly improving the global optimization capability of switching paths. Furthermore, by combining a safety assessment network guidance mechanism with a multi-objective reinforcement learning reward function design, this method effectively solves the core defects of traditional path optimization models, such as single objective, poor security, and unstable training, exhibiting stronger engineering adaptability and decision-making intelligence. Through training in conjunction with historical data and simulation samples, this method also supports incremental learning and continuous strategy optimization, meeting the requirements of power distribution systems for rapid generation, evaluation, and high-reliability execution of bus switching paths under different time periods, load structures, and fault scenarios.

[0005] Therefore, how to provide a method for optimizing the switching path of the ring network box bus based on deep reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a method for optimizing the switching path of ring network busbars based on deep reinforcement learning. This invention integrates graph structure modeling and a dual-strategy deep reinforcement learning mechanism to construct a path optimization model with structure awareness and safety assessment capabilities. This enables intelligent generation, dynamic adjustment, and multi-objective collaborative optimization of the switching path of the ring network busbar, improving the global optimality, feasibility, and response efficiency of the switching strategy, and ensuring the safe and stable operation of the power distribution system.

[0007] The ring network bus switching path optimization method based on deep reinforcement learning according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect real-time operating data of all ring network boxes in the power distribution system. The data includes the voltage amplitude, current direction, active load, reactive load, fault status, switch closing status and network topology of each node.

[0009] S2. Construct a graph structure model of the power distribution system. Model each ring network box bus, load node, and branch switch as nodes in the graph structure. Model the physical connection lines as edges with direction and weights. The weights include line impedance, current capacity, and voltage level information.

[0010] S3. Based on the constructed graph structure model and the running state information, define the state space of reinforcement learning. The state includes node electrical feature vectors, edge attribute matrices, network connectivity matrices, candidate switching targets, and fault isolation requirements.

[0011] S4. Define the action space as the set of all executable bus switching operations. The operation includes the closing or opening behavior of a single switch in the ring network box. Each complete switching path consists of multiple atomic actions, forming a path action sequence.

[0012] S5. Construct a graph attention-guided dual-strategy deep reinforcement learning model, the model including: a graph attention encoder, a path policy network, and a security evaluation network;

[0013] S6. Construct a multi-objective reinforcement learning reward function, wherein the reward function takes path steps, load transfer cost, operation time delay, voltage offset index and path risk level as inputs, and forms the final path reward value through weighted combination.

[0014] S7. Using historical bus switching data and the constructed simulated power distribution scenario, a strategy optimization method based on the Actor-Critic structure is adopted to jointly train the dual-strategy network. Time difference error is used for value evaluation and update, and an experience replay mechanism is used to improve training stability until the strategy network converges.

[0015] S8. During actual operation, the current power distribution system status is input into the dual-strategy deep reinforcement learning model after training convergence, and the bus switching path action sequence that satisfies electrical safety constraints and the optimal objective function is dynamically generated.

[0016] S9. Execute the actual control operations of each target switch in the ring network box in sequence according to the bus switching path action sequence to realize bus switching operations for fault isolation, load transfer or reconfiguration of power supply path, and complete the intelligent reconfiguration of the power distribution system.

[0017] Optionally, the real-time operating data of the ring network box includes the voltage amplitude, current direction, active load, reactive load, fault status, switch position status, and network topology of each node.

[0018] Optionally, S2 specifically includes:

[0019] S21. Analyze the structural composition and electrical connection relationship of all ring network boxes in the power distribution system, extract bus nodes, load nodes, feeder nodes, switch nodes and intermediate tie nodes, and assign a unique node number and node type code to each type of node.

[0020] S22. Construct the extracted nodes into a set of nodes in a graph structure model. Each node in the set records its physical coordinates, the list of connected superior or subordinate nodes, the area identifier, the power supply zone it belongs to, and the initialization attributes of its node electrical status.

[0021] S23. Model the connection relationship of all physical lines in the power distribution system and generate the edge set in the graph structure model. Each edge in the edge set consists of a starting node and a target node, and is accompanied by edge direction attribute, connection type identifier and unique edge number.

[0022] S24. Assign line attribute information to each edge in the edge set, including line impedance estimate, rated current capacity, voltage level label, channel affiliation, whether it is a cross-regional connection, and power supply path offset coefficient.

[0023] S25. Construct a directed graph structure representation based on the node set and edge set, and map the connection relationship between nodes and edges into an adjacency matrix and a direction mask matrix. The adjacency matrix is ​​used to represent the structural connection strength between nodes, and the direction mask matrix is ​​used to identify the feasible channels of directed edges in path search.

[0024] S26. The directed graph structure model is hierarchically classified according to node type to form a structure partitioning graph, and stored as a data structure that can be parsed by a graph neural network. The structure is used as the basis for structure-aware input in the subsequent state representation construction and reinforcement learning path strategy generation process.

[0025] Optionally, S3 specifically includes:

[0026] S31. Extract the current electrical operating parameters of all nodes from the directed graph structure model, and construct the node electrical state feature vector. The feature vector includes voltage amplitude, current direction, active power, reactive power, frequency offset, node type identifier, equipment operating status identifier and region code.

[0027] S32. Extract the connection attributes of each edge in the directed graph structure and construct an edge attribute description matrix. The attributes include the impedance level, voltage level, switch linkage status, current on / off status, line load rate, connection directionality mark and operable status label of the connection line.

[0028] S33. Construct a network connectivity matrix based on the node set and edge set. The connectivity matrix is ​​used to record the reachability relationship and power outage status between each node in the graph, and to identify the integrity and validity of the power supply path under the current topology.

[0029] S34. Identify faulty nodes, isolated nodes, and redundant power supply nodes based on the current operating status and fault diagnosis results, and filter out all candidate operation nodes and target recovery areas that can perform switching tasks based on the topology diagram.

[0030] S35. Integrate the node electrical state feature vector, edge attribute description matrix, network connectivity matrix, fault identification vector and candidate operation set to construct a high-dimensional state input tensor, which is used to characterize the structure and operating state of the power distribution system at the current moment;

[0031] S36. Perform dimension alignment, normalization, and missing value repair on the high-dimensional state input tensor to ensure the integrity of the data input, and use it as the state space input of the deep reinforcement learning model to drive the policy generation and path decision-making process.

[0032] Optionally, S4 specifically includes:

[0033] S41. Based on the directed graph structure model and electrical operating status, identify all switch nodes in a controllable state and filter out the target switch set that has operating authority and is not in a fault isolation state.

[0034] S42. Establish a corresponding set of atomic actions for each target switch. The atomic actions include the closing or opening operation of a single switch, and mark the direction of action, changes in electrical connection and the range of the affected area.

[0035] S43. Based on the current topology and candidate target power supply areas, construct all possible combinations of bus switching paths. The path is an action sequence formed by combining multiple atomic actions in the order of operation. Each action sequence satisfies the topology reachability requirements of the starting point and the ending point.

[0036] S44. Perform path validity verification on all candidate path action sequences, eliminate unexecutable paths caused by loop conflicts, path duplication, physical interlocks and isolation restrictions, and retain the set of valid action sequences that meet the basic operational constraints.

[0037] S45. Construct an action space representation structure for each valid action sequence, and clarify the number of operation steps, the target switch number of each step, the expected action direction, the expected influence area number, and the operation dependencies between each step.

[0038] S46. The action space structure is used as the action input of the dual-policy deep reinforcement learning model, which is used by the path policy network for policy sampling and path selection during training and inference.

[0039] Optionally, S5 specifically includes:

[0040] S51. Construct a graph attention encoder module. The input consists of node feature vectors, edge attribute matrices, and adjacency structure indices from the power distribution system graph structure model. A multi-channel attention mechanism is used to aggregate adjacency context information for each node, generating a node embedding representation with topology awareness. The embedding representation H of node i... i Defined as:

[0041]

[0042] Where, α ij Let X be the attention weight of node i to its neighboring node j, W be the node transformation weight matrix, and X be the attention weight of node i to its neighboring node j. j Let j be the feature vector of node j;

[0043] S52. The graph embedding state representation is input into the path policy generation network, which includes a state projection layer, an action scoring layer, and a candidate selection layer. The state vector is compressed and reconstructed using deep feature encoding. The embedding features of each node in the path are extracted, and the policy score of the path action sequence is calculated, defined as follows:

[0044]

[0045] Where, τ k Let H be the k-th candidate path, T be the number of atomic actions contained in this path, and H be the number of atomic actions contained in this path. ntLet U be the embedding vector of the t-th node in the path, and let U, V, and b be learnable parameters. Finally, output the currently executable combination of path actions and their priority order.

[0046] S53. Simultaneously construct a security assessment network, receiving the same state inputs and action sequence outputs as the path policy generation network. Independently quantify and model indicators such as path voltage stability, load transfer balance, and path connectivity, and generate security label vectors. These vectors are then fused with the policy score to obtain the final comprehensive score of the path.

[0047] S(τ k )=λ1·Softmax(Q(τ k ))+λ2·(1-||R k ||2)

[0048] Where λ1 and λ2 are the policy confidence and risk suppression coefficients, respectively, and R k Let ||R| be the multidimensional risk vector for path k. k ||2 represents its second norm;

[0049] S54. Establish a cross-guidance mechanism between the path policy generation network and the security assessment network, and jointly fuse the action scores of the policy network and the path labels of the security network to form a final path action candidate set. The candidate set is sorted according to the policy distribution weight and security confidence, and path sequences that do not meet the continuity constraints or have topological conflicts are eliminated.

[0050] S55. To achieve multi-task path evaluation training, the graph attention encoder, path policy generation network, and security evaluation network are constructed into a parallel fusion structure. A unified input interface and joint output structure are defined to complete the construction of the dual-policy collaborative scheduling structure. The structure serves as a complete graph attention-guided dual-policy deep reinforcement learning model framework for the optimal policy generation and path selection process of bus switching paths.

[0051] Optionally, S6 specifically includes:

[0052] S61. Determine the multi-dimensional evaluation objectives for bus switching path optimization and construct a reward evaluation index system, including seven basic evaluation dimensions: number of path switching steps, action execution time, node voltage deviation, load power deviation, electrical connectivity restoration speed, power outage coverage, and path risk level.

[0053] S62. For each candidate path action sequence, execute its complete switching process in the form of simulation in the reinforcement learning training framework, and collect in real time the topology changes, key node status changes, fault isolation success rate, load transfer status stability and the impact range of each step of the operation during the switching execution process.

[0054] S63. Based on the collected switching execution data, calculate the individual scores for each reward evaluation dimension. The individual scores are constructed using normalized mapping and constraint boundary projection to ensure that different dimension indicators can be compared and integrated on the same scale.

[0055] S64. Assign initial weight parameters to each reward dimension, construct a weighted aggregation function and form a multi-objective combined reward output. The weighting function supports dynamic adjustment of weight ratios during training based on the risk level, policy confidence and convergence speed of the current model output policy, thereby realizing a balance and control mechanism between efficiency and safety.

[0056] S65. Introduce a reward sensitivity adjustment mechanism to perform attribution analysis on the feedback results of each round of path actions, determine the response strength of the reward function to high-risk operations, path oscillation behavior and topology conflict events, and automatically adjust the harmonic coefficient between target items to adapt to different load scenarios and topology evolution conditions.

[0057] S66. The resulting multi-objective reinforcement learning reward function is integrated into the dual-policy deep reinforcement learning training process as the basis for constructing the joint loss of the path policy network and the safety assessment network. This provides feedback direction for policy updates and drives the policy network to iteratively converge toward the optimal solution space that simultaneously satisfies switching efficiency, electrical safety, and structural coherence.

[0058] Optionally, S7 specifically includes:

[0059] S71. Construct an offline training environment. Based on the actual topology of the power distribution system, historical operating data, and typical fault conditions, generate a simulation training sample set covering multiple operating scenarios. The sample set includes node state sequences, operable action sequences, and path execution result labels.

[0060] S72. Initialize the graph attention-guided dual-strategy deep reinforcement learning model structure, which includes three parts: graph attention encoder, path policy generation network and safety evaluation network, and set the initial parameters, number of training rounds, learning rate, batch size and update frequency for each module.

[0061] S73. A joint policy optimization method based on the Actor-Critic framework is adopted, and the path policy network and the safety assessment network are trained synchronously. The path policy network is responsible for generating path action policies, and the safety assessment network is responsible for generating path execution feasibility feedback. The two are updated with parameters through a joint loss function during training.

[0062] S74. Introduce a time difference error mechanism to calculate the cumulative evaluation deviation of the path action sequence under different state transitions, which serves as the basis for updating the value evaluation network in policy optimization, thereby improving the model's ability to judge global returns during the path selection process.

[0063] S75. Configure an experience replay mechanism to record historical state transition sequences, path execution behavior and reward feedback results, build an efficient training sample pool, and extract unseen combined states in batches from it for model training to prevent overfitting and improve policy generalization ability.

[0064] S76. Set the stability judgment criteria for the policy, continuously monitor the path convergence speed, reward improvement trend and action sequence fluctuation range during the training process, and judge whether the policy network has converged to a stable output state. If the preset stopping condition is met, the training process is ended and the model parameters are fixed.

[0065] Optionally, S8 specifically includes:

[0066] S81. Obtain all operating data of the current power distribution system, and update the graph structure model, node state vector, edge attribute matrix and connectivity mask in real time based on the preset data parsing module to construct the state input tensor at the current operating moment;

[0067] S82. Input the state input tensor into the trained and converged graph attention-guided dual-policy deep reinforcement learning model. First, obtain the structure-aware representation of the current topology through the graph attention encoder, and then generate multiple bus switching path candidate action sequences through the path policy network.

[0068] S83. Input the action sequences of multiple bus switching candidate paths into the safety assessment network for electrical safety analysis, and calculate the safety scores of each path in terms of voltage stability, load balance, power supply restoration effect and connectivity guarantee.

[0069] S84. Based on the policy confidence output by the path policy network and the risk level output by the security assessment network, perform multidimensional sorting and joint screening on all candidate path action sequences, and eliminate path sequences that do not meet structural constraints or have high risk levels.

[0070] S85. Select the path action sequence with the highest comprehensive score from the reserved path set as the optimal bus switching scheme in the current scheduling cycle, and output the specific operation of each step contained in the path and the corresponding target switch number to the control command buffer module.

[0071] S86. The optimal bus switching path action sequence is handed over to the scheduling and execution module for feasibility verification before action scheduling, and marked as pending execution status, and then proceeds to the next physical action control process.

[0072] Optionally, S9 specifically includes:

[0073] S91. Each operation instruction in the optimal bus switching path action sequence is sent to the scheduling execution module in the order of operation. The instruction includes the target switch number, operation type identifier, execution timestamp and associated fault recovery path marker.

[0074] S92. Call the power distribution automation control system to sequentially trigger the control units of each target switch in the ring network box to complete the physical closing or opening operation of the switch. During the execution process, the operation response status, feedback signal confirmation information and abnormal alarm information are collected in real time.

[0075] S93. After each operation is completed, update and verify the current topology status, regenerate the graph structure model and refresh the electrical status information, confirm whether the previous action has produced the expected structural change effect, and determine whether there are any over-limit, voltage imbalance or unplanned power outage phenomena.

[0076] S94. If a control anomaly, communication failure, or equipment failure occurs during any operation step, the current path execution process will be interrupted, the system state will be rolled back to the previous operation point, the fault information will be recorded, and the path replanning mechanism will be started.

[0077] S95. After all operations in the path sequence are successfully executed, the bus power supply path reconstruction of the target area is completed, and the power supply strategy configuration table, switch status monitoring diagram and real-time operation scheduling diagram are updated.

[0078] S96. Write the operation instructions, time records, execution results, electrical state changes and path stability data of the entire switching process into the operation log module to provide feedback for subsequent incremental updates and online optimization of the reinforcement learning model.

[0079] The beneficial effects of this invention are:

[0080] This invention achieves intelligent optimization of the entire process of switching paths for ring network busbars by constructing a graph structure model and introducing a dual-strategy deep reinforcement learning architecture guided by graph attention. Compared with existing technical solutions, it has significant beneficial effects.

[0081] First, this invention constructs a graph structure model with directionality and weight information based on real-time operating data of the power distribution system. This model can accurately describe the electrical connections and topological dependencies between buses, loads, feeders, and switches, providing a high-fidelity structural semantic foundation for subsequent strategy generation and path selection. By aggregating features of this graph structure using a graph attention encoder, this invention significantly improves the model's sensitivity to local connectivity and changes in electrical state, enhancing the contextual awareness of path strategies.

[0082] Secondly, the dual-strategy deep reinforcement learning model proposed in this invention combines the path generation network and the safety assessment network in parallel and collaboratively. This ensures operational effectiveness while comprehensively evaluating the risk level of each path across multiple dimensions, including voltage stability, load balancing, and path connectivity. This effectively avoids the uncontrollable behavior caused by neglecting safety constraints in traditional single-strategy models. Through the design of a joint training mechanism and a cross-guidance module, a dynamic trade-off between path feasibility and global optimality is achieved, improving the adaptability and reliability of path output results under complex operating conditions.

[0083] Furthermore, this invention constructs a multi-objective reinforcement learning reward function system, introducing multiple performance indicators such as operation time, switching steps, voltage fluctuation, power outage range, and topology reconfiguration cost, to achieve a comprehensive characterization of path optimization objectives during reinforcement learning training. Simultaneously, by combining a time-difference error mechanism and an experience replay strategy, the model achieves more stable convergence and stronger generalization ability during the training phase. Especially under scenarios of dynamic load changes, node failure to operate, or multiple faults, the proposed method can still quickly output physically feasible and safe bus switching paths, effectively shortening fault handling time and improving power restoration efficiency.

[0084] In summary, this invention significantly enhances the intelligent scheduling capabilities of ring network power distribution systems in complex environments. It not only achieves optimal generation of switching paths and steady-state safety control, but also possesses scalable engineering deployment value, providing crucial path planning and decision support technologies for building a new generation of smart power distribution networks. Attached Figure Description

[0085] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0086] Figure 1 This is an overall flowchart of the ring network bus switching path optimization method based on deep reinforcement learning proposed in this invention;

[0087] Figure 2 This is a schematic diagram of the graph attention-guided dual-strategy deep reinforcement learning model proposed in this invention.

[0088] Figure 3 This is a flowchart illustrating the dynamic generation and execution of the ring network box bus switching path proposed in this invention. Detailed Implementation

[0089] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0090] refer to Figure 1-3 The method for optimizing the switching path of the ring network box busbar based on deep reinforcement learning includes the following steps:

[0091] S1. Collect real-time operating data of all ring network boxes in the power distribution system. The data includes the voltage amplitude, current direction, active load, reactive load, fault status, switch closing status and network topology of each node.

[0092] S2. Construct a graph structure model of the power distribution system. Model each ring network box bus, load node, and branch switch as nodes in the graph structure. Model the physical connection lines as edges with direction and weights. The weights include line impedance, current capacity, and voltage level information.

[0093] S3. Based on the constructed graph structure model and the running state information, define the state space of reinforcement learning. The state includes node electrical feature vectors, edge attribute matrices, network connectivity matrices, candidate switching targets, and fault isolation requirements.

[0094] S4. Define the action space as the set of all executable bus switching operations. The operation includes the closing or opening behavior of a single switch in the ring network box. Each complete switching path consists of multiple atomic actions, forming a path action sequence.

[0095] S5. Construct a graph attention-guided dual-strategy deep reinforcement learning model, the model including: a graph attention encoder, a path policy network, and a security evaluation network;

[0096] S6. Construct a multi-objective reinforcement learning reward function, wherein the reward function takes path steps, load transfer cost, operation time delay, voltage offset index and path risk level as inputs, and forms the final path reward value through weighted combination.

[0097] S7. Using historical bus switching data and the constructed simulated power distribution scenario, a strategy optimization method based on the Actor-Critic structure is adopted to jointly train the dual-strategy network. Time difference error is used for value evaluation and update, and an experience replay mechanism is used to improve training stability until the strategy network converges.

[0098] S8. During actual operation, the current power distribution system status is input into the dual-strategy deep reinforcement learning model after training convergence, and the bus switching path action sequence that satisfies electrical safety constraints and the optimal objective function is dynamically generated.

[0099] S9. Execute the actual control operations of each target switch in the ring network box in sequence according to the bus switching path action sequence to realize bus switching operations for fault isolation, load transfer or reconfiguration of power supply path, and complete the intelligent reconfiguration of the power distribution system.

[0100] This invention achieves full-process optimized control of the switching path of the ring network busbar by constructing a complete closed-loop process from data acquisition, graph structure modeling, state-action space construction, deep reinforcement learning modeling, path policy output to final action execution. Compared with traditional scheduling methods based on fixed rules and path traversal, this method has advantages such as strong structural understanding, high policy generation accuracy, and timely execution response, which can significantly improve the autonomous decision-making ability and intelligent operation level of the power distribution system in complex topology and variable load environments.

[0101] In this embodiment, the real-time operating data of the ring network box includes the voltage amplitude, current direction, active load, reactive load, fault status, switch position status, and network topology of each node.

[0102] By introducing multi-dimensional electrical operation data such as voltage, current, active load, reactive load, fault status, switch status, and topology, a more complete and dynamic input data system than traditional methods has been established. This information system is characterized by its high timeliness, comprehensive state coverage, and clear physical meaning, enabling the model to have stronger environmental awareness and contextual adaptation capabilities in state expression and strategy matching, providing a solid data support foundation for path strategy generation.

[0103] In this embodiment, S2 specifically includes:

[0104] S21. Analyze the structural composition and electrical connection relationship of all ring network boxes in the power distribution system, extract bus nodes, load nodes, feeder nodes, switch nodes and intermediate tie nodes, and assign a unique node number and node type code to each type of node.

[0105] S22. Construct the extracted nodes into a set of nodes in a graph structure model. Each node in the set records its physical coordinates, the list of connected superior or subordinate nodes, the area identifier, the power supply zone it belongs to, and the initialization attributes of its node electrical status.

[0106] S23. Model the connection relationship of all physical lines in the power distribution system and generate the edge set in the graph structure model. Each edge in the edge set consists of a starting node and a target node, and is accompanied by edge direction attribute, connection type identifier and unique edge number.

[0107] S24. Assign line attribute information to each edge in the edge set, including line impedance estimate, rated current capacity, voltage level label, channel affiliation, whether it is a cross-regional connection, and power supply path offset coefficient.

[0108] S25. Construct a directed graph structure representation based on the node set and edge set, and map the connection relationship between nodes and edges into an adjacency matrix and a direction mask matrix. The adjacency matrix is ​​used to represent the structural connection strength between nodes, and the direction mask matrix is ​​used to identify the feasible channels of directed edges in path search.

[0109] S26. The directed graph structure model is hierarchically classified according to node type to form a structure partitioning graph, and stored as a data structure that can be parsed by a graph neural network. The structure is used as the basis for structure-aware input in the subsequent state representation construction and reinforcement learning path strategy generation process.

[0110] By mapping electrical units such as buses, loads, feeders, and switches to structured nodes in a directed graph model and establishing a set of edges with electrical attributes, a formalized, high-fidelity distribution network graph structure is constructed. This structure not only preserves the connectivity and hierarchy of the physical power grid but also facilitates deep encoding by structure-aware models such as graph neural networks. It solves the problems of information sparsity and structural instability in traditional topology abstraction methods, and helps improve the structural understanding and feature extraction capabilities of subsequent policy networks.

[0111] In this embodiment, S3 specifically includes:

[0112] S31. Extract the current electrical operating parameters of all nodes from the directed graph structure model, and construct the node electrical state feature vector. The feature vector includes voltage amplitude, current direction, active power, reactive power, frequency offset, node type identifier, equipment operating status identifier and region code.

[0113] S32. Extract the connection attributes of each edge in the directed graph structure and construct an edge attribute description matrix. The attributes include the impedance level, voltage level, switch linkage status, current on / off status, line load rate, connection directionality mark and operable status label of the connection line.

[0114] S33. Construct a network connectivity matrix based on the node set and edge set. The connectivity matrix is ​​used to record the reachability relationship and power outage status between each node in the graph, and to identify the integrity and validity of the power supply path under the current topology.

[0115] S34. Identify faulty nodes, isolated nodes, and redundant power supply nodes based on the current operating status and fault diagnosis results, and filter out all candidate operation nodes and target recovery areas that can perform switching tasks based on the topology diagram.

[0116] S35. Integrate the node electrical state feature vector, edge attribute description matrix, network connectivity matrix, fault identification vector and candidate operation set to construct a high-dimensional state input tensor, which is used to characterize the structure and operating state of the power distribution system at the current moment;

[0117] S36. Perform dimension alignment, normalization, and missing value repair on the high-dimensional state input tensor to ensure the integrity of the data input, and use it as the state space input of the deep reinforcement learning model to drive the policy generation and path decision-making process.

[0118] By constructing a high-dimensional state tensor that integrates node features, edge attributes, connectivity relationships, and fault markers, a refined modeling of the operating state of the power distribution system is achieved. This state representation not only possesses good spatiotemporal continuity and structural consistency, but also dynamically adapts to different network scales and operational scenarios, providing a solid foundation for the generalization ability and accuracy of the strategy model in path decision-making, and solving the problems of single state dimension and weak abstract expression ability in traditional methods.

[0119] In this embodiment, S4 specifically includes:

[0120] S41. Based on the directed graph structure model and electrical operating status, identify all switch nodes in a controllable state and filter out the target switch set that has operating authority and is not in a fault isolation state.

[0121] S42. Establish a corresponding set of atomic actions for each target switch. The atomic actions include the closing or opening operation of a single switch, and mark the direction of action, changes in electrical connection and the range of the affected area.

[0122] S43. Based on the current topology and candidate target power supply areas, construct all possible combinations of bus switching paths. The path is an action sequence formed by combining multiple atomic actions in the order of operation. Each action sequence satisfies the topology reachability requirements of the starting point and the ending point.

[0123] S44. Perform path validity verification on all candidate path action sequences, eliminate unexecutable paths caused by loop conflicts, path duplication, physical interlocks and isolation restrictions, and retain the set of valid action sequences that meet the basic operational constraints.

[0124] S45. Construct an action space representation structure for each valid action sequence, and clarify the number of operation steps, the target switch number of each step, the expected action direction, the expected influence area number, and the operation dependencies between each step.

[0125] S46. The action space structure is used as the action input of the dual-policy deep reinforcement learning model, which is used by the path policy network for policy sampling and path selection during training and inference.

[0126] By constructing complete path sequences based on atomic operations and performing feasibility screening and structural dependency modeling on all action sets, the strategy effectively avoids the failure problems caused by path non-executability, operation conflicts, or circulation risks in traditional switching strategies. The defined action space not only clearly reflects the operable boundaries of the system, but also embodies the physical constraints of bus switching control through operation dependency logic, improving the constraint consistency of strategy training and the engineering executability of path output.

[0127] In this embodiment, S5 specifically includes:

[0128] S51. Construct a graph attention encoder module. The input consists of node feature vectors, edge attribute matrices, and adjacency structure indices from the power distribution system graph structure model. A multi-channel attention mechanism is used to aggregate adjacency context information for each node, generating a node embedding representation with topology awareness. The embedding representation H of node i... i Defined as:

[0129]

[0130] Where, α ij Let X be the attention weight of node i to its neighboring node j, W be the node transformation weight matrix, and X be the attention weight of node i to its neighboring node j. j Let j be the feature vector of node j;

[0131] S52. The graph embedding state representation is input into the path policy generation network, which includes a state projection layer, an action scoring layer, and a candidate selection layer. The state vector is compressed and reconstructed using deep feature encoding. The embedding features of each node in the path are extracted, and the policy score of the path action sequence is calculated, defined as follows:

[0132]

[0133] Where, τ k Let H be the k-th candidate path, T be the number of atomic actions contained in this path, and H be the number of atomic actions contained in this path. nt Let U be the embedding vector of the t-th node in the path, and let U, V, and b be learnable parameters. Finally, output the currently executable combination of path actions and their priority order.

[0134] S53. Simultaneously construct a security assessment network, receiving the same state inputs and action sequence outputs as the path policy generation network. Independently quantify and model indicators such as path voltage stability, load transfer balance, and path connectivity, and generate security label vectors. These vectors are then fused with the policy score to obtain the final comprehensive score of the path.

[0135] S(τ k )=λ1·Softmax(Q(τ k ))+λ2·(1-||Rk ||2)

[0136] Where λ1 and λ2 are the policy confidence and risk suppression coefficients, respectively, and R k Let ||R| be the multidimensional risk vector for path k. k ||2 represents its second norm;

[0137] S54. Establish a cross-guidance mechanism between the path policy generation network and the security assessment network, and jointly fuse the action scores of the policy network and the path labels of the security network to form a final path action candidate set. The candidate set is sorted according to the policy distribution weight and security confidence, and path sequences that do not meet the continuity constraints or have topological conflicts are eliminated.

[0138] S55. To achieve multi-task path evaluation training, the graph attention encoder, path policy generation network, and security evaluation network are constructed into a parallel fusion structure. A unified input interface and joint output structure are defined to complete the construction of the dual-policy collaborative scheduling structure. The structure serves as a complete graph attention-guided dual-policy deep reinforcement learning model framework for the optimal policy generation and path selection process of bus switching paths.

[0139] This invention constructs a dual-policy deep reinforcement learning model with a graph attention encoder as the perception entry point and a path policy network and a security assessment network operating collaboratively. This structure not only extracts multi-level embedded information from the structure for path decision-making but also uses a parallel security assessment mechanism to screen the output path for risks, effectively preventing unsafe policies from entering the execution process. Compared to traditional reinforcement learning models, this structure possesses stronger topology modeling capabilities, path security constraint capabilities, and multi-objective decision fusion capabilities.

[0140] In this embodiment, S6 specifically includes:

[0141] S61. Determine the multi-dimensional evaluation objectives for bus switching path optimization and construct a reward evaluation index system, including seven basic evaluation dimensions: number of path switching steps, action execution time, node voltage deviation, load power deviation, electrical connectivity restoration speed, power outage coverage, and path risk level.

[0142] S62. For each candidate path action sequence, execute its complete switching process in the form of simulation in the reinforcement learning training framework, and collect in real time the topology changes, key node status changes, fault isolation success rate, load transfer status stability and the impact range of each step of the operation during the switching execution process.

[0143] S63. Based on the collected switching execution data, calculate the individual scores for each reward evaluation dimension. The individual scores are constructed using normalized mapping and constraint boundary projection to ensure that different dimension indicators can be compared and integrated on the same scale.

[0144] S64. Assign initial weight parameters to each reward dimension, construct a weighted aggregation function and form a multi-objective combined reward output. The weighting function supports dynamic adjustment of weight ratios during training based on the risk level, policy confidence and convergence speed of the current model output policy, thereby realizing a balance and control mechanism between efficiency and safety.

[0145] S65. Introduce a reward sensitivity adjustment mechanism to perform attribution analysis on the feedback results of each round of path actions, determine the response strength of the reward function to high-risk operations, path oscillation behavior and topology conflict events, and automatically adjust the harmonic coefficient between target items to adapt to different load scenarios and topology evolution conditions.

[0146] S66. The resulting multi-objective reinforcement learning reward function is integrated into the dual-policy deep reinforcement learning training process as the basis for constructing the joint loss of the path policy network and the safety assessment network. This provides feedback direction for policy updates and drives the policy network to iteratively converge toward the optimal solution space that simultaneously satisfies switching efficiency, electrical safety, and structural coherence.

[0147] By constructing a reward evaluation system covering multiple indicators such as path steps, voltage stability, load balancing, connectivity restoration, and risk level, and introducing dynamic weighting and sensitivity adjustment mechanisms, this invention achieves adaptive control between efficiency and safety during reinforcement learning. This reward mechanism enables the policy model to quickly locate the optimal solution direction while avoiding path deviations from engineering feasibility due to a single dominant objective, significantly improving the policy training convergence speed and the stability and controllability of the policy results.

[0148] In this embodiment, S7 specifically includes:

[0149] S71. Construct an offline training environment. Based on the actual topology of the power distribution system, historical operating data, and typical fault conditions, generate a simulation training sample set covering multiple operating scenarios. The sample set includes node state sequences, operable action sequences, and path execution result labels.

[0150] S72. Initialize the graph attention-guided dual-strategy deep reinforcement learning model structure, which includes three parts: graph attention encoder, path policy generation network and safety evaluation network, and set the initial parameters, number of training rounds, learning rate, batch size and update frequency for each module.

[0151] S73. A joint policy optimization method based on the Actor-Critic framework is adopted, and the path policy network and the safety assessment network are trained synchronously. The path policy network is responsible for generating path action policies, and the safety assessment network is responsible for generating path execution feasibility feedback. The two are updated with parameters through a joint loss function during training.

[0152] S74. Introduce a time difference error mechanism to calculate the cumulative evaluation deviation of the path action sequence under different state transitions, which serves as the basis for updating the value evaluation network in policy optimization, thereby improving the model's ability to judge global returns during the path selection process.

[0153] S75. Configure an experience replay mechanism to record historical state transition sequences, path execution behavior and reward feedback results, build an efficient training sample pool, and extract unseen combined states in batches from it for model training to prevent overfitting and improve policy generalization ability.

[0154] S76. Set the stability judgment criteria for the policy, continuously monitor the path convergence speed, reward improvement trend and action sequence fluctuation range during the training process, and judge whether the policy network has converged to a stable output state. If the preset stopping condition is met, the training process is ended and the model parameters are fixed.

[0155] This invention constructs a stable, fast-converging, and highly generalizable training framework by introducing an Actor-Critic-based joint optimization mechanism, a time-difference error-driven policy iteration process, and experience replay to enhance training sample diversity. Compared with the policy oscillation, overfitting, and underexploration problems existing in traditional reinforcement learning policy training, this training process is more suitable for handling large-scale graph-structured state spaces and multi-objective path decision-making tasks, significantly improving the model reliability before system deployment.

[0156] In this embodiment, S8 specifically includes:

[0157] S81. Obtain all operating data of the current power distribution system, and update the graph structure model, node state vector, edge attribute matrix and connectivity mask in real time based on the preset data parsing module to construct the state input tensor at the current operating moment;

[0158] S82. Input the state input tensor into the trained and converged graph attention-guided dual-policy deep reinforcement learning model. First, obtain the structure-aware representation of the current topology through the graph attention encoder, and then generate multiple bus switching path candidate action sequences through the path policy network.

[0159] S83. Input the action sequences of multiple bus switching candidate paths into the safety assessment network for electrical safety analysis, and calculate the safety scores of each path in terms of voltage stability, load balance, power supply restoration effect and connectivity guarantee.

[0160] S84. Based on the policy confidence output by the path policy network and the risk level output by the security assessment network, perform multidimensional sorting and joint screening on all candidate path action sequences, and eliminate path sequences that do not meet structural constraints or have high risk levels.

[0161] S85. Select the path action sequence with the highest comprehensive score from the reserved path set as the optimal bus switching scheme in the current scheduling cycle, and output the specific operation of each step contained in the path and the corresponding target switch number to the control command buffer module.

[0162] S86. The optimal bus switching path action sequence is handed over to the scheduling and execution module for feasibility verification before action scheduling, and marked as pending execution status, and then proceeds to the next physical action control process.

[0163] By inputting the real-time operating status of the power distribution system into a converged strategy model and combining the path strategy output with the security assessment network for collaborative screening, this invention can stably output bus switching paths that meet topological constraints, electrical safety, and multi-objective optimality requirements in practical applications. Compared with existing methods that can only provide offline recommendations or static solutions, this method has significant advantages such as high real-time performance, strong adaptability, and the engineering feasibility of path results, significantly improving the automation level of power distribution system fault response and load dispatch.

[0164] In this embodiment, S9 specifically includes:

[0165] S91. Each operation instruction in the optimal bus switching path action sequence is sent to the scheduling execution module in the order of operation. The instruction includes the target switch number, operation type identifier, execution timestamp and associated fault recovery path marker.

[0166] S92. Call the power distribution automation control system to sequentially trigger the control units of each target switch in the ring network box to complete the physical closing or opening operation of the switch. During the execution process, the operation response status, feedback signal confirmation information and abnormal alarm information are collected in real time.

[0167] S93. After each operation is completed, update and verify the current topology status, regenerate the graph structure model and refresh the electrical status information, confirm whether the previous action has produced the expected structural change effect, and determine whether there are any over-limit, voltage imbalance or unplanned power outage phenomena.

[0168] S94. If a control anomaly, communication failure, or equipment failure occurs during any operation step, the current path execution process will be interrupted, the system state will be rolled back to the previous operation point, the fault information will be recorded, and the path replanning mechanism will be started.

[0169] S95. After all operations in the path sequence are successfully executed, the bus power supply path reconstruction of the target area is completed, and the power supply strategy configuration table, switch status monitoring diagram and real-time operation scheduling diagram are updated.

[0170] S96. Write the operation instructions, time records, execution results, electrical state changes and path stability data of the entire switching process into the operation log module to provide feedback for subsequent incremental updates and online optimization of the reinforcement learning model.

[0171] By establishing an operation sequence execution mechanism, an action verification feedback mechanism, and a failure rollback mechanism, this invention constructs a closed-loop control switching execution system. Each operation step can undergo topology verification and electrical stability monitoring, ensuring the safe and effective execution of physical control actions in the actual system. Compared to the shortcomings of traditional systems, such as lack of feedback guarantees and irreversible malfunctions, this method provides an interruptible, verifiable, and traceable safety guarantee chain, effectively improving the engineering controllability of bus switching operations and the automation level of system operation and maintenance.

[0172] Example 1:

[0173] To verify the feasibility of this invention in practice, it was applied to a core power distribution area in a large city. The power distribution network adopts a ring network structure, powered by a single 110kV substation, covering 32 ring network box nodes, 88 10kV lines, and over 120 intelligent sectionalizing switches. This area contains various types of critical loads, such as hospitals, transportation hubs, shopping malls, and high-rise residential communities. The power distribution network structure is complex, and load fluctuations are frequent, placing extremely high demands on power supply continuity and rapid fault isolation capabilities during daily operation.

[0174] The original system relied primarily on fixed switching rules and manual experience for scheduling. Path planning prioritized "shortest path" or "fewest operations," making it difficult to dynamically adapt to fault locations, load changes, and topology evolution. This often resulted in problems such as infeasible switching paths, operational conflicts, or temporary power outages of some loads, severely impacting power supply reliability. In particular, when single-point faults or switch failures occurred, scheduling responses often took several minutes, sometimes even requiring temporary command intervention, leading to low operational efficiency and system instability.

[0175] After deploying the method of this invention in the above scenario, the scheduling system first acquires real-time operating data of all ring network boxes through the acquisition module, including voltage, current, active load, reactive load, switch status, and fault location information. All information is structured and encoded into a graph model, and then used as a state input tensor to input into a graph attention-guided dual-strategy deep reinforcement learning model.

[0176] When a circuit breaker trips on a main line, the system immediately identifies the power outage area and automatically generates candidate switching paths based on the graph structure model. The path strategy network outputs five sets of feasible switching operation sequences, and the safety assessment network performs voltage over-limit risk analysis, path connectivity verification, and load redistribution assessment on each sequence. The final selected optimal path includes three switch opening and closing operations, all located within the adjacent ring network box of the original power outage area, possessing minimal load disturbance and minimum path length.

[0177] In actual operation, the average completion time from fault identification to strategy generation is 1.1 seconds; the average time to complete the entire process is no more than 10 seconds, far lower than the response cycle of approximately 12 minutes for traditional scheduling processes in this region. Statistical analysis of historical switching logs shows that in 21 consecutive fault handling incidents, the success rate of the proposed solution reached 100%, the average load transfer current offset rate was controlled within 5%, and there were no over-limit voltage nodes, no islanding areas, and no records of malfunctioning linkage switches during the switching process.

[0178] Compared to the original system, this method significantly improves the speed and accuracy of scheduling response. The operational results demonstrate good structural feasibility and electrical safety, avoiding path redundancy, conflict dependencies, and overload risks inherent in traditional rule-based methods. More importantly, the model possesses continuous training and self-learning capabilities, automatically adapting to topology evolution and load distribution adjustments after long-term deployment, achieving a shift from rule-dependent to policy-driven intelligent scheduling. This embodiment verifies that the method of this invention has the advantages of being implementable, verifiable, and continuously optimizable in multi-node, high-density power distribution scenarios. It not only improves fault recovery efficiency but also significantly enhances the dynamic handling capability of the power distribution network in the face of uncertain events, possessing significant engineering practical value and promising prospects for widespread application.

[0179] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning, characterized in that, Includes the following steps: S1. Collect real-time operating data of all ring network boxes in the power distribution system; S2. Construct a graph structure model of the power distribution system, model each ring network box bus, load node, and branch switch as nodes in the graph structure, and model the physical connection lines as edges with direction and weight. S3. Based on the constructed graph structure model and runtime state information, define the state space for reinforcement learning; S4. Define the action space as the set of all executable bus switching operations; S5. Construct a graph attention-guided dual-strategy deep reinforcement learning model, including a graph attention encoder, a path policy network, and a security evaluation network; S6. Construct a multi-objective reinforcement learning reward function and form the final path reward value through weighted combination; S7. Using historical bus switching data and the constructed simulated power distribution scenario, a strategy optimization method based on the Actor-Critic structure is used to train the dual-strategy deep reinforcement learning model. Time difference error is used for value evaluation and update, and an experience replay mechanism is used to improve training stability. S8. Input the current power distribution system status into the trained and converged dual-strategy deep reinforcement learning model, and dynamically generate a bus switching path action sequence that satisfies electrical safety constraints and the optimal objective function. S9. Execute the actual control operations of each target switch in the ring main unit in sequence according to the bus switching path action sequence; S5 specifically includes: S51. Construct a graph attention encoder module. The input is the node feature vector, edge attribute matrix and adjacency structure index in the power distribution system graph structure model. The adjacency context information of each node is aggregated using a multi-channel attention mechanism. The output is a graph embedding state representation with structure awareness. The state representation retains the local topological differences and edge weight sensitivity between each bus segment. S52. Input the graph embedding state representation into the path policy generation network. The path policy generation network includes a state projection layer, an action scoring layer and a candidate selection layer. The state vector is compressed and reconstructed using deep feature encoding to form a path action scoring sequence. The network outputs the currently executable path action combination and its priority order. S53. Simultaneously construct a safety assessment network, receive the same state input and action sequence output as the path strategy generation network, and use the regional conflict determination unit, voltage stability detection unit and path connectivity verification unit to score the electrical safety of each candidate path, and output the multi-dimensional safety label and risk level identifier of each path. S54. Establish a cross-guidance mechanism between the path policy generation network and the security assessment network, and jointly fuse the action scores of the policy network and the path labels of the security network to form a final path action candidate set. The candidate set is sorted according to the policy distribution weight and security confidence, and path sequences that do not meet the continuity constraints or have topological conflicts are eliminated. S55. Construct the graph attention encoder, path policy generation network and security evaluation network into a parallel fusion structure, define a unified input interface and joint output structure, and complete the construction of the dual-policy collaborative scheduling structure. The structure serves as a complete graph attention-guided dual-policy deep reinforcement learning model framework.

2. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, The real-time operating data of the ring network box includes the voltage amplitude, current direction, active load, reactive load, fault status, switch position, and network topology of each node.

3. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, S2 specifically includes: S21. Analyze the structural composition and electrical connection relationship of all ring network boxes in the power distribution system, extract bus nodes, load nodes, feeder nodes, switch nodes and intermediate tie nodes, and assign a unique node number and node type code to each type of node. S22. Construct the extracted nodes into a set of nodes in a graph structure model. Each node in the set records its physical coordinates, the list of connected superior or subordinate nodes, the area identifier, the power supply zone it belongs to, and the initialization attributes of its node electrical status. S23. Model the connection relationship of all physical lines in the power distribution system and generate the edge set in the graph structure model. Each edge in the edge set consists of a starting node and a target node, and is accompanied by edge direction attribute, connection type identifier and unique edge number. S24. Assign line attribute information to each edge in the edge set, including line impedance estimate, rated current capacity, voltage level label, channel affiliation, whether it is a cross-regional connection, and power supply path offset coefficient. S25. Construct a directed graph structure representation based on the node set and edge set, and map the connection relationship between nodes and edges into an adjacency matrix and a direction mask matrix; S26. Classify the directed graph structure model according to node type to form a structure partitioning graph, and store it as a data structure that can be parsed by a graph neural network.

4. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, S3 specifically includes: S31. Extract the current electrical operating parameters of all nodes from the directed graph structure model, and construct the node electrical state feature vector. The feature vector includes voltage amplitude, current direction, active power, reactive power, frequency offset, node type identifier, equipment operating status identifier and region code. S32. Extract the connection attributes of each edge in the directed graph structure and construct an edge attribute description matrix. The attributes include the impedance level, voltage level, switch linkage status, current on / off status, line load rate, connection directionality mark and operable status label of the connection line. S33. Construct a network connectivity matrix based on the node set and edge set; S34. Identify faulty nodes, isolated nodes, and redundant power supply nodes based on the current operating status and fault diagnosis results, and filter out all candidate operation nodes and target recovery areas that can perform switching tasks based on the topology diagram. S35. Integrate the node electrical state feature vector, edge attribute description matrix, network connectivity matrix, fault identifier vector and candidate operation set to construct a high-dimensional state input tensor; S36. Perform dimension alignment, normalization, and missing value repair on the high-dimensional state input tensor.

5. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, S4 specifically includes: S41. Based on the directed graph structure model and electrical operating status, identify all switch nodes in a controllable state and filter out the target switch set that has operating authority and is not in a fault isolation state. S42. Establish a corresponding set of atomic actions for each target switch. The atomic actions include the closing or opening operation of a single switch, and mark the direction of action, changes in electrical connection and the range of the affected area. S43. Based on the current topology and candidate target power supply areas, construct all possible combinations of bus switching paths. The path is an action sequence formed by combining multiple atomic actions in the order of operation. Each action sequence satisfies the topology reachability requirements of the starting point and the ending point. S44. Perform path validity verification on all candidate path action sequences, eliminate unexecutable paths caused by loop conflicts, path duplication, physical interlocks and isolation restrictions, and retain the set of valid action sequences that meet the basic operational constraints. S45. Construct an action space representation structure for each valid action sequence, and clarify the number of operation steps, the target switch number of each step, the expected action direction, the expected influence area number, and the operation dependencies between each step. S46. Use the action space structure as the action input for the dual-policy deep reinforcement learning model.

6. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, S6 specifically includes: S61. Determine the multi-dimensional evaluation objectives for bus switching path optimization and construct a reward evaluation index system, including seven basic evaluation dimensions: number of path switching steps, action execution time, node voltage deviation, load power deviation, electrical connectivity restoration speed, power outage coverage, and path risk level. S62. For each candidate path action sequence, execute its complete switching process in a simulation form within a dual-strategy deep reinforcement learning training framework, and collect in real time the topology changes, key node state changes, fault isolation success rate, load transfer state stability, and the impact range of each step during the switching execution process. S63. Based on the collected switching execution data, calculate the individual scores for each reward evaluation dimension. The individual scores are constructed using normalized mapping and constraint boundary projection. S64. Assign initial weight parameters to each reward dimension, construct a weighted aggregation function and form a multi-objective combined reward output. The weighting function supports dynamic adjustment of weight ratios during training based on the risk level, policy confidence and convergence speed of the current model output policy. S65. Introduce a reward sensitivity adjustment mechanism to perform attribution analysis on the feedback results of each round of path actions, determine the response strength of the reward function to high-risk operations, path oscillation behavior and topology conflict events, and automatically adjust the harmonic coefficient between target items to adapt to different load scenarios and topology evolution conditions. S66. Integrate the final multi-objective reinforcement learning reward function into the dual-policy deep reinforcement learning training process.

7. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, Specifically, S7 includes: S71. Construct an offline training environment. Based on the actual topology of the power distribution system, historical operating data, and typical fault conditions, generate a simulation training sample set covering multiple operating scenarios. The sample set includes node state sequences, operable action sequences, and path execution result labels. S72. Initialize the graph attention-guided dual-strategy deep reinforcement learning model structure, which includes three parts: graph attention encoder, path policy generation network and safety evaluation network, and set the initial parameters, number of training rounds, learning rate, batch size and update frequency for each module. S73. A joint policy optimization method based on the Actor-Critic framework is adopted, and the path policy network and the safety assessment network are trained synchronously. The path policy network is responsible for generating path action policies, and the safety assessment network is responsible for generating path execution feasibility feedback. The two are updated with parameters through a joint loss function during training. S74. Introduce a time difference error mechanism to calculate the cumulative evaluation deviation of the path action sequence under different state transitions; S75. Configure an experience replay mechanism to record historical state transition sequences, path execution behaviors and reward feedback results, build an efficient training sample pool, and extract unseen combined states in batches from it. S76. Set the stability judgment criteria for the policy, continuously monitor the path convergence speed, reward improvement trend and action sequence fluctuation range during the training process, and judge whether the policy network has converged to a stable output state. If the preset stopping condition is met, the training process is ended and the model parameters are fixed.

8. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, S8 specifically includes: S81. Obtain all operating data of the current power distribution system, and update the graph structure model, node state vector, edge attribute matrix and connectivity mask in real time based on the preset data parsing module to construct the state input tensor at the current operating moment; S82. Input the state input tensor into the trained and converged graph attention-guided dual-policy deep reinforcement learning model. First, obtain the structure-aware representation of the current topology through the graph attention encoder, and then generate multiple bus switching path candidate action sequences through the path policy network. S83. Input the action sequences of multiple bus switching candidate paths into the safety assessment network for electrical safety analysis, and calculate the safety scores of each path in terms of voltage stability, load balance, power supply restoration effect and connectivity guarantee. S84. Based on the policy confidence output by the path policy network and the risk level output by the security assessment network, perform multidimensional sorting and joint screening on all candidate path action sequences, and eliminate path sequences that do not meet structural constraints or have high risk levels. S85. Select the path action sequence with the highest comprehensive score from the reserved path set as the optimal bus switching scheme in the current scheduling cycle, and output the specific operation of each step contained in the path and the corresponding target switch number to the control command buffer module. S86. The optimal bus switching path action sequence is handed over to the scheduling execution module for feasibility verification before action scheduling, and marked as pending execution.

9. The method for optimizing the switching path of a ring network box busbar based on deep reinforcement learning according to claim 1, characterized in that, S9 specifically includes: S91. Each operation instruction in the optimal bus switching path action sequence is sent to the scheduling execution module in the order of operation. The instruction includes the target switch number, operation type identifier, execution timestamp and associated fault recovery path marker. S92. Call the power distribution automation control system to sequentially trigger the control units of each target switch in the ring network box to complete the physical closing or opening operation of the switch. During the execution process, the operation response status, feedback signal confirmation information and abnormal alarm information are collected in real time. S93. After each operation is completed, update and verify the current topology state, regenerate the graph structure model and refresh the electrical state information. S94. If a control anomaly, communication failure, or equipment failure occurs during any operation step, the current path execution process will be interrupted, the system state will be rolled back to the previous operation point, the fault information will be recorded, and the path replanning mechanism will be started. S95. After all operations in the path sequence are successfully executed, the bus power supply path reconstruction of the target area is completed, and the power supply strategy configuration table, switch status monitoring diagram and real-time operation scheduling diagram are updated. S96. Write the operation instructions, time records, execution results, electrical status changes and path stability data of the entire process of this switching operation into the operation log module.

Images