A power distribution district topology optimization generation method based on deep reinforcement learning

Abstract

This invention relates to the field of power system automation technology, specifically to a method for generating optimal distribution transformer area topology based on deep reinforcement learning. The steps include: obtaining the number of users, power supply radius, and node spacing of the distribution network; encoding the topology structure and dynamic operating characteristics using a dual-channel state matrix; decoupling the action space of distribution network branch decisions into a two-dimensional decision combination of backbone node selection and branch extension length based on a deep reinforcement learning network; dynamically updating the network topology based on action decision results using an agent; and allocating user loads using a multi-stage priority strategy; setting a composite reward function; and training the deep reinforcement learning network to convergence using a progressive training strategy to obtain the optimal topology structure. Compared with existing technologies, this invention, based on deep reinforcement learning, achieves optimal topology generation under power supply radius constraints through dynamic agent decision-making on branch extension and user allocation, effectively reducing line losses and supporting real-time access to distributed energy resources.

Description

Technical Field

[0001] This invention relates to the field of power system automation technology, and in particular to a power system automation technology. Background Technology

[0002] Distribution network topology optimization is a core technology for improving power grid reliability and reducing line losses. Current methods rely on manual experience or static algorithms, which suffer from poor dynamic response capabilities, unreasonable branch configuration, and unbalanced user allocation, resulting in high line loss rates and low resource utilization.

[0003] For example, Chinese patent application CN202411782677.5 discloses a distribution network topology reconfiguration method based on an improved DQN. This scheme constructs a state space based on historical loads according to the autocorrelation of load time series, but fails to separate topology information from load characteristics. This results in the inability to reflect the impact of changes in node depth on load characteristics as the topology expands. Furthermore, it changes the action object of model decision-making from the topology structure to action switches, leading to insufficient extraction of topology features and a lack of support for dynamic branch expansion. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology, such as insufficient extraction of topological features and lack of support for dynamic branch expansion, and to provide a distribution area topology optimization generation method based on deep reinforcement learning.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for generating distribution radio area topology optimization based on deep reinforcement learning, comprising the following steps:

[0007] The physical parameters of the distribution network are obtained, including the number of users in the distribution area, the power supply radius, and the node spacing. The physical parameters of the power grid are transformed into the state space, and the topology and dynamic operation characteristics are encoded by a dual-channel state matrix.

[0008] The dual-channel state matrix is ​​input into a deep reinforcement learning network, and the action space of the distribution network branch decision is decoupled into a two-dimensional decision combination of trunk node selection and branch extension length. The network topology is dynamically updated by the agent based on the action decision results, and a multi-stage priority strategy is adopted to allocate user load.

[0009] A deep reinforcement learning network is configured with a composite reward function to guide the agent to balance fast power supply and reduced line loss. A progressive training strategy is used to train the deep reinforcement learning network until convergence, and the output generates the optimal topology structure.

[0010] As a preferred technical solution, the dual-channel state matrix includes two channels representing the topological structure and dynamic operating characteristics, respectively. It is expressed as follows:

[0011] ,

[0012] In the formula, It is an adjacency matrix. User distribution characteristics, For depth features, This represents the total number of nodes in the distribution area.

[0013] Adjacency Matrix Used to represent the topology of a transformer area:

[0014] ,

[0015] In the formula, , Represents a node With nodes There are connecting lines. Represents a node With nodes There is no connecting line between them;

[0016] node User distribution characteristics for:

[0017] ,

[0018] Deep features For nodes The number of hops to the nearest trunk node.

[0019] As a preferred technical solution, the deep reinforcement learning network extracts topological space features through a dual-channel convolutional neural network and performs [further processing] through fully connected layers. Value prediction is performed, and gradient pruning and weight decay strategies are used to improve the model's generalization ability.

[0020] The adjacency matrix and the feature matrix containing user distribution and node depth in the input dual-channel state matrix are used to extract spatial features through two convolutional layers with different kernel sizes, and then the feature map is output uniformly through adaptive average pooling.

[0021] The feature map is flattened and dimensionality is gradually reduced through three fully connected layers. A random deactivation layer is added after the second fully connected layer to prevent overfitting. The final output dimension is the size of the action space.

[0022] The loss function is designed as follows:

[0023] ,

[0024] In the formula: For prediction The value is the output of the deep reinforcement learning network; For the goal The value is the target network's calculated value, and B is the batch size.

[0025] As a preferred technical solution, the reward function of the deep reinforcement learning network is shown in the following formula:

[0026] ,

[0027] ,

[0028] ,

[0029] ,

[0030] ,

[0031] In the formula, As a stability reward, This is the nominal voltage value. For the distribution of radio station area number Node voltage; As an economic reward, For the first The equivalent resistance of the line, For the first The current value of each line; As a risk penalty, For nodes load rate, For the line The current value, For the line Rated current carrying capacity.

[0032] As a preferred technical solution, the deep reinforcement learning network employs a three-stage training mechanism to dynamically adjust the exploration rate. With reward weight ;

[0033] The exploration rate decay formula is shown below:

[0034] ,

[0035] In the formula, For the first The exploration rate of the wheel is used to control the proportion of random actions; For training rounds;

[0036] As the number of training rounds increases, the reward weights for each training phase are progressively reduced. :

[0037] An early stop mechanism is set up so that training stops and the topology is generated when the sum of the absolute values ​​of the differences between the reward values ​​of each round and the average reward value in the rounds preceding the previous round is less than a set reward value difference threshold.

[0038] As a preferred technical solution, the two-dimensional decision combination of trunk node selection and branch extension length is represented as follows:

[0039] ,

[0040] In the formula, This represents the floor function; The original action number, , representing a high-dimensional discrete action space; The maximum number of branches in a single trunk line. , This refers to the number of nodes on the main trunk line.

[0041] As a preferred technical solution, the action space is embedded with power grid physical constraint rules to verify the validity of the constraints. Invalid actions are filtered out through the rules, as shown in the following formula:

[0042] ,

[0043] In the formula, The current number of nodes is the number of nodes created, and l is the branch extension length, which is the number of nodes added in a single extension.

[0044] As a preferred technical solution, the dynamic updating of the power grid topology based on action decision results includes: creating new nodes according to action decisions, and updating the adjacency matrix and node depth, as shown in the following formula:

[0045] ,

[0046] In the formula, Numbering is the numbering of the main node. For the new node, For the current node, To extend the length of the branch, The node depth.

[0047] As a preferred technical solution, the multi-stage priority strategy allocates user load based on real-time load forecasting and dynamic priority adjustment to achieve intelligent allocation of user load and ensure balanced operation of the power grid, as detailed below:

[0048] Compute node load rate :

[0049] ,

[0050] In the formula, For nodes Current load power, For nodes Rated capacity;

[0051] Perform node priority evaluation:

[0052] ,

[0053] In the formula, The node depth is represented by 0 for trunk nodes and hop count for branch nodes. For voltage deviation; ， ， These are weighting coefficients, which are dynamically adjusted through reinforcement learning.

[0054] The user access principle is: new users are accessed to the node with the highest priority. If multiple nodes have the same priority, the node with the smaller depth is selected.

[0055] When node load rate When the load rate exceeds the set node load threshold, new user access is rejected and branch expansion is triggered. The branch expansion mechanism is as follows: when a set number of users are rejected due to node overload, the main node with the largest available capacity is selected to expand the branch.

[0056] If the node voltage deviation If the node voltage deviation exceeds the set threshold, the node will be removed from the candidate list.

[0057] As a preferred technical solution, the dynamic adjustment of the weighting coefficients in the node priority evaluation is specifically as follows:

[0058] In high-load scenarios, where the average node load rate exceeds a set average load rate threshold, the weighting coefficient is increased. α Prioritize low-load nodes;

[0059] In low-voltage scenarios, where the average voltage deviation at nodes exceeds a set average voltage deviation threshold, the weighting coefficient is increased.γ Nodes with stable voltage should be selected first.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] 1) This invention proposes a distribution network topology optimization generation method based on deep reinforcement learning. An intelligent agent creates new nodes based on action decisions, updates the adjacency matrix and node depth, and dynamically updates the power grid topology. Furthermore, a multi-stage priority strategy is employed to allocate user loads. Based on real-time load forecasting and dynamic priority adjustment, intelligent allocation of user loads is achieved, ensuring balanced power grid operation. This ensures optimal matching between branch expansion and load distribution, solving the problem of topology and load separation optimization in traditional methods. It achieves optimal topology generation under power supply radius constraints, effectively reducing line losses and supporting real-time access to distributed energy resources.

[0062] 2) This invention transforms power grid physical parameters into a state space that can be processed by deep reinforcement learning. It solves the problem of separating topology information from load characteristics in traditional methods by encoding topology structure and dynamic operational features using a dual-channel matrix. Furthermore, it decouples the discrete action space from a two-dimensional combination of backbone node selection and branch expansion decisions, reducing decision complexity through mathematical mapping while embedding power grid physical constraint rules. This reduces the original action dimension by more than 50% and provides hard constraints to ensure all actions comply with power grid safety regulations.

[0063] 3) This invention designs a composite reward function, which combines multiple dimensions such as construction reward, depth reward, and empty branch penalty to quantify the quality of power grid operation, balance economy, stability and user experience, guide the agent to balance short-term benefits (fast power supply) and long-term goals (reducing line loss), and guide the agent to generate the optimal topology.

[0064] 4) The topology optimization generation network of this invention employs gradient pruning and weight decay strategies to enhance the model's generalization ability, addressing the problem of insufficient topological feature extraction in traditional DQN networks. Simultaneously, the topology optimization generation network adopts a progressive training strategy, dynamically adjusting the exploration rate and reward weights through a three-stage training mechanism. Early training emphasizes topology exploration, the middle stage balances exploration and utilization, and the later stage converges to the optimal strategy. Furthermore, an early stopping mechanism is implemented to reduce ineffective training rounds. This solves the problems of slow convergence and susceptibility to local optima in traditional DRL training. Attached Figure Description

[0065] Figure 1 This is a flowchart of the distribution radio station topology optimization generation method based on deep reinforcement learning of the present invention;

[0066] Figure 2 This is a diagram showing the topology generation result of the distribution radio station area in a specific embodiment of the present invention. Detailed Implementation

[0067] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0068] Example 1

[0069] Step 1, Environment Modeling and State Encoding, transforms the physical parameters of the power grid into a state space that can be processed by deep reinforcement learning. By encoding the topology and dynamic operating features through a dual-channel matrix, the problem of separating the expression of topology information and load features in traditional methods is solved.

[0070] Step 2, hierarchical action space design: In view of the high-dimensional discrete characteristics of distribution network branch decision, the action space is decoupled into a two-dimensional combination of "main node selection - branch expansion decision". The decision complexity is reduced by mathematical mapping, while embedding the physical constraint rules of the power grid.

[0071] Step 3, Topology Update and User Allocation: The power grid topology is dynamically updated based on the action decision results. A multi-stage priority strategy is used to allocate user loads to ensure optimal matching between branch expansion and load distribution, thus solving the problem of topology and load separation optimization in traditional methods.

[0072] Step 4: Design a composite reward function. By combining nonlinear construction rewards, depth rewards, empty branch penalties, and other multi-dimensional indicators, the topology quality is quantitatively evaluated, guiding the agent to balance short-term gains (fast power supply) with long-term goals (reducing line loss).

[0073] Step 5: Network design and optimization. Construct a dual-channel convolutional neural network (CNN) to extract topological spatial features, combine it with a fully connected layer to achieve Q-value prediction, and improve the model's generalization ability through gradient pruning and weight decay strategies.

[0074] Step 6: Progressive training strategy. A three-stage training mechanism is adopted to dynamically adjust the exploration rate and reward weight. In the early stage, the focus is on topology exploration. In the middle stage, exploration and utilization are balanced. In the later stage, the strategy converges to the optimal strategy, which solves the problem of slow convergence and easy getting trapped in local optima in DRL training.

[0075] The specific implementation of each step is as follows:

[0076] Step 1: Environment modeling and state coding. The physical parameters of the power grid are transformed into a state space that can be processed by deep reinforcement learning. The topology structure and dynamic operation features are encoded by a dual-channel matrix, which solves the problem of separating the expression of topology information and load features in traditional methods.

[0077] 1.1 Parameter Initialization

[0078] According to the power supply radius Spacing between nodes Calculate the number of nodes in the main trunk line of the transformer area. :

[0079] ，

[0080] In the formula, This represents the floor function.

[0081] To avoid incomplete topology results, the following constraints are set:

[0082] ，

[0083] This means ensuring that the number of nodes on the main trunk line in the transformer area is at least 5.

[0084] ，

[0085] That is, the total number of nodes in the distribution area is greater than the number of nodes on the main line.

[0086] 1.2 Construction of Dual-Channel State Matrix

[0087] The dual-channel state matrix includes two channels representing the topology and operational characteristics, respectively.

[0088] For channel 1, the adjacency matrix A is used to represent the topology of the transformer area:

[0089] ，

[0090] In the formula, , Represents a node With nodes There are connecting lines. Represents a node With nodes There is no connecting line between them. A For dimension The adjacency matrix, This represents the total number of nodes in the region.

[0091] Corresponding to channel 2, it includes user distribution features and depth features, among which user distribution features for:

[0092] ，

[0093] Deep features For nodes The number of hops to the nearest trunk node is used to reflect the depth of branch extension and to suppress over-expansion.

[0094] The final state matrix is ​​shown in the following equation:

[0095] ，

[0096] Step 1 separates static topology and dynamic load features using a dual-channel approach, improving DRL learning efficiency. Dynamic normalization adapts to depth changes during topology expansion.

[0097] The main procedure for implementing step 2 is as follows:

[0098] class PowerGridEnv:

[0099] def __init_(self, params):

[0100] self.max_main_nodes = int(params['supply_radius'] / / params['node_interval'])

[0101] assert self.max_main_nodes>= 5, "Insufficient number of main nodes. Please adjust the power supply radius or node spacing."

[0102] assert params['max_total_nodes']>self.max_main_nodes, "The total number of nodes must be greater than the number of main nodes"

[0103] def _get_state(self):

[0104] state = np.zeros((2, self.max_total_nodes, self.max_total_nodes),dtype=np.float32)

[0105] state[0] = self.adj_matrix # Adjacency matrix

[0106] max_depth = max(self.branch_depth.values()) if self.branch_depth else1

[0107] for node in range(self.current_nodes):

[0108] if node <self.max_main_nodes:

[0109] capacity = 3 # Backbone node capacity

[0110] else:

[0111] capacity = 4.8 # Branch node capacity (increased by 20%)

[0112] state[1, 0, node] = self.user_distribution[node] / capacity # User distribution

[0113] state[1, 1, node] = self.branch_depth.get(node, 0) / (max_depth + 1e-5) # Node depth

[0114] return state

[0115] Step 2: Hierarchical motion space design

[0116] 1) Action space decomposition

[0117] The discrete action space is decoupled into a two-dimensional decision of "main node selection + branch extension length", reducing the action dimension. This is illustrated in the following equation:

[0118] ，

[0119] In the formula, The original action number, , representing a high-dimensional discrete action space; The maximum number of branches in a single trunk line. .

[0120] 2) Validity constraint verification

[0121] Invalid actions (such as nodes with 0 branches, exceeding the branch limit, etc.) are filtered out using rules. As shown in the following formula:

[0122] ，

[0123] In the formula, The current number of nodes is the number of nodes created, and l is the branch extension length, which is the number of nodes added in a single extension.

[0124] Step 2 reduces the original motion dimension by more than 50% using a two-dimensional motion space. Hard constraints ensure that all actions comply with power grid safety regulations.

[0125] The main procedure for implementing step 2 is as follows:

[0126] max_branch = int(0.5 * self.max_main_nodes)

[0127] main_node = action / / (max_branch + 1)

[0128] branch_type = action % (max_branch + 1)

[0129] def _is_valid(self, main_node, branch_type):

[0130] if main_node == 0 and branch_type>0:

[0131] return False # Node 0 is prohibited from extending branches

[0132] if self.main_branches[main_node]>= 5:

[0133] return False # The number of branches in a single main trunk exceeds the limit.

[0134] if self.current_nodes + branch_type>self.max_total_nodes:

[0135] return False # Exceeded total node capacity

[0136] return True

[0137] Step 3: Topology Update and User Assignment

[0138] 3.1) Branch expansion algorithm

[0139] Create a new node based on the action decision, and update the adjacency matrix and node depth. As shown in the following equation:

[0140] ，

[0141] In the formula, Numbering is the numbering of the main node. To extend the length of the branch, The node depth.

[0142] 3.2) User allocation strategy

[0143] Based on real-time load forecasting and dynamic priority adjustment, intelligent allocation of user load is achieved, ensuring balanced operation of the power grid.

[0144] Compute node load rate :

[0145] ，

[0146] In the formula, For nodes Current load power, For nodes Rated capacity.

[0147] Then, node priority is evaluated:

[0148] ，

[0149] In the formula, This represents the node depth (0 for trunk nodes and hop count for branch nodes). This is the voltage deviation coefficient. ， ， These are weighting coefficients, which are dynamically adjusted through reinforcement learning.

[0150] The dynamic weights are adjusted as follows:

[0151] High load scenarios >0.8): Increase the α weight to prioritize low-load nodes.

[0152] Low voltage scenarios ( >5%): Increase the weight of γ and prioritize nodes with stable voltage.

[0153] The user access principle is as follows:

[0154] New users are connected to the node with the highest priority. If multiple nodes have the same priority, the node with the smallest depth (closest to the trunk) is selected. When the node load rate... When the value is greater than 0.9, new user access is automatically rejected, and branch capacity expansion is triggered. If the node voltage deviation... If the percentage is greater than 10%, remove the node from the candidate list.

[0155] The automatic capacity expansion mechanism is set as follows: The trigger condition is that three consecutive users are denied access due to node overload. When the condition is met, the backbone node m with the largest available capacity is selected to expand the branch, and the branch length is... =1~3.

[0156] Step 4: Design a composite reward function to quantify the quality of power grid operation through multi-dimensional reward items, balance economy, stability and user experience, and guide the agent to generate the optimal topology.

[0157] The reward function is designed as follows:

[0158] ，

[0159] in: As a stability bonus:

[0160] ，

[0161] In the formula, The nominal value is taken as 400V. The node voltage is used to ensure that the closer the node voltage is to the nominal value, the higher the reward.

[0162] Economic incentives:

[0163] ，

[0164] ，

[0165] In the formula, Let be the equivalent resistance of the i-th line. Let be the current value of the i-th line.

[0166] Risk penalty:

[0167] ，

[0168] In the formula, Let i be the load rate of node i. Let j be the current value of line j. Let be the rated current carrying capacity of line j.

[0169] This is to punish overload phenomena.

[0170] Step 5: Design and optimization of dual-channel convolutional neural network. Construct a dual-channel convolutional neural network (CNN) to extract topological spatial features. Combine it with a fully connected layer to achieve Q-value prediction. Improve the model's generalization ability through gradient pruning and weight decay strategies to solve the problem of insufficient topological feature extraction in traditional DQN networks.

[0171] 5.1) Network Architecture Design

[0172] Dual-channel input processing:

[0173] Input1: Adjacency matrix A∈ ;

[0174] Input2: Feature matrix F∈ (User distribution + node depth);

[0175] Table 1. Network structure of a dual-channel convolutional neural network

[0176] Network layer Output Dimension Input (2, N, N) Conv2d → ReLU → BN (32, N, N) Conv2d → ReLU (64, N, N) AdaptiveAvgPool2d (64, 8, 8) Flatten 4096 (i.e., 64×8×8) Linear 512 Dropout 512 Linear 256 Output num_actions

[0177] The designed dual-channel convolutional neural network has the following design features: the input is a dual-channel N×N matrix (i.e., input1 and input2), and spatial features are extracted through two convolutional layers (using 32 and 64 3×3 convolutional kernels respectively, preserving resolution and introducing ReLU nonlinear activation). After adaptive average pooling, the output is uniformly converted into a 64-channel 8×8 feature map, which is then flattened and gradually reduced in dimensionality through fully connected layers (4096→512→256). After the 512-dimensional layer, 30% Dropout is added to prevent overfitting. The final output dimension is the size of the action space.

[0178] 5.2) Optimization Strategy

[0179] Gradient clipping mechanism:

[0180] ，

[0181] In the formula: The gradient represents the direction of parameter updates during backpropagation. This is the gradient clipping threshold used to prevent gradient explosion.

[0182] Weight decay:

[0183] ，

[0184] In the formula: This is the regularization coefficient, used to control model complexity and suppress overfitting; This refers to network weights.

[0185] The loss function is designed as follows:

[0186] ，

[0187] In the formula: To predict the Q-value, which is the output of the deep reinforcement learning network; B represents the target Q value, i.e., the target network computed value, and B represents the batch size.

[0188] Step 6: Progressive training strategy, employing a three-stage training mechanism to dynamically adjust the exploration rate ( ) and reward weight ( The early stage focuses on topology exploration, the middle stage balances exploration and utilization, and the later stage converges to the optimal strategy, solving the problem of slow convergence and easy getting trapped in local optima in DRL training.

[0189] The exploration rate decay formula is shown below:

[0190] ，

[0191] In the formula, Let be the exploration rate in round t, used to control the proportion of random actions; t is the training round.

[0192] To help deep reinforcement learning models find a balance between complexity and generalization ability and avoid overfitting, the stage weight adjustment rules are designed as follows:

[0193] ，

[0194] Finally, to reduce invalid training rounds, an early stopping mechanism is implemented. Training stops and the generated topology is output when the following equation is satisfied:

[0195] ，

[0196] In the formula, As a reward value, For the first Rewards at the next epoch. for Next epoch The average reward over one epoch.

[0197] Example 2

[0198] This embodiment is a specific implementation example of the method described in Embodiment 1 above for generating distribution network topology. The result of the network topology generation is as follows: Figure 2As shown, by simply inputting three parameters—the number of users in the distribution area, the power supply radius, and the node spacing—the optimal topology that meets the relevant conditions can be generated through deep reinforcement learning optimization.

[0199] The deep reinforcement learning (DRL) driven distribution area topology optimization method proposed in this invention generates, through dynamic interaction between the agent and the environment, a method that generates, for example, a method for optimizing distribution area topology. Figure 2 The topology presented demonstrates high reliability, low loss, and balanced scalability. Compared to traditional heuristic algorithms, this method achieves breakthrough improvements in core indicators such as structural rationality, economy, and scalability, as shown in the table below.

[0200] Table 2 Comparison of optimization results between the topology optimization method of this invention and the traditional genetic algorithm

[0201]

[0202] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for generating distribution radio station topology optimization based on deep reinforcement learning, characterized by the following steps: include: The physical parameters of the distribution network, including the number of users in the distribution area, the power supply radius, and the node spacing, are obtained. These physical parameters are then transformed into a state space, and the topology and dynamic operating characteristics are encoded using a dual-channel state matrix. The dual-channel state matrix includes two channels representing the topology and dynamic operating characteristics, respectively. It is expressed as follows: , In the formula, It is an adjacency matrix. User distribution characteristics, For depth features, This represents the total number of nodes in the distribution area. Adjacency matrix Used to represent the topology of the transformer area: dual-channel state matrix , In the formula, , Represents a node With nodes There are connecting lines. Represents a node With nodes There is no connecting line between them; node User distribution characteristics for: , Deep features For nodes Hop count to the nearest backbone node; A dual-channel state matrix is ​​input into a deep reinforcement learning network. The action space of distribution network branch decisions is decoupled into a two-dimensional decision combination of backbone node selection and branch extension length. The agent dynamically updates the grid topology based on the action decision results and uses a multi-stage priority strategy to allocate user loads. This multi-stage priority strategy allocates user loads based on real-time load forecasting and dynamic priority adjustment to achieve intelligent allocation of user loads and ensure balanced grid operation, as detailed below: Compute node load rate : , In the formula, For nodes Current load power, For nodes Rated capacity; Perform node priority evaluation: , In the formula, The node depth is represented by 0 for trunk nodes and hop count for branch nodes. For voltage deviation; ， ， These are weighting coefficients, which are dynamically adjusted through reinforcement learning. The user access principle is: new users are accessed to the node with the highest priority. If multiple nodes have the same priority, the node with the smaller depth is selected. When node load rate When the load rate exceeds the set node load threshold, new user access is rejected and branch expansion is triggered. The branch expansion mechanism is as follows: when a set number of users are rejected due to node overload, the main node with the largest available capacity is selected to expand the branch. If the node voltage deviation If the node voltage deviation exceeds the set threshold, the node will be removed from the candidate list. A deep reinforcement learning network is configured with a composite reward function to guide the agent to balance fast power supply and reduced line loss. A progressive training strategy is used to train the deep reinforcement learning network until convergence, and the output generates the optimal topology structure.

2. The method for generating distribution radio station topology optimization based on deep reinforcement learning according to claim 1, characterized in that, The deep reinforcement learning network described above extracts topological space features through a dual-channel convolutional neural network and performs [further processing] through fully connected layers. Value prediction is performed, and gradient pruning and weight decay strategies are used to improve the model's generalization ability. The adjacency matrix and the feature matrix containing user distribution and node depth in the input dual-channel state matrix are used to extract spatial features through two convolutional layers with different kernel sizes, and then the feature map is output uniformly through adaptive average pooling. The feature map is flattened and dimensionality is gradually reduced through three fully connected layers. A random deactivation layer is added after the second fully connected layer to prevent overfitting. The final output dimension is the size of the action space. The loss function is designed as follows: , In the formula: For prediction The value is the output of the deep reinforcement learning network; For the goal The value is the target network's calculated value, and B is the batch size.

3. The method for generating distribution radio station topology optimization based on deep reinforcement learning according to claim 1, characterized in that, The reward function of the deep reinforcement learning network is shown in the following equation: , , , , , In the formula, As a stability reward, This is the nominal voltage value. For the distribution of radio station area number Node voltage; As an economic reward, For the first The equivalent resistance of the line. For the first The current value of each line; As a risk penalty, For nodes load rate, For the line The current value, For the line Rated current carrying capacity.

4. The distribution network topology optimization generation method based on deep reinforcement learning according to claim 3, characterized in that, The deep reinforcement learning network employs a three-stage training mechanism to dynamically adjust the exploration rate. With reward weight ; The exploration rate decay formula is shown below: , In the formula, For the first The exploration rate of the wheel is used to control the proportion of random actions; For training rounds; As the number of training rounds increases, the reward weights for each training phase are progressively reduced. : An early stop mechanism is set up so that training stops and the topology is generated when the sum of the absolute values ​​of the differences between the reward values ​​of each round and the average reward value in the rounds preceding the previous round is less than a set reward value difference threshold.

5. The method for generating distribution radio station topology optimization based on deep reinforcement learning according to claim 1, characterized in that, The two-dimensional decision combination of backbone node selection and branch extension length is represented as follows: , In the formula, This represents the floor function; The original action number, , representing a high-dimensional discrete action space; The maximum number of branches in a single trunk line. , This refers to the number of nodes on the main trunk line.

6. The method for generating distribution radio station topology optimization based on deep reinforcement learning according to claim 5, characterized in that, The action space is embedded with power grid physical constraint rules to verify the validity of the constraints. Invalid actions are filtered out through the rules, as shown in the following formula: , In the formula, The current number of nodes is the number of nodes created, and l is the branch extension length, which is the number of nodes added in a single extension.

7. The method for generating distribution radio station topology optimization based on deep reinforcement learning according to claim 1, characterized in that, The dynamic update of the power grid topology based on action decision results includes: creating new nodes according to action decisions, and updating the adjacency matrix and node depth, as shown in the following formula: , In the formula, Numbering is the numbering of the main node. For the new node, For the current node, To extend the length of the branch, The node depth.

8. The method for generating distribution radio station topology optimization based on deep reinforcement learning according to claim 1, characterized in that, The specific steps for dynamically adjusting the weighting coefficients in the node priority evaluation are as follows: In high-load scenarios, where the average node load rate exceeds a set average load rate threshold, the weighting coefficient is increased. α Prioritize low-load nodes; In low-voltage scenarios, where the average voltage deviation at nodes exceeds a set average voltage deviation threshold, the weighting coefficient is increased. γ Nodes with stable voltage should be selected first.

Images