Power distribution network flexible resource cluster division model training method, application method and system

Abstract

The invention provides a power distribution network flexible resource cluster division model training method, application method and system, and the method comprises the steps: carrying out the initialization of a random forest algorithm based on the operation characteristics of a power distribution network; on the basis of the environment state of the power distribution network, an belonging-greedy algorithm is adopted, cluster numbers are repeatedly given to all nodes except the first node, and the state transition experience given with the cluster numbers is stored in an experience playback pool until the number of the state transition experiences in the experience playback pool reaches an experience number threshold value; randomly extracting a plurality of state transition experiences from the experience playback pool to train a value function of a random forest algorithm, and taking the trained value function as a power distribution network flexible resource cluster division model; distributing a cluster number for each node according to the network structure characteristics of the power distribution network and the value function of each agent; by intelligently distributing and clustering the resources in the power distribution network, the operation efficiency and reliability of the power distribution network can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power distribution network operation control, and particularly relates to a power distribution network flexible resource cluster division model training method, an application method and a system. BACKGROUND

[0002] With the wide access of distributed power sources, electric vehicles, large-scale user-side resources, etc., the power distribution network needs higher flexibility to adapt to the volatility and uncertainty of these resources. This requires the power distribution network to dynamically and adaptively manage these resources to improve the operational efficiency and reliability of the power grid. In this context, the power distribution network in various places is effectively aggregating flexible resources such as distributed generators, controllable loads, and distributed energy storage facilities to build virtual power plants, enabling them to participate in power market transactions and grid regulation and operation, thereby improving overall energy management efficiency.

[0003] The construction of flexible resource clusters involves aggregating new energy generation equipment, energy storage equipment, and user-side controllable flexible loads into a controllable resource aggregate, which can fully leverage the complementary advantages of each resource and effectively participate in grid regulation. Cluster division, as the first step in implementing group regulation and control strategies, is self-evidently important. However, with the wide access of distributed power sources, electric vehicles, and large-scale user-side resources in the power distribution network, traditional cluster division methods face major challenges. These methods are often based on static models and assumptions, which cannot fully capture the volatility and uncertainty of resources and are difficult to adapt to real-time changes in grid operating conditions. For example, the output of photovoltaic and wind power generation is significantly affected by weather changes, the charging demand of electric vehicles fluctuates significantly at different times, and the response of user-side controllable loads is influenced by both price signals and user behavior patterns. These factors together make the supply-demand balance of the power grid more complex and place higher demands on cluster division strategies. Traditional cluster division methods may oversimplify these dynamic characteristics, ignoring the temporal and spatial correlations and interactions between resources, resulting in the failure to fully exploit the regulation potential of resources. In addition, traditional methods may also fail to fully utilize modern information technologies such as big data analysis, artificial intelligence, and real-time communication technologies, which could provide more accurate decision support for cluster division. SUMMARY

[0004] To solve the above problems in the prior art, the present application proposes a power distribution network flexible resource cluster division model training method, comprising:

[0005] Based on the operating characteristics of the power distribution network, the random forest algorithm is initialized;

[0006] Based on the distribution network environment, the ∈-greedy algorithm is used to repeatedly assign cluster numbers to all nodes except node 1, and store the state transition experience assigned to cluster numbers into the experience replay pool until the number of state transition experiences in the experience replay pool reaches the experience number threshold.

[0007] Multiple state transition experiences are randomly extracted from the experience replay pool to train the value function of the random forest algorithm, and the trained value function is used as the flexible resource cluster partitioning model of the distribution network.

[0008] Node 1 is the root node of the distribution network; the state transition experience includes the distribution network environment state before assigning a cluster number to the node, the action of assigning a cluster number to the node, the reward obtained by assigning a cluster number to the node, and the distribution network environment state after assigning a cluster number to the node.

[0009] Preferably, the initialization of the random forest algorithm based on the operating characteristics of the distribution network includes:

[0010] Set the hyperparameters of the current value random forest Q network in the random forest algorithm, and set the cluster number of node 1 to 1;

[0011] Randomly initialize the parameters of the current value random forest Q network, and initialize the parameters of the target value random forest Q network in the random forest algorithm with the same hyperparameters and parameters;

[0012] Initialize the experience replay pool;

[0013] Based on the operating characteristics of the distribution network, initialize the distribution network environment state;

[0014] The hyperparameters include one or more of the following: the number of decision trees, the maximum number of features considered at each split point, the maximum depth of the decision tree, the minimum number of samples required to split in the decision tree, the minimum number of samples in the leaf nodes of the decision tree, and whether bootstrap sampling or the criteria for selecting split features are used when constructing each tree; the distribution network environment status includes one or more of the following: the active and reactive power output of each flexible resource, the connection relationship between each bus, the resistance, reactance, conductance, and susceptance on the lines, and the power of the generator nodes or load nodes on each bus.

[0015] Preferably, based on the distribution network environment status, an ∈-greedy algorithm is used to assign cluster numbers to all nodes except node 1, and the state transition experience of assigning cluster numbers is stored in the experience replay pool, including:

[0016] Obtain the current status of the power distribution network environment;

[0017] Based on the power distribution network environment status and the current value random forest Q network, cluster numbers are assigned to all nodes except node 1 using the ∈-greedy algorithm.

[0018] Calculate various indicators of the distribution network after assigning cluster numbers, update the distribution network environment status after assigning cluster numbers, and calculate the reward obtained by assigning cluster numbers to nodes.

[0019] The distribution network environment state before assigning a cluster number to a node, the action of assigning a cluster number to a node, the reward obtained by assigning a cluster number to a node, and the distribution network environment state after assigning a cluster number to a node are stored as state transition experiences in the experience replay pool.

[0020] The action of assigning a cluster number to a node includes either assigning a cluster number to a node that is the same as that of its upstream node, or assigning a cluster number to a node that is the largest currently assigned cluster number plus one; the upstream node is the first node along the power flow source direction of all nodes except node 1.

[0021] Preferably, the cluster number is assigned to the node using the ∈-greedy algorithm, including:

[0022] The node is assigned a cluster number by randomly selecting an action with a probability less than a preset probability threshold ∈, and the node is assigned a cluster number by the action with the highest value in the current value random forest Q network with a probability greater than or equal to ∈.

[0023] Preferably, the step of randomly selecting multiple state transition experiences from the experience replay pool to train the value function of the random forest algorithm, and using the trained value function as the flexible resource cluster partitioning model for the distribution network, includes:

[0024] Randomly select multiple state transition experiences from the experience replay pool;

[0025] The current value random forest Q-network is trained multiple times using the aforementioned state transition experience;

[0026] Update the target value random forest Q network based on the current number of training iterations of the value random forest Q network;

[0027] Determine whether the preset termination condition has been met. If so, use the value function corresponding to the trained target value random forest Q network as the flexible resource cluster partitioning model for the distribution network and end the process. Otherwise, based on the distribution network environment state and the current value random forest Q network, assign cluster numbers to all nodes except node 1 using the ∈-greedy algorithm.

[0028] Preferably, training the current value random forest Q-network using the state transition experience includes:

[0029] The current value random forest Q network is trained using the distribution network environment state before assigning cluster numbers to nodes and the action of assigning cluster numbers to nodes in the state transition experience as inputs, and the reward obtained by assigning cluster numbers to nodes in the same state transition experience as output.

[0030] Preferably, updating the target value random forest Q-network based on the training count of the current value random forest Q-network includes:

[0031] Determine whether the current training iterations of the value random forest Q-network have reached the threshold and whether the current value random forest Q-network has converged:

[0032] If any of these conditions are met, then the parameters of the current value random forest Q network are copied to the target value random forest Q network; otherwise, the target value random forest Q network remains unchanged.

[0033] Preferably, the value function is determined by a preset objective function;

[0034] The objective function includes, but is not limited to, the active power network loss or renewable energy absorption rate after the distribution network participates in scheduling in this cluster division method.

[0035] Based on the same inventive concept, the present invention also provides a training system for a flexible resource cluster partitioning model of a distribution network, comprising: an initialization module, an experience acquisition module, and a training module;

[0036] The initialization module is used to initialize the random forest algorithm based on the operating characteristics of the power distribution network.

[0037] The experience acquisition module is used to assign cluster numbers to all nodes except node 1 using the ∈-greedy algorithm based on the distribution network environment status, and store the state transition experience assigned to the cluster numbers into the experience replay pool until the number of state transition experiences in the experience replay pool reaches the experience quantity threshold.

[0038] The training module is used to randomly extract multiple state transition experiences from the experience replay pool to train the value function of the random forest algorithm, and use the trained value function as the flexible resource cluster partitioning model of the distribution network.

[0039] Node 1 is the root node of the distribution network; the state transition experience includes the distribution network environment state before assigning a cluster number to the node, the action of assigning a cluster number to the node, the reward obtained by assigning a cluster number to the node, and the distribution network environment state after assigning a cluster number to the node.

[0040] Preferably, the initialization module is specifically used for:

[0041] Set the hyperparameters of the current value random forest Q network in the random forest algorithm, and set the cluster number of node 1 to 1;

[0042] Randomly initialize the parameters of the current value random forest Q network, and initialize the parameters of the target value random forest Q network in the random forest algorithm with the same hyperparameters and parameters;

[0043] Initialize the experience replay pool;

[0044] Based on the operating characteristics of the distribution network, initialize the distribution network environment state;

[0045] The hyperparameters include one or more of the following: the number of decision trees, the maximum number of features considered at each split point, the maximum depth of the decision tree, the minimum number of samples required to split in the decision tree, the minimum number of samples in the leaf nodes of the decision tree, and whether bootstrap sampling or the criteria for selecting split features are used when constructing each tree; the distribution network environment status includes one or more of the following: the active and reactive power output of each flexible resource, the connection relationship between each bus, the resistance, reactance, conductance, and susceptance on the lines, and the power of the generator nodes or load nodes on each bus.

[0046] Preferably, the experience acquisition module, based on the distribution network environment status, uses an ∈-greedy algorithm to assign cluster numbers to all nodes except node 1, and stores the state transition experience assigned to the cluster numbers in the experience replay pool, including:

[0047] Obtain the current status of the power distribution network environment;

[0048] Based on the power distribution network environment status and the current value random forest Q network, cluster numbers are assigned to all nodes except node 1 using the ∈-greedy algorithm.

[0049] Calculate various indicators of the distribution network after assigning cluster numbers, update the distribution network environment status after assigning cluster numbers, and calculate the reward obtained by assigning cluster numbers to nodes.

[0050] The distribution network environment state before assigning a cluster number to a node, the action of assigning a cluster number to a node, the reward obtained by assigning a cluster number to a node, and the distribution network environment state after assigning a cluster number to a node are stored as state transition experiences in the experience replay pool.

[0051] The action of assigning a cluster number to a node includes either assigning a cluster number to a node that is the same as that of its upstream node, or assigning a cluster number to a node that is the largest currently assigned cluster number plus one; the upstream node is the first node along the power flow source direction of all nodes except node 1.

[0052] Preferably, the experience acquisition module assigns cluster numbers to nodes using an ∈-greedy algorithm, including:

[0053] The node is assigned a cluster number by randomly selecting an action with a probability less than a preset probability threshold ∈, and the node is assigned a cluster number by the action with the highest value in the current value random forest Q network with a probability greater than or equal to ∈.

[0054] Preferably, the training module is specifically used for:

[0055] Randomly select multiple state transition experiences from the experience replay pool;

[0056] The current value random forest Q-network is trained multiple times using the aforementioned state transition experience;

[0057] Update the target value random forest Q network based on the current number of training iterations of the value random forest Q network;

[0058] Determine whether the preset termination condition has been met. If so, use the value function corresponding to the trained target value random forest Q network as the flexible resource cluster partitioning model for the distribution network and end the process. Otherwise, based on the distribution network environment state and the current value random forest Q network, assign cluster numbers to all nodes except node 1 using the ∈-greedy algorithm.

[0059] Preferably, the training module uses the state transition experience to train the current value random forest Q-network, including:

[0060] The current value random forest Q network is trained using the distribution network environment state before assigning cluster numbers to nodes and the action of assigning cluster numbers to nodes in the state transition experience as inputs, and the reward obtained by assigning cluster numbers to nodes in the same state transition experience as output.

[0061] Preferably, the training module updates the target value random forest Q-network based on the number of training iterations of the current value random forest Q-network, including:

[0062] Determine whether the current training iterations of the value random forest Q-network have reached the threshold and whether the current value random forest Q-network has converged:

[0063] If any of these conditions are met, then the parameters of the current value random forest Q network are copied to the target value random forest Q network; otherwise, the target value random forest Q network remains unchanged.

[0064] Preferably, the value function in the training module is determined by a preset objective function;

[0065] The objective function includes, but is not limited to, the active power network loss or renewable energy absorption rate after the distribution network participates in scheduling in this cluster division method.

[0066] This invention also provides a method for applying a flexible resource cluster partitioning model for power distribution networks, including:

[0067] All nodes in the distribution network that can be divided into clusters are regarded as intelligent agents. Each intelligent agent includes a state space, an action space, a current value random forest Q network, and a pre-trained value function. The state space includes the upper and lower limits of the output of all adjustable flexibility resources of each node in the distribution network, the installed capacity, and the cluster number of the upstream node. The action space corresponds to the actions of the node to assign the cluster number.

[0068] Based on the network structure characteristics of the power distribution network and the value function of each agent, a cluster number is assigned to each node.

[0069] Preferably, the step of assigning a cluster number to each node based on the network structure characteristics of the distribution network and the value function of each agent includes:

[0070] Based on the network structure characteristics of the distribution network, find the upstream nodes of all nodes;

[0071] Based on the cluster number of the upstream node, the cluster number that can be assigned to each node is obtained;

[0072] Compare the value functions of the agents corresponding to each node after assigning a cluster number, and use the cluster number with the highest value as the cluster number of that node.

[0073] Based on the same inventive concept, this invention also provides an application system for a flexible resource cluster partitioning model for power distribution networks, comprising: an intelligent agent module and a partitioning module;

[0074] The intelligent agent module is used to treat all nodes in the distribution network that can be divided into clusters as intelligent agents. Each intelligent agent includes a state space, an action space, a current value random forest Q network, and a pre-trained value function. The state space includes the upper and lower limits of the output of all adjustable flexibility resources of each node in the distribution network, the installed capacity, and the cluster number of the upstream node. The action space corresponds to the actions assigned to the cluster number by the node.

[0075] The partitioning module is used to assign a cluster number to each node based on the network structure characteristics of the power distribution network and the value function of each agent.

[0076] Preferably, the partitioning module is specifically used for:

[0077] Based on the network structure characteristics of the distribution network, find the upstream nodes of all nodes;

[0078] Based on the cluster number of the upstream node, the cluster number that can be assigned to each node is obtained;

[0079] For each node, the value function of the corresponding agent after assigning a cluster number to each node is compared, and the cluster number with the highest value is used as the cluster number of that node.

[0080] The present invention also provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0081] The memory is used to store one or more programs;

[0082] When the one or more programs are executed by the at least one processor, the aforementioned training method for the flexible resource cluster partitioning model of the distribution network, or the aforementioned application method for the flexible resource cluster partitioning model of the distribution network, is implemented.

[0083] The present invention also provides a computer-readable storage medium having an executable program stored thereon, wherein when the executable program is executed, it implements the training method for the flexible resource cluster partitioning model of the distribution network as described above, or the application method for the flexible resource cluster partitioning model of the distribution network as described above.

[0084] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0085] This invention provides a training method, application method, and system for a flexible resource cluster partitioning model in a distribution network. The method includes: initializing a random forest algorithm based on the operating characteristics of the distribution network; repeatedly assigning cluster numbers to all nodes except node 1 using a greedy algorithm based on the distribution network environment, and storing the state transition experiences assigned to cluster numbers in an experience replay pool until the number of state transition experiences in the pool reaches a threshold; randomly selecting multiple state transition experiences from the experience replay pool to train the value function of the random forest algorithm, and using the trained value function as the flexible resource cluster partitioning model for the distribution network; furthermore, treating all clusterable nodes in the distribution network as agents, and assigning a cluster number to each node based on the network structure characteristics of the distribution network and the value function of each agent. This invention helps improve the operating efficiency and reliability of the power grid by intelligently allocating and clustering resources in the distribution network. This technology can be applied to the resource optimization and scheduling strategies of power grid companies, especially addressing the flexibility characteristics of modern distribution networks containing distributed power sources, electric vehicles, and large-scale user-side resources. This invention enables the dynamic, adaptive, and purposeful allocation of flexible and adjustable resources in the distribution network, thereby guiding power grid companies to manage their flexible resources more effectively. Attached Figure Description

[0086] Figure 1 This is a schematic diagram of the training method for a flexible resource cluster partitioning model in a power distribution network provided by the present invention.

[0087] Figure 2 This is a schematic diagram illustrating the training process of the value function of an intelligent agent provided by the present invention.

[0088] Figure 3 This is a schematic diagram of the structure of a training system for a flexible resource cluster partitioning model in a power distribution network, provided by the present invention.

[0089] Figure 4 This is a schematic diagram of the application method of a flexible resource cluster partitioning model for power distribution networks provided by the present invention;

[0090] Figure 5 This is a schematic diagram illustrating the process of assigning network numbers to nodes provided by the present invention;

[0091] Figure 6 This is a schematic diagram illustrating the process of mapping the value of state-action pairs using a random forest Q-network, as provided by the present invention.

[0092] Figure 7 This is a schematic diagram of the application system structure of a flexible resource cluster partitioning model for power distribution networks provided by the present invention;

[0093] Figure 8 A flowchart illustrating an example of a flexible resource cluster partitioning method for a power distribution network provided by the present invention;

[0094] Figure 9 A schematic diagram of the first 100 training iterations of the RF-QN method used in AGC scheduling in a specific example of a flexible resource cluster partitioning method for a distribution network provided by the present invention.

[0095] Figure 10 A schematic diagram of the first 100 training iterations of the RF-QN method used in AVC scheduling in a specific example of a flexible resource cluster partitioning method for distribution networks provided by the present invention.

[0096] Figure 11 This is a schematic diagram of the pre-training comprehensive evaluation score of the RFQN method in a specific example of a flexible resource cluster partitioning method for distribution networks provided by the present invention.

[0097] Figure 12 This is a schematic diagram of the comprehensive evaluation score after training using the RFQN method in a specific example of a flexible resource cluster partitioning method for distribution networks provided by the present invention.

[0098] Figure 13 This is a schematic diagram of the cluster partitioning method for two time periods in a specific example of a flexible resource cluster partitioning method for a distribution network provided by the present invention.

[0099] Figure 14 A specific example of a flexible resource clustering method for a distribution network provided by this invention; a schematic diagram of the clustering method for 22 time periods;

[0100] Figure 15 An electronic device provided by the present invention. Detailed Implementation

[0101] This invention provides a novel cluster partitioning method that comprehensively considers the dynamic characteristics of resources, user response patterns, and the real-time operating status of the power grid. This new method will make virtual power plants more flexible and responsive, enabling them to better participate in market transactions and power grid regulation, thereby improving the overall operating efficiency and reliability of the power grid. It will provide power grid operators with more precise and real-time control tools, bring greater economic benefits to electricity market participants, and provide end users with a more stable and economical power supply.

[0102] Specifically, this invention proposes a reinforcement learning algorithm based on a random forest Q-network that can be applied to a dynamic cluster partitioning method for flexible resources. Through this dynamic cluster partitioning method, in scenarios where terminal computing power is sufficient and the market rules for aggregators frequently exchange resource management information are relatively well-established, the potential for flexible resource regulation can be fully explored. In this scenario, aggregators will no longer participate in the management of fixed resources, but will instead rely on computing power to participate in power grid dispatching, managing different flexible resources at different times, thus achieving more efficient resource utilization.

[0103] This invention focuses on distribution networks with a high proportion of flexible resources. Based on the distribution network topology, power data of each node, line parameters, and the maximum output range of flexible resources as state features, it employs a random forest Q-network model for multi-agent reinforcement learning training. This allows for the determination of an optimal dynamic cluster partitioning method with a set objective function, based on the action value information of different agents. The results of this invention not only provide power grid companies with scientific and reasonable guidance for the dynamic partitioning of flexible resources but also effectively improve the operational efficiency and resource utilization flexibility of distribution networks.

[0104] Example 1:

[0105] This invention provides a training method for a flexible resource cluster partitioning model in a power distribution network, such as... Figure 1 As shown, it includes:

[0106] Step A1: Initialize the random forest algorithm based on the operating characteristics of the distribution network;

[0107] Step A2: Based on the distribution network environment status, the ∈-greedy algorithm is used to repeatedly assign cluster numbers to all nodes except node 1, and the state transition experience assigned to cluster numbers is stored in the experience replay pool until the number of state transition experiences in the experience replay pool reaches the experience quantity threshold.

[0108] Step A3: Randomly select multiple state transition experiences from the experience replay pool to train the value function of the random forest algorithm, and use the trained value function as the flexible resource cluster partitioning model for the distribution network;

[0109] Among them, node 1 is the root node of the distribution network; the state transition experience includes the distribution network environment state before assigning a cluster number to the node, the action of assigning a cluster number to the node, the reward obtained by assigning a cluster number to the node, and the distribution network environment state after assigning a cluster number to the node.

[0110] Step A1 specifically includes: setting the hyperparameters of the current value random forest Q network in the random forest algorithm, and setting the cluster number of node 1 to 1;

[0111] Randomly initialize the parameters of the current value random forest Q network, and initialize the parameters of the target value random forest Q network in the random forest algorithm with the same hyperparameters and parameters;

[0112] Initialize the experience replay pool;

[0113] Based on the operating characteristics of the distribution network, initialize the distribution network environment state;

[0114] The hyperparameters include one or more of the following: the number of decision trees, the maximum number of features considered at each split point, the maximum depth of the decision tree, the minimum number of samples required to split in the decision tree, the minimum number of samples in the leaf nodes of the decision tree, and whether bootstrap sampling or the criteria for selecting split features are used when constructing each tree; the distribution network environment status includes one or more of the following: the active and reactive power output of each flexibility resource, the connection relationship between each bus, the resistance, reactance, conductance, and susceptance on the lines, and the power of the generator nodes or load nodes on each bus.

[0115] The current value Random Forest Q-Network represents a network used to estimate the expected reward of a state-action pair. At its core is the Q-function, based on random forest regression, which represents the predicted long-term reward obtained by performing a specific action given a state. The Q-function is expressed as... Where s represents the state and a represents the action. The Q function can be called the value function or the reward function, and we will not distinguish between them hereafter.

[0116] The target value random forest Q network is a copy of the current value random forest Q network, and its role is to provide a stable Q-value target.

[0117] The experience replay pool is a memory used to store the agent's experiences during interactions with the environment, including the state, action, reward, and next state for each step. Initializing the experience replay pool involves clearing the experience replay pool matrix; storing the experience replay pool involves sequentially storing the state, action, reward, and next state of that step into the experience replay pool matrix as a vector. The experience replay pool can be a matrix, a dictionary, or other data structures.

[0118] In this process, based on the operating characteristics of the distribution network, the initialization of the distribution network environment state is neither completely random nor completely fixed, but rather adopts a semi-randomized initialization strategy based on the actual operating characteristics of the power grid. Specifically: the cluster number of node 1 (the root node) is fixed and initialized to 1; the initial cluster numbers of the remaining nodes are generated using an ∈-greedy algorithm, which includes both random exploration components (randomly selecting inherited or newly created clusters with probability ∈) and strategic selection (selecting the optimal action predicted by the current RF-Q network with probability 1-∈). Other state parameters of the power grid (such as the upper and lower limits of node output) are set with initial values ​​based on historical operating data or typical scenarios to ensure that the initialization state is both representative and retains necessary randomness.

[0119] In step A2, based on the distribution network environment state, an ∈-greedy algorithm is used to assign cluster numbers to all nodes except node 1, and the state transition experience of assigning cluster numbers is stored in the experience replay pool, including:

[0120] Obtain the current status of the power distribution network environment;

[0121] Based on the power distribution network environment status and the current value random forest Q network, cluster numbers are assigned to all nodes except node 1 using the ∈-greedy algorithm.

[0122] Calculate various indicators of the distribution network after assigning cluster numbers, update the distribution network environment status after assigning cluster numbers, and calculate the reward obtained by assigning cluster numbers to nodes.

[0123] The distribution network environment state before assigning a cluster number to a node, the action of assigning a cluster number to a node, the reward obtained by assigning a cluster number to a node, and the distribution network environment state after assigning a cluster number to a node are stored as state transition experiences in the experience replay pool.

[0124] The action of assigning a cluster number to a node includes either assigning a cluster number to a node that is the same as that of its upstream node, or assigning a cluster number to a node that is the largest currently assigned cluster number plus one; the upstream node is the first node along the power flow source direction of all nodes except node 1.

[0125] Specifically, a node is assigned a cluster number by randomly selecting an action with a probability less than a preset probability threshold ∈, and a node is assigned a cluster number by selecting the action with the highest current value in the random forest Q-network with a probability greater than or equal to ∈. The value of ∈ can be set to 0.1 to 0.3.

[0126] In step A2, the state transition experiences stored in the experience replay pool are complete transition tuples (s t ,a t ,r t ,s t+1 ), where a t This represents the cluster number assignment action (inheritance or creation) for a single node at time t. Each complete cluster partitioning process performs the following operations in node order: First, based on the global state s at time t... t Select action a for a node t Calculate the immediate reward r brought by this action. t (Based on the change in the objective function), a new state s is observed. t+1 (This reflects the overall network state after the node cluster number changes, or in other words, the global state at time t+1), and this set of transition data is independently stored in the experience replay pool. Therefore, completing the assignment of values ​​to all nodes will generate N-1 experience data points (excluding the root node), where N is the total number of nodes. Although these data originate from the same round of partitioning, they are used as independent samples for training, thus maintaining the Markov property while improving data utilization. This design ensures that each node's decision is based on the latest network state, while achieving efficient learning through experience replay.

[0127] The objective function is set before training as needed, representing some of the interests of the power grid company, including but not limited to active power network losses and renewable energy absorption rates after the distribution network participates in dispatching according to this cluster partitioning method. Different cluster partitioning methods will result in different calculated values ​​for the same objective function within the same resource allocation of the distribution network. Network losses or renewable energy absorption rates can be used as a single optimization objective, or multiple indicators can be combined through weighted summation to form a comprehensive objective function, and even expanded to include other new optimization indicators in the future. This open design allows the solution to adapt to the differentiated needs of different power grid dispatching scenarios. For example, it can prioritize optimizing absorption rates in areas with high renewable energy penetration, focus on reducing network losses in older power grids, or adopt multi-objective trade-off optimization in complex scenarios, fully demonstrating the adaptability and scalability of the technical solution.

[0128] Step A2 requires multiple cluster number assignments, which is an iterative accumulation process. Each cluster number assignment generates a new batch of state transition experiences (e) and stores them in the experience replay pool. Only when the number of accumulated experiences (count) in the pool reaches a preset threshold (e.g., 10^4) will the subsequent training process be triggered. This design ensures the sufficiency and diversity of training data. Typically, the cluster number assignment step needs to be executed dozens to hundreds of times (the exact number depends on the power grid size and experience pool capacity) to accumulate enough training samples. This batch training method ensures both learning efficiency and stability of the training process.

[0129] The value function is determined by the objective function. The Random Forest Q-Network in this invention predicts the long-term value of state-action pairs. Its relationship with the objective function lies in the fact that the immediate reward calculated by the objective function is used to train the network, ensuring that the output action-action pair value reflects the prediction of the cumulative future benefits of the action in the current state, including effects such as reduced active power loss and increased renewable energy absorption. Specifically, the network uses states and actions from historical data as input to learn the correlation between immediate rewards and the optimal value of the next state. This allows it to select the action that maximizes value during runtime, ultimately achieving long-term synergy between dynamic cluster partitioning and grid optimization objectives.

[0130] The cluster partitioning scheme is divided into a training phase and a decision phase. In the training phase, the reward function is directly calculated from the objective function (such as active power loss or renewable energy absorption rate), representing the immediate feedback from the environment to the agent's actions, gradually bringing the value function of action pairs closer to the predicted long-term value. The trained value function is then placed into an experience pool, which contains the value of different actions under different states. In the decision phase, the knowledge in the experience pool is used to select the action that maximizes the long-term value under the given state, rather than simply pursuing a single-step reward.

[0131] Step A3 specifically includes: randomly sampling several data points (i.e., state transition experience) to train the current value random forest Q-network; this step needs to be performed multiple times;

[0132] Update the target value random forest Q network based on the current number of training iterations of the value random forest Q network;

[0133] Determine whether the preset termination condition has been met. If so, use the value function corresponding to the trained target value random forest Q network as the flexible resource cluster partitioning model for the distribution network and end the process. Otherwise, based on the distribution network environment state and the current value random forest Q network, assign cluster numbers to all nodes except node 1 using the ∈-greedy algorithm.

[0134] The current value random forest Q-network is trained using state transition experience, including:

[0135] The current value random forest Q-network is trained using the distribution network environment state before assigning cluster numbers to nodes and the action of assigning cluster numbers to nodes from the state transition experience as input, and the reward obtained by the node in the same state transition experience for assigning cluster numbers as output. In other words, the input data is state-action pairs, and the corresponding rewards for these state-action pairs are used as output.

[0136] The target value random forest Q-network is updated based on the number of training iterations of the current value random forest Q-network. This means that the parameters of the target value random forest Q-network remain unchanged each time the parameters of the current value random forest Q-network are updated, until a certain number of steps or a specific condition is met, at which point the parameters of the current value random forest Q-network are copied into the target network. The specific condition could be that the target value random forest Q-network is updated when the network converges to a certain extent, for example, when the average TD error (temporal difference error) of 10 consecutive training batches is less than 0.01.

[0137] In this embodiment, the termination condition is set to the training time exceeding a preset time threshold and the length of the Markov sequence during training exceeding a preset length threshold.

[0138] A specific training process for the value function of an agent is as follows: Figure 2 As shown, Figure 2 The middle Markov sequence e represents a single transition experience of the agent's interaction with the environment, typically containing a quadruple.<s,a,r,s'> (Current state, action, reward, next state), these experiences are stored in the experience replay pool for training. The number of e is used for Markov sequence length counting.

[0139] count: This corresponds to the "count" in the right-hand statement and is used to track one of the following two scenarios:

[0140] 1. Experience replay pool filling progress: Record the number of transferred experiences collected, and start sampling training only after reaching the set value (to avoid insufficient data in the early stage);

[0141] 2. Target network update cycle: Record the current network training steps, and update the target value random forest Q network parameters when count > N (e.g., N = 100).

[0142] Example 2:

[0143] This invention provides a training system for a flexible resource cluster partitioning model in a power distribution network, such as... Figure 3 As shown, it includes:

[0144] Initialization module, experience acquisition module, and training module;

[0145] The initialization module is used to initialize the random forest algorithm based on the operating characteristics of the power distribution network.

[0146] The experience acquisition module is used to assign cluster numbers to all nodes except node 1 using the ∈-greedy algorithm based on the distribution network environment status, and store the state transition experience assigned to the cluster numbers into the experience replay pool until the number of state transition experiences in the experience replay pool reaches the experience quantity threshold.

[0147] The training module is used to randomly extract multiple state transition experiences from the experience replay pool to train the value function of the random forest algorithm, and use the trained value function as the flexible resource cluster partitioning model of the distribution network.

[0148] Node 1 is the root node of the distribution network; the state transition experience includes the distribution network environment state before assigning a cluster number to the node, the action of assigning a cluster number to the node, the reward obtained by assigning a cluster number to the node, and the distribution network environment state after assigning a cluster number to the node.

[0149] Preferably, the initialization module is specifically used for:

[0150] Set the hyperparameters of the current value random forest Q network in the random forest algorithm, and set the cluster number of node 1 to 1;

[0151] Randomly initialize the parameters of the current value random forest Q network, and initialize the parameters of the target value random forest Q network in the random forest algorithm with the same hyperparameters and parameters;

[0152] Initialize the experience replay pool;

[0153] Based on the operating characteristics of the distribution network, initialize the distribution network environment state;

[0154] The hyperparameters include one or more of the following: the number of decision trees, the maximum number of features considered at each split point, the maximum depth of the decision tree, the minimum number of samples required to split in the decision tree, the minimum number of samples in the leaf nodes of the decision tree, and whether bootstrap sampling or the criteria for selecting split features are used when constructing each tree; the distribution network environment status includes one or more of the following: the active and reactive power output of each flexible resource, the connection relationship between each bus, the resistance, reactance, conductance, and susceptance on the lines, and the power of the generator nodes or load nodes on each bus.

[0155] Preferably, the experience acquisition module, based on the distribution network environment status, uses an ∈-greedy algorithm to assign cluster numbers to all nodes except node 1, and stores the state transition experience assigned to the cluster numbers in the experience replay pool, including:

[0156] Obtain the current status of the power distribution network environment;

[0157] Based on the power distribution network environment status and the current value random forest Q network, cluster numbers are assigned to all nodes except node 1 using the ∈-greedy algorithm.

[0158] Calculate various indicators of the distribution network after assigning cluster numbers, update the distribution network environment status after assigning cluster numbers, and calculate the reward obtained by assigning cluster numbers to nodes.

[0159] The distribution network environment state before assigning a cluster number to a node, the action of assigning a cluster number to a node, the reward obtained by assigning a cluster number to a node, and the distribution network environment state after assigning a cluster number to a node are stored as state transition experiences in the experience replay pool.

[0160] The action of assigning a cluster number to a node includes either assigning a cluster number to a node that is the same as that of its upstream node, or assigning a cluster number to a node that is the largest currently assigned cluster number plus one; the upstream node is the first node along the power flow source direction of all nodes except node 1.

[0161] Preferably, the experience acquisition module assigns cluster numbers to nodes using an ∈-greedy algorithm, including:

[0162] The node is assigned a cluster number by randomly selecting an action with a probability less than a preset probability threshold ∈, and the node is assigned a cluster number by the action with the highest value in the current value random forest Q network with a probability greater than or equal to ∈.

[0163] Preferably, the training module is specifically used for:

[0164] Randomly select multiple state transition experiences from the experience replay pool;

[0165] The current value random forest Q-network is trained multiple times using the aforementioned state transition experience;

[0166] Update the target value random forest Q network based on the current number of training iterations of the value random forest Q network;

[0167] Determine whether the preset termination condition has been met. If so, use the value function corresponding to the trained target value random forest Q network as the flexible resource cluster partitioning model for the distribution network and end the process. Otherwise, based on the distribution network environment state and the current value random forest Q network, assign cluster numbers to all nodes except node 1 using the ∈-greedy algorithm.

[0168] Preferably, the training module uses the state transition experience to train the current value random forest Q-network, including:

[0169] The current value random forest Q network is trained using the distribution network environment state before assigning cluster numbers to nodes and the action of assigning cluster numbers to nodes in the state transition experience as inputs, and the reward obtained by assigning cluster numbers to nodes in the same state transition experience as output.

[0170] Preferably, the training module updates the target value random forest Q-network based on the number of training iterations of the current value random forest Q-network, including:

[0171] Determine whether the current training iterations of the value random forest Q-network have reached the threshold and whether the current value random forest Q-network has converged:

[0172] If any of these conditions are met, then the parameters of the current value random forest Q network are copied to the target value random forest Q network; otherwise, the target value random forest Q network remains unchanged.

[0173] Preferably, the value function in the training module is determined by a preset objective function;

[0174] The objective function includes, but is not limited to, the active power network loss or renewable energy absorption rate after the distribution network participates in scheduling in this cluster division method.

[0175] Example 3:

[0176] This invention also provides an application method for a flexible resource cluster partitioning model in a distribution network, or a method for flexible resource cluster partitioning in a distribution network, such as... Figure 4 As shown, it includes:

[0177] Step B1: Treat all nodes in the distribution network that can be divided into clusters as agents. Each agent includes a state space, an action space, a current value random forest Q network, and a pre-trained value function. The state space includes the upper and lower limits of the output of all adjustable flexibility resources of each node in the distribution network, the installed capacity, and the cluster number of the upstream node. The action space corresponds to the actions assigned to the cluster number by the node.

[0178] Step B2: Assign a cluster number to each node based on the network structure characteristics of the distribution network and the value function of each agent.

[0179] In step B1, all nodes in the distribution network that can be divided into clusters are regarded as intelligent agents, which means that the same number of intelligent agents are set according to the number of nodes in the distribution network that can be divided into clusters.

[0180] The distribution network is radial, with voltage levels of 10kV or 400V. All nodes that can be clustered represent resource-based or non-resource-based nodes recognized by the power grid company as capable of being group-dispatched and controlled. These nodes can participate in power system dispatching in the form of aggregators or virtual power plants. Resource-based nodes include the flexible resources that the power grid dispatching department or aggregator dispatching unit can use to regulate power through various means; non-resource-based nodes do not contain controllable flexible resources and only serve as connection channels between different resource-based nodes.

[0181] The agent comprises a state space, an action space, a current-value random forest Q-network, and a pre-trained value function. The state space includes the upper and lower limits of output of all adjustable flexibility resources of each node in the distribution network, the installed capacity, and the cluster number of the upstream node. The action space corresponds to the actions assigned to the cluster number by the node. An upstream node is the first node along the power flow source direction of all nodes except node 1 in the radial network. Node 1 is typically designed as the root node (root node / power source node) of the entire network, i.e., the source of power supply (such as the PCC point or main substation connecting to the upper-level grid). Node 1 is the only node without an upstream node, as it is the source itself.

[0182] Each node can be assigned a cluster number, which is a positive integer less than the maximum number of clusters specified, representing which cluster the node belongs to.

[0183] The action space includes two types: one is that the cluster number of the node is the same as that of its upstream node; the other is that the cluster number of the node is the largest currently assigned cluster number plus one. Assigning a cluster number to a node is as follows: Figure 5 As shown.

[0184] The current value random forest Q-network (or RF-QN) is a mapping of the agent's value to different state-action pairs. This mapping is achieved by fitting the values ​​of corresponding states and actions using random forest regression. The well-fitted value function can then be considered a flexible resource cluster partitioning model for the distribution network. The process of fitting the values ​​of corresponding states and actions using random forest regression is as follows: Figure 6 As shown.

[0185] Step B2 specifically includes: assigning a cluster number to each node based on the network structure characteristics and the value function of each agent; finding the upstream nodes of all nodes based on the network structure; obtaining two possible cluster numbers that can be assigned to each node based on the cluster numbers of the upstream nodes; and then, by comparing the value functions of the agents corresponding to each node (RF-Q network), using the cluster number with the higher value as the cluster number of that node.

[0186] There are two possibilities for the cluster number: either the node's cluster number is the same as its upstream node's, or the node's cluster number is the largest currently assigned cluster number plus one. See [link to relevant documentation]. Figure 5 The default cluster number for node 1 is 1.

[0187] Example 4:

[0188] This invention also provides an application system for a flexible resource cluster partitioning model for power distribution networks, such as... Figure 7 As shown, it includes: an agent module and a partitioning module;

[0189] The intelligent agent module is used to treat all nodes in the distribution network that can be divided into clusters as intelligent agents. Each intelligent agent includes a state space, an action space, a current value random forest Q network, and a pre-trained value function. The state space includes the upper and lower limits of the output of all adjustable flexibility resources of each node in the distribution network, the installed capacity, and the cluster number of the upstream node. The action space corresponds to the actions assigned to the cluster number by the node.

[0190] The partitioning module is used to assign a cluster number to each node based on the network structure characteristics of the power distribution network and the value function of each agent.

[0191] Preferably, the partitioning module is specifically used for:

[0192] Based on the network structure characteristics of the distribution network, find the upstream nodes of all nodes;

[0193] Based on the cluster number of the upstream node, the cluster number that can be assigned to each node is obtained;

[0194] For each node, the value function of the corresponding agent after assigning a cluster number to each node is compared, and the cluster number with the highest value is used as the cluster number of that node.

[0195] Example 5:

[0196] This embodiment provides an example of a flexible resource cluster partitioning method for a distribution network, such as... Figure 8 As shown, it includes:

[0197] Step S1: Treat all nodes in the distribution network that can be divided into clusters as intelligent agents;

[0198] Step S2: Define the objective function for the cluster partitioning method;

[0199] Step S3: Set the reward of the reinforcement learning model according to the objective function, and train the agent's value function accordingly;

[0200] Step S4: Assign a cluster number to each node based on the network structure characteristics and the value function of each agent.

[0201] In step S1, all nodes in the distribution network that can be divided into clusters are regarded as intelligent agents, which means that the same number of intelligent agents are set according to the number of nodes in the distribution network that can be divided into clusters.

[0202] The distribution network is radial, with voltage levels of 10kV or 400V. All nodes that can be clustered represent resource-based or non-resource-based nodes recognized by the power grid company as capable of being group-dispatched and controlled. These nodes can participate in power system dispatching in the form of aggregators or virtual power plants. Resource-based nodes include the flexible resources that the power grid dispatching department or aggregator dispatching unit can use to regulate power through various means; non-resource-based nodes do not contain controllable flexible resources and only serve as connection channels between different resource-based nodes.

[0203] Each agent contains a specific state space, action space, RF-Q network, and reward function.

[0204] The state space, through a combination of dynamic and static parameters, captures real-time operating status (such as current adjustable potential) while preserving inherent network characteristics (such as resource capacity limitations).

[0205] The action space is determined by the real-time network state (such as recalculation required if the upstream node changes), rather than by fixed rules.

[0206] The value function of the action pair is fitted by random forest regression to predict long-term returns, relying on real-time training data.

[0207] The reward function is directly related to the real-time performance of the power grid, driving the agent to adapt to changes.

[0208] The state space includes the upper and lower limits of the output of all adjustable flexibility resources of each node in the distribution network and the installed capacity, as well as the cluster number of the upstream node, as shown in Equation (1).

[0209]

[0210] in, Represents the state space at time t. and This represents the upper and lower limits of the active power output of resource l at time t. Let represent the installed capacity of distributed resource l, c(i) represent the cluster number of the upstream node of node i, and ρ(i) represent the set of flexibility resources of node i. The upstream node refers to the first node along the power flow source direction of all nodes except node 1 in the radial network.

[0211] Each node can be assigned a cluster number, which is a positive integer less than the maximum number of clusters, representing which cluster the node belongs to. The preset maximum number of clusters is shown in equation (2).

[0212] N = max i∈B a(i), a(i)≤M (2)

[0213] Where M represents the preset maximum number of clusters, a(i) represents the cluster number to which node i belongs, and B represents the set of network nodes.

[0214] The action space includes two types: one is that the cluster number of this node is the same as that of its upstream node; the other is that the cluster number of this node is the largest currently assigned cluster number plus one. See [link to action space]. Figure 3 As shown in equation (3).

[0215]

[0216] in, This represents the action space of node i.

[0217] The RF-Q network is a mapping of the value of different state-action pairs of the agent, as shown in Equation (4).

[0218]

[0219] Where v represents the value of the state-action pair. Let 's' represent the regression function, or value function, of the RF-Q network, where 's' represents a specific state and 'a' represents a specific action. The mapping of different state-action pairs to their values ​​is achieved by fitting the values ​​of the corresponding states and actions using random forest regression. See [link to relevant documentation]. Figure 6 .

[0220] The reward function is determined by the objective function of step S2.

[0221] The objective function in step S2 represents some of the interests of the power grid company, including but not limited to active power network losses and renewable energy absorption rates after the distribution network participates in dispatching using this cluster partitioning method. Under the same resource allocation in the distribution network, different cluster partitioning methods will result in different calculated values ​​for the same objective function.

[0222] Participating in scheduling by using a cluster partitioning method means decomposing the power command of the distribution network into each cluster according to a specific ratio, and then each cluster uses its own method to solve and regulate the power output of its internal resources.

[0223] The active network loss is shown in equation (5).

[0224]

[0225] Where D1 represents the active network loss, T represents the total number of time periods, and ρ(i) is the set of flexibility resources in node i. This represents the active power output of the flexible resource l at time t when the distribution network responds to a power command. p represents the active power of the load at node i. PCC,t This represents the active power injected into the power grid at time t (PCC point, point of common coupling).

[0226] The renewable energy absorption rate is shown in equation (6).

[0227]

[0228] Where D2 represents the renewable energy absorption rate, ρ new (i) represents the set of distributed new energy sources in node i. This represents the maximum active power output of the flexible resource l at time t when the distribution network responds to a power command.

[0229] In step S3, the value function of the agent is trained, see [link to relevant documentation]. Figure 2 This includes the following steps:

[0230] Step S3-1: Set the hyperparameters of the Random Forest Q network (i.e., the current value Random Forest Q network), and set the cluster number of node 1 to 1.

[0231] Step S3-2: Initialize the current value random forest Q network and the target value random forest Q network;

[0232] Step S3-3: Initialize the experience replay pool;

[0233] Step S3-4: Initialize the distribution network environment state. Using the ∈-greedy algorithm, assign cluster numbers to all nodes except node 1 and store them in the experience replay pool.

[0234] Steps S3-5: When the experience replay pool reaches a certain quantity, randomly sample several data points from it to train the current value random forest Q network;

[0235] Steps S3-6: Update the target value random forest Q network based on the number of training iterations of the current value random forest Q network.

[0236] The hyperparameters of the Random Forest Q-Network include the number of decision trees, the maximum number of features considered at each split point, the maximum depth of the decision tree, the minimum number of samples required to split in the decision tree, the minimum number of samples in the leaf nodes of the decision tree, whether to use bootstrap sampling when building each tree, and the criteria for selecting split features.

[0237] The current value Random Forest Q-Network represents a network used to estimate the expected reward of a state-action pair. At its core is the Q-function, based on random forest regression, which represents the predicted long-term reward obtained by performing a specific action given a state. The Q-function is expressed as... Where s represents the state and a represents the action.

[0238] The target value random forest Q-network is a copy of the current value random forest Q-network, and its role is to provide a stable Q-value target.

[0239] The experience replay pool is a memory used to store the agent's experiences during interactions with the environment, including the state, action, reward, and next state for each step. Initializing the experience replay pool involves clearing the experience replay pool matrix; storing the experience replay pool involves sequentially storing the state, action, reward, and next state of that step into the experience replay pool matrix as a vector. The experience replay pool can be a matrix, a dictionary, or other data structures.

[0240] The environmental conditions of a distribution network include all the conditions required to calculate the distribution network, including the active and reactive power output of each flexibility resource, the connection relationships between each bus, the resistance, reactance, conductance, susceptance on the lines, and the power of the generator nodes and load nodes on each bus.

[0241] The ∈-greedy algorithm means assigning a cluster number to a node by randomly selecting an action with a low probability, and assigning a cluster number to a node by selecting an action with a higher value in the random forest Q network with a higher probability.

[0242] Sampling several data points to train a random forest Q-network of current value means using the state-action pairs of these data points as input to the random forest machine learning regression model, and using the corresponding rewards of these state-action pairs as output to train the random forest regression model.

[0243] The target value random forest Q network is updated based on the number of training iterations of the current value random forest Q network. This means that the parameters of the target network remain unchanged each time the parameters of the Q network are updated, until a certain number of steps or specific conditions are met, at which point the parameters of the Q network are copied into the target network.

[0244] Step S4 assigns a cluster number to each node based on the network structure characteristics and the value function of each agent. According to the network structure, the upstream nodes of all nodes are found. Based on the cluster numbers of the upstream nodes, two possible cluster numbers that can be assigned to each node are obtained. Then, based on the comparison of the value functions of the agents corresponding to each node (RF-Q network), the cluster number with the larger value is used as the cluster number of that node.

[0245] There are two possibilities for the cluster number: either the node's cluster number is the same as its upstream node's, or the node's cluster number is the largest currently assigned cluster number plus one. See [link to relevant documentation]. Figure 5 The default cluster number for node 1 is 1.

[0246] Example 6:

[0247] To verify the feasibility of the RF-QN (Random Forest Q-Network) method provided in this invention for dynamic cluster partitioning of flexible resources in distribution networks, Markov processes and the method proposed in this invention were used to train dynamic cluster partitioning methods for flexible resources participating in AGC (Automatic Generation Control) and AVC (Automatic Voltage Control) scheduling, respectively. The first 100 training iterations using the RF-QN method for AGC scheduling are as follows: Figure 9 As shown, the first 100 training iterations using the RF-QN method for AVC scheduling are as follows: Figure 10 As shown.

[0248] In this scheme, the core idea of ​​Markov processes is implicitly embedded in the design of the reinforcement learning framework, mainly reflected in: 1) the construction of each state transition experience e = (s, a, r, s') strictly follows the Markov property assumption, that is, the next state s' depends only on the current state s and action a; 2) the training process of the random forest Q network learns the state-action value function through the experience replay pool, which is essentially solving the Markov decision process (MDP) without explicitly modeling the state transition probabilities. Specifically... Figure 2 The generation and storage of the Markov sequence e are the key technical means to realize this process.

[0249] The training process using the RFQN method is in a fluctuating upward phase, and this fluctuation is also due to the use of the ∈-greedy algorithm. A comparison is also made with the model that used static partitioning before training, see [link to comparison]. Figures 11-12 Similar conclusions to those of the DQN method can be observed. Comparing the model with the pre-training model, it can be found that the RF-QN algorithm significantly improves the scores of the distribution network in participating in AGC and AVC scheduling. Because it is closer to the upper limit of the reward for iterative convergence, the dynamic cluster partitioning method is superior to the fixed static cluster partitioning method. Especially in time periods 2 and 22, the model finds better cluster partitioning methods, such as... Figures 13-14 As shown.

[0250] Example 7

[0251] like Figure 15As shown, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, processor, and transceiver component are connected via a bus; the memory can be used to store executable programs, and an exemplary executable program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be accessed and / or modified when instructions are executed.

[0252] The processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, and it is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to realize the corresponding method flow or corresponding function, so as to realize the steps of the training method or application method of the flexible resource cluster partitioning model of the distribution network in the above embodiments.

[0253] Example 8

[0254] Based on the same inventive concept, this invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory). This readable storage medium is a memory device within an electronic device used to store programs and data. It is understood that the storage medium here can include both built-in storage media within the electronic device and extended storage media supported by the electronic device. The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Loading and executing one or more instructions stored in the storage medium by the processor can implement the steps of the training method or application method of the flexible resource cluster partitioning model for a distribution network in the above embodiments.

[0255] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0256] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0257] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0258] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0259] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for training a power distribution network flexible resource cluster division model, characterized in that, The method comprises the following steps: initializing the random forest algorithm based on the operation characteristics of the power distribution network; assigning cluster numbers to all nodes except the first node based on the state of the power distribution network environment using an ε-greedy algorithm, repeatedly performing the operation multiple times, and storing the state transition experience of assigning cluster numbers into an experience replay pool until the number of state transition experiences in the experience replay pool reaches a threshold value; randomly selecting multiple state transition experiences from the experience replay pool to train the value function of the random forest algorithm, and using the trained value function as a flexible resource cluster division model for the power distribution network. The first node is the root node of the power distribution network. The state transition experience includes the state of the power distribution network before assigning cluster numbers to the node, the action of assigning cluster numbers to the node, the reward obtained by assigning cluster numbers to the node, and the state of the power distribution network after assigning cluster numbers to the node.

2. The power distribution grid flexible resource cluster partitioning model training method of claim 1, wherein, The method of initializing the random forest algorithm based on the operation characteristics of the power distribution network comprises the following steps: setting the hyperparameters of the current value random forest Q network in the random forest algorithm, and setting the cluster number of the first node to 1; randomly initializing the parameters of the current value random forest Q network, and initializing the parameters of the target value random forest Q network in the random forest algorithm with the same hyperparameters and parameters; initializing the experience replay pool; initializing the state of the power distribution network based on the operation characteristics of the power distribution network; The hyperparameters include one or more of the following: the number of decision trees, the maximum number of features considered at each split point, the maximum depth of the decision tree, the minimum number of samples required for splitting in the decision tree, the minimum number of samples in the leaf node of the decision tree, whether to use bootstrap sampling or select the standard of splitting features when building each tree; the state of the power distribution network includes one or more of the following: the active and reactive power output of each flexible resource, the connection relationship between buses, the resistance, reactance, conductance, susceptance on the line, and the power of the generator node or load node on each bus.

3. The power distribution grid flexible resource cluster partitioning model training method of claim 2, wherein, Based on the state of the power distribution network, an ε-greedy algorithm is used to assign cluster numbers to all nodes except the first node, and the state transition experience of assigning cluster numbers is stored in the experience replay pool, comprising: obtaining the current state of the power distribution network; assigning cluster numbers to all nodes except the first node based on the state of the power distribution network and the current value random forest Q network using an ε-greedy algorithm; calculating various indicators of the power distribution network after assigning cluster numbers, updating the state of the power distribution network after assigning cluster numbers, and calculating the reward obtained by assigning cluster numbers to the node; storing the state of the power distribution network before assigning cluster numbers to the node, the action of assigning cluster numbers to the node, the reward obtained by assigning cluster numbers to the node, and the state of the power distribution network after assigning cluster numbers to the node as state transition experience into the experience replay pool; The action of assigning cluster numbers to the node includes assigning the cluster number of the node to be consistent with the cluster number of its upstream node or assigning the cluster number of the node to be one more than the current largest cluster number. The upstream node is the first node in the direction of power flow source for all nodes except the first node.

4. The power distribution grid flexible resource cluster partitioning model training method of claim 3, wherein, The nodes are numbered by an ∈-greedy algorithm, including: The nodes are numbered by an action of randomly selecting the nodes with a probability less than a preset probability threshold ∈, and the nodes are numbered by an action of the highest value of the current value random forest Q network with a probability greater than or equal to ∈.

5. The power distribution grid flexible resource cluster partitioning model training method of claim 3, wherein, The value function of the random forest algorithm is trained by randomly selecting multiple state transition experiences from the experience replay pool, and the trained value function is used as a flexible resource cluster division model of the power distribution network. Multiple state transition experiences are randomly selected from the experience replay pool. The current value random forest Q network is trained multiple times by using the state transition experiences. The target value random forest Q network is updated according to the number of training times of the current value random forest Q network. It is determined whether a preset termination condition is reached, if yes, the value function corresponding to the trained target value random forest Q network is used as the flexible resource cluster division model of the power distribution network and the process is ended, otherwise, the nodes other than the node 1 are numbered by the ∈-greedy algorithm based on the power distribution network environment state and the current value random forest Q network.

6. The power distribution grid flexible resource cluster partitioning model training method of claim 5, wherein, The current value random forest Q network is trained by using the state transition experiences, including: The current value random forest Q network is trained by using the state transition experiences, including:

7. The power distribution grid flexible resource cluster partitioning model training method of claim 5, wherein, The target value random forest Q network is updated according to the number of training times of the current value random forest Q network, including: It is determined whether the number of training times of the current value random forest Q network reaches a number threshold and whether the current value random forest Q network converges: If either of the two is yes, the parameters of the current value random forest Q network are copied to the target value random forest Q network, otherwise, the target value random forest Q network remains unchanged.

8. The power distribution grid flexible resource cluster partitioning model training method of claim 5, wherein, The value function is determined by a preset target function; The target function includes but is not limited to active network loss or new energy consumption rate after the power distribution network participates in dispatching in the cluster division mode.

9. A power distribution network flexible resource cluster division model training system, characterized in that, It includes: An initialization module, an experience acquisition module and a training module; The initialization module is used to initialize the random forest algorithm based on the operating characteristics of the power distribution network; The experience acquisition module is used to repeatedly number all nodes other than the node 1 by the ∈-greedy algorithm based on the power distribution network environment state, and store the state transition experiences of numbering the nodes into an experience replay pool until the number of state transition experiences in the experience replay pool reaches an experience number threshold; The training module is used to train the value function of the random forest algorithm by randomly selecting multiple state transition experiences from the experience replay pool, and use the trained value function as the flexible resource cluster division model of the power distribution network. The node 1 is the root node of the power distribution network; the state transition experience includes the power distribution network environment state before the nodes are numbered, the action of numbering the nodes, the reward obtained by numbering the nodes, and the power distribution network environment state after the nodes are numbered. 10.The power distribution grid flexible resource cluster partitioning model training system of claim 9, wherein, The initialization module is specifically configured to: set the hyperparameters of the current value random forest Q network in the random forest algorithm, and set the cluster number of the node No. 1 to 1; randomly initialize the parameters of the current value random forest Q network, and initialize the parameters of the target value random forest Q network in the random forest algorithm with the same hyperparameters and parameters; initialize the experience replay pool; initialize the power distribution network environment state based on the operation characteristics of the power distribution network; The hyperparameters include one or more of the following: the number of decision trees, the maximum number of features considered at each split point, the maximum depth of the decision tree, the minimum number of samples required for splitting in the decision tree, the minimum number of samples in the leaf node of the decision tree, whether to use bootstrap sampling or select the standard of the partition feature when building each tree; the power distribution network environment state includes one or more of the following: the active and reactive power output of each flexibility resource, the connection relationship between buses, the resistance, reactance, conductance, susceptance on the line, and the power of the generator node or load node on each bus. 11.The power distribution grid flexible resource cluster partitioning model training system of claim 10, wherein, The experience acquisition module assigns cluster numbers to all nodes except node No. 1 based on the power distribution network environment state using the ε-greedy algorithm, and stores the state transition experience of assigning cluster numbers into the experience replay pool, including: obtain the current power distribution network environment state; assign cluster numbers to all nodes except node No. 1 based on the power distribution network environment state and the current value random forest Q network using the ε-greedy algorithm; calculate the indicators of the power distribution network after assigning cluster numbers, update the power distribution network environment state after assigning cluster numbers, and calculate the reward obtained by assigning cluster numbers to the nodes; store the power distribution network environment state before assigning cluster numbers to the nodes, the action of assigning cluster numbers to the nodes, the reward obtained by assigning cluster numbers to the nodes, and the power distribution network environment state after assigning cluster numbers to the nodes as state transition experience into the experience replay pool; The action of assigning cluster numbers to the nodes includes that the cluster number of the node is consistent with the upstream node or the cluster number of the node is one more than the currently assigned maximum cluster number; the upstream node is the first node in the direction of the power flow source of all nodes except node No.

1.

12. The power distribution grid flexible resource cluster partitioning model training system of claim 11, wherein, The experience acquisition module assigns cluster numbers to the nodes using the ε-greedy algorithm, including: randomly select the action of assigning cluster numbers to the nodes with a probability less than a preset probability threshold ε, and assign cluster numbers to the nodes with the action with the highest value of the current value random forest Q network with a probability greater than or equal to ε.

13. The power distribution grid flexible resource cluster partitioning model training system of claim 11, wherein, The training module is specifically configured to: randomly extract a plurality of state transition experiences from the experience replay pool; train the current value random forest Q network multiple times using the state transition experiences; update the target value random forest Q network according to the number of training times of the current value random forest Q network; determining whether a preset termination condition is reached, if yes, taking the value function corresponding to the trained target value random forest Q network as a flexible resource cluster division model of the power distribution network and ending, otherwise, retraining the current value random forest Q network based on the power distribution network environment state and the current value random forest Q network, and assigning cluster numbers to all nodes except the node 1 by using an ε-greedy algorithm.

14. The power distribution grid flexible resource cluster partitioning model training system of claim 13, wherein, The training module trains the current value random forest Q network by using the state transition experience, including: The current value random forest Q network is trained by taking the power distribution network environment state before the node is assigned with the cluster number and the action of assigning the cluster number to the node in the state transition experience as input, and taking the reward obtained by assigning the cluster number to the node in the same state transition experience as output.

15. The power distribution grid flexible resource cluster partitioning model training system of claim 13, wherein, The training module updates the target value random forest Q network according to the training times of the current value random forest Q network, including: determining whether the training times of the current value random forest Q network reach a times threshold and whether the current value random forest Q network converges: if any one is yes, copying the parameters of the current value random forest Q network to the target value random forest Q network; otherwise, keeping the target value random forest Q network unchanged.

16. The power distribution grid flexible resource cluster partitioning model training system of claim 13, wherein, The value function in the training module is determined by a preset target function; The target function includes but is not limited to the active network loss or new energy consumption rate of the power distribution network after participating in the dispatch in the cluster division mode.

17. A power distribution network flexible resource cluster partitioning model application method, characterized in that, including: all nodes of all divisible clusters in the power distribution network are regarded as agents, each agent includes a state space, an action space, a current value random forest Q network and a pre-trained value function; the state space includes the output upper and lower limits, the installed capacity of all adjustable flexible resources of each node of the power distribution network and the cluster number of the upstream node; the action space corresponds to the action of assigning the cluster number to the node; according to the network structure characteristics of the power distribution network and the value function of each agent, assigning a cluster number to each node.

18. The power distribution grid flexible resource cluster partitioning model application method of claim 17, wherein, According to the network structure characteristics of the power distribution network and the value function of each agent, assigning a cluster number to each node, including: finding all upstream nodes of all nodes according to the network structure characteristics of the power distribution network; obtaining the cluster number that can be assigned to each node according to the cluster number of the upstream node; for each node, respectively comparing the value functions of the corresponding agents after each node is assigned with a cluster number, and taking the cluster number with the maximum value as the cluster number of the node.

19. A power distribution network flexible resource cluster partitioning model application system, characterized in that, including: an agent module and a division module; The agent module is configured to regard all nodes of all divisible clusters in the power distribution network as agents, each agent includes a state space, an action space, a current value random forest Q network and a pre-trained value function; the state space includes the output upper and lower limits, the installed capacity of all adjustable flexible resources of each node of the power distribution network and the cluster number of the upstream node; the action space corresponds to the action of assigning the cluster number to the node; The division module is configured to assign a cluster number to each node according to the network structure characteristics of the power distribution network and the value function of each agent.

20. The power distribution grid flexible resource cluster partitioning model application system of claim 19, wherein, The division module is specifically configured to: find all upstream nodes of all nodes according to the network structure characteristics of the power distribution network; According to the cluster number of the upstream node, the cluster number that can be assigned to each node is obtained; For each node, the value functions of the corresponding agents after each node assigns a cluster number are compared respectively, and the cluster number with the maximum value is taken as the cluster number of the node.

21. An electronic device, comprising: Comprise: At least one processor and memory; The memory and the processor are connected through a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the power distribution network flexible resource cluster division model training method according to any one of claims 1 to 8 or the power distribution network flexible resource cluster division model application method according to any one of claims 17-18 is implemented.

22. A computer-readable storage medium, characterized in that, The execution program is stored thereon, and when the execution program is executed, the power distribution network flexible resource cluster division model training method according to any one of claims 1 to 8 or the power distribution network flexible resource cluster division model application method according to any one of claims 17-18 is implemented.

Images