Power distribution multi-zone load transfer method and device based on improved deep reinforcement learning

Abstract

The invention provides a power distribution multi-transformer-area load transfer method and device based on improved deep reinforcement learning, which can be used in the technical field of artificial intelligence, and the method comprises the steps: constructing a power distribution multi-transformer-area fault switching and load transfer model in a fault scene based on a power distribution multi-transformer-area structure according to the state data and unit cost of power distribution multi-transformer-areas; converting into an agent model in combination with a Markov decision process; based on an improved deep reinforcement learning algorithm, an intelligent agent model is trained, a load transfer module is constructed, adaptive fault switching is performed on real-time state quantities and random noise values of the power distribution multiple areas, an optimal load transfer action is generated, and rapid adaptive decision and accurate control of the power distribution multiple areas in a fault scene are realized. The adaptability to a high-dimensional dynamic operation environment is effectively improved; important load power supply is preferentially guaranteed under the condition of source load power imbalance, and the three-phase equilibrium state of the system is synchronously optimized, so that the power supply reliability and the system safety and stability are remarkably enhanced.

Description

Technical Field

[0001] This invention relates to the field of power distribution network fault handling technology, particularly to the field of artificial intelligence technology, and especially to a method and apparatus for transferring loads between multiple distribution areas based on improved deep reinforcement learning. Background Technology

[0002] With the deepening of energy structure transformation and the rapid development of new energy technologies, the penetration rate of distributed renewable energy, represented by wind power and photovoltaics, in distribution networks continues to rise. However, the inherent intermittency and volatility of new energy power generation pose new challenges to the safe and stable operation of distribution networks. When a distribution substation experiences a fault, the traditional single-substation operation mode often struggles to restore power in a timely manner, especially when the output of new energy sources is unstable, which can easily lead to power outages for critical loads and seriously affect power supply reliability. Against this backdrop, the multi-substation collaborative operation mode demonstrates significant advantages. By establishing flexible interconnections between substations, excess power from new energy sources in some substations can be promptly transferred to other substations, effectively achieving power sharing and ensuring continuous power supply to critical loads. It is worth noting that the multi-substation distribution system can operate in parallel with the main grid or switch to islanded operation mode when disconnected from the main grid. This flexible operation mode places higher demands on fault handling capabilities, especially in islanded operation mode, where achieving smooth fault switching between multiple substations and seamless load priority transfer becomes crucial to ensuring the reliability of power supply to critical loads. Therefore, researching technologies for smooth fault switching across multiple distribution areas and seamless load priority transfer is of great significance for improving the power supply reliability of distribution networks in environments with a high proportion of renewable energy access.

[0003] Existing technologies have conducted a series of studies on distribution network fault recovery and load transfer problems. Some existing technologies use traditional optimization algorithms for solutions, but these suffer from low computational efficiency and susceptibility to the "curse of dimensionality" when dealing with complex optimization problems in large-scale power grids. Other existing technologies use heuristic optimization algorithms to formulate recovery strategies. While these can optimize the recovery process to some extent, their fixed search strategies are not adaptable enough to high-dimensional and dynamically changing operating conditions, easily getting trapped in local optima and making it difficult to achieve optimal control. In addition, some advanced methods based on deep reinforcement learning have improved the convergence and robustness of the algorithms, but they still have significant shortcomings: on the one hand, these methods fail to fully consider the severe scenario where the source-side output may not be able to meet the total load demand under fault conditions, and lack precise control strategies based on load priority when there is a power deficit; on the other hand, during the load transfer process, they ignore the three-phase imbalance problem caused by the differentiated access of a large number of single-phase loads, which may lead to new risks such as equipment overload and voltage quality deterioration in practical engineering applications.

[0004] In summary, existing technologies have limitations in handling fault recovery in multi-distribution systems, especially in islanded operation, in terms of computational efficiency, dynamic adaptability, handling of source-load imbalance, and three-phase imbalance management. There is an urgent need for a comprehensive solution that can achieve smooth fault switching and seamless intelligent load transfer. Summary of the Invention

[0005] One objective of this invention is to provide a load transfer method for multiple distribution substations based on improved deep reinforcement learning. This method enables rapid adaptive decision-making and precise control of multiple distribution substations under fault scenarios, effectively improving adaptability to high-dimensional dynamic operating environments. By intelligently generating optimal load transfer strategies, it can prioritize power supply to critical loads in cases of source-load power imbalance and simultaneously optimize the three-phase balance of the system, thereby significantly enhancing power supply reliability and system safety and stability during islanded operation. Another objective of this invention is to provide a load transfer device for multiple distribution substations based on improved deep reinforcement learning. A further objective of this invention is to provide a computer-readable medium. A final objective of this invention is to provide a computer device.

[0006] To achieve the above objectives, this invention discloses a method for load transfer across multiple distribution areas based on improved deep reinforcement learning, comprising:

[0007] Based on the multi-distribution area structure, and according to the status data and unit cost of the multi-distribution area, a fault switching and load transfer model for the multi-distribution area under fault scenarios is constructed.

[0008] By incorporating Markov decision processes, the fault switching and load transfer model for multiple distribution areas under fault scenarios is transformed into an intelligent agent model.

[0009] The agent model is trained based on an improved deep reinforcement learning algorithm to construct a load transfer model.

[0010] By using a load transfer model, adaptive fault switching is performed on the real-time status variables and random noise values ​​of multiple distribution areas to generate the optimal load transfer action.

[0011] Preferably, the fault switching and load transfer model for multiple distribution areas under fault scenarios includes an objective function and constraints;

[0012] Based on the multi-distribution area structure, and according to the status data and unit cost of the multi-distribution area, a fault switching and load transfer model for the multi-distribution area under fault scenarios is constructed, including:

[0013] Based on the multi-distribution area structure, an objective function is established according to the status data and unit cost of the multi-distribution area.

[0014] Based on the state data, constraints are established, including power flow constraints, distributed generation output constraints, energy storage constraints, and node voltage constraints.

[0015] Preferably, the agent model includes a state space, an action space, and a reward function;

[0016] Combining Markov decision processes, the model for fault switching and load transfer in multi-distribution areas under fault scenarios is transformed into an intelligent agent model, including:

[0017] A state space is generated based on the real-time power output, real-time load power demand, sectional switch status, tie switch status, node voltage, energy storage charge status, and branch power within the multiple distribution areas.

[0018] The action space is generated based on the active power of the disconnected load and the binary variables of the opening and closing states of the sectionalizing switch and the tie switch.

[0019] A reward function is generated based on the objective function and a preset penalty value.

[0020] Preferably, the agent model is trained based on an improved deep reinforcement learning algorithm to construct a load transfer model, including:

[0021] Cluster the samples in the pre-defined experience pool and determine the target cluster based on the observed samples;

[0022] Priority calculation is performed on each sample in the target cluster to determine the samples to be extracted.

[0023] By employing a dynamic soft update strategy, the network parameters of the agent model are iteratively updated based on the extracted samples to construct a load transfer model.

[0024] Preferably, priority calculation is performed on each sample in the target cluster to determine the samples to be extracted, including:

[0025] Calculate the error value and sampling frequency priority for each sample in the target cluster;

[0026] A comprehensive priority for each sample is generated based on the error value and sampling frequency priority of each sample.

[0027] The probability of each sample being drawn is generated based on the overall priority.

[0028] Based on the preset extraction conditions and the extraction probability, the sample to be extracted is determined.

[0029] Preferably, through a load transfer model, adaptive fault switching is performed on the acquired real-time status variables and random noise values ​​of multiple distribution areas to generate the optimal load transfer action, including:

[0030] The real-time status variables and random noise values ​​of multiple distribution areas are input into the load transfer model to perform adaptive fault switching and generate the optimal load transfer action.

[0031] Execute the optimal load transfer action to complete the seamless load transfer operation.

[0032] This invention also discloses a power distribution multi-area load transfer device based on improved deep reinforcement learning, comprising:

[0033] The basic model building unit is used to build a fault switching and load transfer model for multiple distribution substations under fault scenarios based on the structure of multiple distribution substations and the status data and unit cost of the multiple distribution substations.

[0034] The intelligent agent model conversion unit is used to combine Markov decision process to convert the power distribution multi-area fault switching and load transfer model in fault scenarios into an intelligent agent model;

[0035] The load transfer model training unit is used to train the agent model based on the improved deep reinforcement learning algorithm to build the load transfer model.

[0036] The optimal load transfer action generation unit is used to adaptively switch faults based on the real-time status variables and random noise values ​​of the acquired power distribution multiple transformer areas through the load transfer model, and generate the optimal load transfer action.

[0037] The present invention also discloses a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0038] The present invention also discloses a computer device, including a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, wherein the processor executes the program to implement the method described above.

[0039] The present invention also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method described above.

[0040] This invention is based on a multi-distribution area structure. Based on the state data and unit cost of the multi-distribution area, it constructs a fault switching and load transfer model for multi-distribution area under fault scenarios. Combining a Markov decision process, the fault switching and load transfer model is transformed into an intelligent agent model. An improved deep reinforcement learning algorithm is used to train the intelligent agent model, constructing a load transfer model. Through the load transfer model, adaptive fault switching is performed on the acquired real-time state variables and random noise values ​​of the multi-distribution area, generating the optimal load transfer action. This enables rapid adaptive decision-making and precise control of the multi-distribution area under fault scenarios, effectively improving its adaptability to high-dimensional dynamic operating environments. By intelligently generating the optimal load transfer strategy, priority can be given to ensuring power supply to important loads under source-load power imbalance, while simultaneously optimizing the three-phase balance state of the system, thereby significantly enhancing the power supply reliability and system safety stability during islanded operation. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a multi-area load transfer method for power distribution based on improved deep reinforcement learning is provided in an embodiment of the present invention;

[0043] Figure 2 A flowchart illustrating another method for load transfer across multiple distribution areas based on improved deep reinforcement learning, provided as an embodiment of the present invention;

[0044] Figure 3 A comparison of convergence results between an improved DDPG algorithm, Deep Q-Network Learning (DQN), and the traditional DDPG algorithm, provided in an embodiment of the present invention;

[0045] Figure 4 This is a bar chart comparing the economic cost and imbalance of a multi-station distribution area after optimization of an improved DDPG algorithm provided in this embodiment of the invention with DQN and the traditional DDPG algorithm.

[0046] Figure 5 This is a bar chart comparing the optimized total load power supply recovery rate of multiple distribution areas and the recovery rate of different load levels between the improved DDPG algorithm provided in this embodiment of the invention and DQN and the traditional DDPG algorithm.

[0047] Figure 6This is a schematic diagram of a multi-zone load transfer device for power distribution based on improved deep reinforcement learning, provided in an embodiment of the present invention.

[0048] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] It should be noted that the power distribution multi-area load transfer method and device based on improved deep reinforcement learning disclosed in this application can be used in the field of artificial intelligence technology, or in any field other than artificial intelligence technology. The application field of the power distribution multi-area load transfer method and device based on improved deep reinforcement learning disclosed in this application is not limited.

[0051] To facilitate understanding of the technical solution provided in this application, the relevant content of the technical solution is first explained below. This invention addresses multi-fault scenarios within distribution substations by establishing a smooth fault switching and seamless load priority transfer model for multiple distribution substations. A multi-objective optimization objective function is constructed, comprehensively considering economic and power quality factors. Secondly, based on the constructed optimization model, the problem of seamless load priority transfer for multiple distribution substations is described as a Markov Decision Process (MDP). Then, an improved Deep Deterministic Policy Gradient (DDPG) algorithm is proposed to solve the MDP. The proposed method introduces the K-medoids clustering algorithm, intelligently classifying and organizing samples in the experience pool based on state space similarity. It also considers a priority experience replay mechanism that comprehensively considers sample sampling frequency and time difference error to update the main network parameters, and then uses a dynamic soft update strategy to optimize the target network parameter update process. Finally, the trained intelligent agent model is used to adaptively generate online strategies for smooth fault switching and seamless load priority transfer for multiple distribution substations. The fault smooth switching and load priority seamless power transfer strategy proposed in this invention can ensure priority power supply to more important loads, thereby improving the power quality of the system while reducing the economic cost of power transfer.

[0052] The traditional DDPG algorithm obtains the results during training. Experience data is stored in an experience pool, and a portion of this experience data is randomly selected as training samples to update network parameters. However, low-quality samples in the randomly selected data cannot efficiently converge the network parameters to their optimal values. This invention selects samples more relevant to the current strategy for parameter updates from the perspective of spatial similarity. It uses the K-medoids algorithm to cluster the data stored in the experience pool and selects samples for updating network parameters based on the similarity between the current strategy and the clustered experience.

[0053] The following example uses a power distribution multi-area load transfer device based on improved deep reinforcement learning as the execution subject to illustrate the implementation process of the power distribution multi-area load transfer method provided in this embodiment of the invention. It is understood that the execution subject of the power distribution multi-area load transfer method provided in this embodiment of the invention includes, but is not limited to, a power distribution multi-area load transfer device based on improved deep reinforcement learning.

[0054] Figure 1 A flowchart of a multi-distribution area load transfer method based on improved deep reinforcement learning provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0055] Step 101: Based on the multi-distribution area structure, construct a fault switching and load transfer model for the multi-distribution area under fault scenarios, according to the status data and unit cost of the multi-distribution area.

[0056] In this embodiment of the invention, for multi-fault scenarios within a distribution substation, a smooth fault switching and seamless load priority transfer model for multiple distribution substations is established. Taking into account both economic and power quality factors, a multi-objective optimization objective function is constructed. The multi-distribution substation fault switching and load transfer model under fault scenarios includes the objective function and constraints.

[0057] This invention establishes a system where adjacent distribution substations are connected via sectionalizing switches to achieve power complementarity between substations. For distribution substations located further apart, a backup power channel is formed through an optimized configuration of interconnecting switches. The entire multi-substation distribution system is connected to the upstream main grid via sectionalizing switches, forming a main grid-distribution network collaborative operation architecture. Considering extreme scenarios such as loss of main grid power supply and internal faults within distribution substations, the proposed multi-substation fault switching and load transfer strategy is as follows:

[0058] 1) During normal operation, each distribution substation draws power from the main grid through transformers. Relying on the frequency and voltage reference values ​​provided by the main grid, the distributed renewable energy (photovoltaic, wind turbine) in the distribution substation uses PQ control to regulate its active and reactive power output. While ensuring power supply to the loads in each substation, it also conducts bidirectional power interaction with the main grid.

[0059] 2) When a fault occurs in the main grid, the distribution substations connected to the main grid automatically disconnect the upstream sectionalizing switch, and multiple distribution substations operate independently. At this time, the micro gas turbine units configured in the distribution substations use V / F control to provide frequency and voltage reference values ​​for the distribution network, while the photovoltaic and wind turbines still use the PQ control strategy to operate in maximum power point tracking (MPPT) mode to maximize the output of renewable energy.

[0060] 3) When a fault occurs in the branch connecting the distribution area, the sectionalizing switch of the corresponding branch is disconnected to achieve fault isolation, and the reliability of load power supply is ensured by closing certain tie switches, thereby achieving smooth fault switching and seamless load priority transfer.

[0061] 4) Considering the uncertainty caused by the fluctuation of renewable energy output, establish a safety backup strategy, that is, if the power supply to important loads is still insufficient after the switch is turned off, the load shedding operation is performed to ensure the power supply reliability of higher priority loads.

[0062] Step 102: Combine MDP to convert the fault switching and load transfer model of multiple distribution areas in the fault scenario into an intelligent agent model.

[0063] In this embodiment of the invention, based on the objective function established in step 101, the problem of multi-distribution area fault switching and load transfer under fault scenarios is described as an MDP. Through continuous interaction and learning between the intelligent agent model and the environment, the maximum reward is sought, thereby finding the optimal load transfer strategy. The intelligent agent model includes a state space, an action space, and a reward function.

[0064] Step 103: Train the agent model based on the improved deep reinforcement learning algorithm to construct the load transfer model.

[0065] In this embodiment of the invention, the deep reinforcement learning algorithm is the Deep Deterministic Policy Gradient (DDPG) algorithm. Specifically, the constructed MDP is solved using an improved DDPG algorithm. The DDPG algorithm includes a master actor network and a target actor network, as well as a master critic network and a target critic network. The master actor network is responsible for outputting deterministic control actions, and the master critic network evaluates the state-action value function. The target network progressively tracks online network parameters through a soft update mechanism, providing a stable learning benchmark for policy optimization and value estimation, thereby effectively ensuring the training stability of the algorithm. The agent is trained offline using the improved DDPG algorithm to achieve optimal parameter fitting and construct a load transfer model.

[0066] Step 104: Using the load transfer model, adaptive fault switching is performed on the real-time status variables and random noise values ​​of the acquired power distribution multiple zones to generate the optimal load transfer action.

[0067] In this embodiment of the invention, when applied online, based on the trained load transfer model, the real-time status variables and random noise values ​​of multiple distribution areas are input, and the optimal load transfer action is output to complete the seamless load transfer operation.

[0068] In the technical solution provided by this invention, based on the multi-distribution area structure, a fault switching and load transfer model for the multi-distribution area under fault scenarios is constructed according to the status data and unit cost of the multi-distribution area. Combining a Markov decision process, the fault switching and load transfer model is converted into an intelligent agent model. An improved deep reinforcement learning algorithm is used to train the intelligent agent model, constructing a load transfer model. Through the load transfer model, adaptive fault switching is performed on the acquired real-time status variables and random noise values ​​of the multi-distribution area, generating the optimal load transfer action. This enables rapid adaptive decision-making and precise control of the multi-distribution area under fault scenarios, effectively improving its adaptability to high-dimensional dynamic operating environments. By intelligently generating the optimal load transfer strategy, priority can be given to ensuring power supply to important loads under source-load power imbalance, and the three-phase balance state of the system can be optimized simultaneously, thereby significantly enhancing the power supply reliability and system safety stability during islanded operation.

[0069] Figure 2 A flowchart of another distribution multi-area load transfer method based on improved deep reinforcement learning provided in this embodiment of the invention is shown below. Figure 2 As shown, the method includes:

[0070] Step 201: Based on the multi-distribution area structure, establish an objective function according to the status data and unit cost of the multi-distribution area.

[0071] In this embodiment of the invention, each step is performed by a power distribution multi-zone load transfer device based on improved deep reinforcement learning.

[0072] In this embodiment of the invention, the multi-distribution area structure includes a branch set, a sectionalizing switch set, a tie switch set, a distribution area set, and a load node set within the distribution area; the status data includes the active power on the branch, the reactive power on the branch, the branch end voltage, the branch resistance, the binary variables of the sectionalizing switch opening and closing status, the binary variables of the tie switch opening and closing status, the active power of the load node before the fault and the active power after the load transfer operation, the three-phase voltage of the distribution area, the average three-phase voltage, the three-phase positive and negative sequence current of the distribution area, and the currents of phases a, b, and c; the unit cost includes network loss cost, switch operation loss cost, and load shedding cost.

[0073] In this embodiment of the invention, the objective function F of the multi-distribution area fault switching and load transfer model under fault scenarios is as follows:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080] Where F is the objective function; F1 is the load transfer cost; F2 is the system imbalance after load transfer; C net C switch C load These are network loss cost, switching operation loss cost, and load shedding cost, respectively. and These are the system voltage imbalance and current imbalance, respectively; c net c switch c load These represent the unit costs of network loss, switching loss, and load shedding, respectively; K l O, L, N, and n represent the branch set, sectionalizing switch set, tie switch set, distribution substation set, and load node set within the distribution substation, respectively; P ij,t Q ij,t U ij,t Let r represent the active power, reactive power, and terminal voltage of branch ij at time t; ij k is the resistance of branch ij; p and k q These are binary variables representing the open and closed states of the sectionalizing switch and the tie switch, respectively, where 0 represents the switch being open and 1 represents the switch being closed. P represents the importance level of the j-th load node within the i-th distribution area. i,j,org and P i,j,res These represent the active power of the j-th load node in the i-th distribution area before the fault and the active power after the load transfer operation, respectively. and Let be the three-phase voltage and the average three-phase voltage of the i-th distribution substation, respectively; Φ is the set of phase a, phase b, and phase c; and These are the three-phase positive and negative sequence currents of the i-th distribution substation, respectively; , , These are the currents of phase a, phase b, and phase c, respectively. for .

[0081] Step 202: Establish constraints based on the state data.

[0082] In this embodiment of the invention, the constraints include power flow constraints, distributed generation output constraints, energy storage constraints, and node voltage constraints.

[0083] The power flow constraints are as follows:

[0084]

[0085] Among them, P i,t and Q i,t Let U be the active power and reactive power of the i-th load node at time t, respectively; i,t and U j,t Let G be the voltage values ​​of the i-th and j-th load nodes at time t, respectively; ij and B ij These are the conductance and susceptance between load node i and load node j, respectively; Let be the phase difference between load node i and load node j.

[0086] The output constraints of distributed power sources are as follows:

[0087]

[0088] in, and These represent the active power output and reactive power output of the distributed power source at time t, respectively. and These are the minimum active and reactive power outputs of the distributed power source at time t, respectively. and These represent the maximum active and reactive power outputs of the distributed power source at time t, respectively.

[0089] To ensure that the state of charge of the energy storage is within the operating range and to avoid battery damage caused by overcharging and over-discharging, the following energy storage constraints are set:

[0090]

[0091] in, The state of charge (SOC) of the stored energy at time t. and A binary variable representing the operating state of energy storage. This indicates that the energy storage is in a charging state. This indicates that the energy storage is in a discharging state; and These are the charging and discharging efficiencies of energy storage, respectively. For energy storage capacity; and These are the charging power and discharging power of the energy storage, respectively. and These represent the minimum and maximum SOC of energy storage, respectively.

[0092] The node voltage constraints are as follows:

[0093]

[0094] Among them, U i,t Let be the voltage amplitude of the i-th load node at time t; and These represent the maximum and minimum values ​​of the voltage amplitude, respectively.

[0095] Step 203: Generate a state space based on the real-time power output, real-time load power demand, sectional switch status, tie switch status, node voltage, energy storage charge status, and branch power in the power distribution multi-unit area.

[0096] In this embodiment of the invention, the state space can be represented as follows:

[0097]

[0098] Wherein, the state space s at time t t Includes: real-time power output at time t within multiple distribution zones. Real-time power demand of the load at time t The segmented switch state at time t The state of the contact switch at time t Node voltage at time t State of charge of energy storage at time t Branch power at time t .

[0099] Step 204: Generate the action space based on the active power of the disconnected load and the binary variables of the opening and closing states of the sectionalizing switch and the tie switch.

[0100] In this embodiment of the invention, the intelligent agent executes actions based on its observed status of the multi-station power distribution system, used to control the opening and closing of switches in the system and, when necessary, the disconnection of partial loads. The action space can be represented as follows:

[0101]

[0102] Wherein, the action space a at time t t Includes: k p,t and k q,t These are binary variables representing the open / closed states of the segmented switch and the tie switch at time t, respectively. Let t be the active power of the load that is removed at time t.

[0103] Step 205: Generate a reward function based on the objective function and the preset penalty value.

[0104] In this embodiment of the invention, based on the selected action, the agent receives a certain reward, which guides the agent to improve its behavior in the direction of maximizing cumulative reward. The reward function, based on the objective function, is set as follows:

[0105]

[0106] Where, r t F is the reward function value at time t; t Let t be the objective function value at time t; α represents the penalty value given to the agent after violating the constraints.

[0107] Step 206: Cluster the samples in the preset experience pool and determine the target cluster based on the observed samples.

[0108] To address the shortcomings of the DDPG algorithm in the parameter update phase, this invention proposes an improved DDPG algorithm. This algorithm uses the K-medoids algorithm to cluster the data stored in the experience pool and selects samples for updating network parameters based on the similarity between the current strategy and the clustering experience. This process can be represented as follows:

[0109] 1) For m experience samples in the experience pool ,in Randomly select g sample data As the initial cluster center.

[0110] 2) Calculate the distance from the remaining samples to the initial cluster centers using Mahalanobis distance, and assign them to the nearest clusters. The expression for Mahalanobis distance is as follows:

[0111]

[0112] Among them, D hi,hj For sample h i To its cluster center sample h j Mahalanobis distance between them; Let be the covariance matrix.

[0113] 3) Calculate the sum of distances J from empirical samples to cluster centers in all clusters:

[0114]

[0115] Among them, D hi,hj is the Mahalanobis distance between the sample h i and its cluster center sample h j .

[0116] 4) Randomly select a sample h from the m - g remaining samples v to replace the initial cluster center sample h j , and repeat steps 2) and 3) to obtain a new J';

[0117] 5) Update the cluster center according to the magnitudes of J and J'. If J' < J, then the new cluster center is updated to ;

[0118] 6) Repeat steps 4) and 5) until all cluster centers no longer change, obtaining the final K clusters.

[0119] Based on the clusters divided above, calculate the Mahalanobis distance between the current policy of the agent and each cluster center, compare multiple Mahalanobis distances, and determine the cluster to which the cluster center corresponding to the smallest Mahalanobis distance belongs as the target cluster; extract samples from the target cluster to update the network parameters. Step 207: Calculate the priority for each sample in the target cluster to determine the extracted samples.

[0120] In the embodiments of the present invention, a sample extraction method that comprehensively considers the sampling frequency and TD error is used to update the parameters of the Actor network and the Critic network. Based on the target cluster, by calculating two indicators, namely the TD error and the extraction frequency of the samples in this cluster, calculate their comprehensive priority, and accordingly obtain a batch of samples with a higher extraction probability to update the parameters of the main network. Step 207 specifically includes:

[0121] Step 2071: Calculate the error value and extraction frequency priority for each sample in the target cluster.

[0122] In the embodiments of the present invention, the TD error value

[0123] of sample i is calculated according to the following formula:

[0124]

[0125] Among them, is a preset discount factor; is the output value of the target Critic network with parameter ; is the output value of the Critic network with parameter .

[0126] Sampling frequency priority based on sample sampling frequency It can be represented as:

[0127]

[0128] in, It is the sampling frequency priority of sample i; u is the penalty constant. i This represents the number of times sample i has been drawn.

[0129] Step 2072: Generate a comprehensive priority for each sample based on the error value and sampling frequency priority of each sample.

[0130] In this embodiment of the invention, the sample synthesis priority considering the TD error value and the decimation frequency priority can be expressed as:

[0131]

[0132] Where, p i The overall priority of sample i; Let be the error value of sample i; Parameters set to avoid an overall priority of 0; Parameters that control the degree of priority amplification; It is the sampling frequency priority of sample i.

[0133] Step 2073: Generate the sampling probability of each sample based on the overall priority.

[0134] In this embodiment of the invention, based on the comprehensive priority of each sample, the probability of each sample being extracted from the target cluster can be expressed as:

[0135]

[0136] in, p is the probability of sample i being selected. i is the overall priority of sample i; k is the total number of samples in the target cluster.

[0137] Step 2074: Determine the sample to be drawn according to the preset extraction conditions and the extraction probability.

[0138] In this embodiment of the invention, the extraction conditions can be set according to actual needs, and this embodiment of the invention does not limit them.

[0139] As an alternative, the extraction condition is to select the top N samples with the highest extraction probability as the extraction samples.

[0140] Specifically, the sampling probabilities of each sample in the target cluster are sorted from high to low, and the samples corresponding to the top N sampling probabilities are selected as the sampling samples.

[0141] Step 208: Using a dynamic soft update strategy, the network parameters of the agent model are iteratively updated based on the extracted samples to construct a load transfer model.

[0142] It is worth noting that traditional soft update strategies optimize the target network parameters by using fixed soft update coefficients, but fixed coefficient values ​​cannot meet the requirements of parameter update magnitude at different training stages.

[0143] In this embodiment of the invention, the update magnitude can be automatically adjusted according to the training process by dynamically adjusting the soft update coefficient. The proposed dynamic soft update can be expressed as:

[0144]

[0145] in, and These are the parameters for the target Actor network and the target Critic network, respectively. and These are the parameters for the main Actor network and the main Critic network, respectively. This is a soft update coefficient; This indicates a truncation function to restrict... The range of change; The minimum soft update coefficient; These are the initial values ​​for the soft update coefficients; This is the soft update reduction factor, used to control the extent of its reduction; This represents the number of training rounds.

[0146] In this embodiment of the invention, it can be seen from the dynamic soft update formula that in the early stages of training, due to changes in the parameter values ​​in the estimated network, the parameter values ​​in the target network change rapidly; in the later stages of training, due to... The decrease of parameters and The updates are now smoother, ensuring the stability of the target network during later training stages.

[0147] Based on the improved DDPG algorithm, the agent model is trained offline to achieve optimal parameter fitting.

[0148] Offline training phase:

[0149] Step 1: Initialize the Actor network, Critic network parameters, corresponding target network parameters, and experience replay pool.

[0150] Step 2: Initialize environment state .

[0151] Step 3: The agent bases its actions on the environment it observes. Get action and receive a reward Enter the next state .

[0152] Step 4: Sample data Store it in the experience replay pool D.

[0153] Step 5: Calculate the current sample Mahalanobis distance to all cluster centers in the experience replay pool D.

[0154] Step 6: Based on the sample extraction probability obtained from the comprehensive priority, select B sample data from the nearest cluster.

[0155] Step 7: Update network parameters based on the extracted empirical samples.

[0156] Step 8: Repeat steps 3-7 until all step lengths have been trained.

[0157] Step 9: Repeat steps 1-8 until all rounds of training are completed.

[0158] Step 209: Input the real-time status variables and random noise values ​​of the multiple distribution areas into the load transfer model to perform adaptive fault switching and generate the optimal load transfer action.

[0159] In this embodiment of the invention, when applied online, based on the trained load transfer model, the real-time status variables and random noise values ​​of multiple distribution areas are input to obtain the optimal load transfer action.

[0160] Specifically, the real-time status variables of multiple distribution areas are added with random noise values ​​and input into the Actor network of the trained load transfer model. The Actor network then outputs the optimal load transfer action.

[0161] Step 210: Execute the optimal load transfer action to complete the seamless load transfer operation.

[0162] The following specific example illustrates the execution process of the power distribution multi-area load transfer method based on improved deep reinforcement learning:

[0163] First, real-time data such as load demand, renewable energy output, and fault information are collected from each distribution area. Then, this data is used as state input to train the improved DDPG algorithm. The improved DDPG algorithm proposed in this invention innovatively introduces a dynamic soft update mechanism and a priority experience replay strategy based on cluster optimization. Specifically, the K-medoids algorithm is used to cluster the state transition samples in the experience pool, and training samples with high relevance to the current strategy are selected based on Mahalanobis distance. Simultaneously, a dynamically adjusted soft update coefficient is used to balance the parameter update requirements at different training stages. During training, the agent generates control actions through the Actor network, and the Critic network evaluates the value of these actions and stores the state transition experience in a shared experience pool. Network parameters are updated by sampling high-probability experiences from the cluster-optimized experience pool until the model converges. After sufficient training, the agent can perceive the grid operating status in real time and autonomously generate the optimal fault switching scheme, significantly improving the recovery capability and operational reliability of the distribution network under complex fault conditions while ensuring continuous power supply to important loads.

[0164] Figure 3 A comparison of convergence results between an improved DDPG algorithm, Deep Q-Network Learning (DQN), and the traditional DDPG algorithm, provided in this embodiment of the invention, is shown in the figure. Figure 3 As shown, the horizontal axis represents the number of training epochs, ranging from 0 to 2000 with intervals of 400; the vertical axis represents the average reward value, ranging from -1200 to 0 with intervals of 200. Figure 3 As shown, in terms of convergence speed, the improved DDPG algorithm of this invention quickly reaches a stable state within 400 training rounds, which is significantly faster than the traditional DDPG and DQN algorithms. This is due to the innovative dynamic soft update mechanism and the experience replay strategy of cluster optimization.

[0165] In the early stages of training, the improved DDPG algorithm of this invention rapidly increased the reward value from -800 to -200 in just 400 rounds, while the reward values ​​obtained by traditional DDPG and DQN were significantly lower than those of the algorithm proposed in this invention. This verifies the significant effect of dynamic soft update coefficient in accelerating the learning efficiency in the early stages.

[0166] In terms of reward value performance, the improved DDPG algorithm of this invention eventually converges stably to around -150, which is an improvement over the traditional DDPG and DQN. This is mainly attributed to the priority experience replay mechanism based on K-medoids clustering, which effectively improves learning efficiency by intelligently selecting high-value training samples.

[0167] Figure 4This is a bar chart comparing the economic cost and imbalance of a multi-distribution zone after optimization, using an improved DDPG algorithm provided in this embodiment of the invention, compared with DQN and the traditional DDPG algorithm. Figure 4 As shown, in terms of economic cost, the improved DDPG algorithm of this invention reduces the economic loss of distribution network fault recovery to 873 yuan, which is 24.5% and 35.4% higher than the traditional DDPG algorithm (1156 yuan) and DQN algorithm (1352 yuan), respectively. This shows that the power transfer strategy obtained by the proposed method can effectively reduce the power outage losses caused in the power transfer process.

[0168] In terms of power quality indicators, the improved DDPG algorithm of this invention also demonstrates significant advantages: the current imbalance is controlled at 1.12%, which is 64.1% and 62.3% lower than the traditional DDPG algorithm and DQN, respectively; the voltage imbalance is maintained at 0.92%, which is better than the performance of DDPG (2.86%) and DQN (2.97%). This improvement stems from the fact that the algorithm takes into account the three-phase imbalance, and achieves power quality optimization under fault conditions by intelligently adjusting the switch state and timely disconnecting the load.

[0169] Of particular note is that the improved DDPG algorithm of this invention achieves an optimal balance across three metrics: significantly reducing economic losses while ensuring good power quality.

[0170] Figure 5 This is a bar chart comparing the optimized total load power supply recovery rate of multiple distribution areas and the recovery rate of different load levels between the improved DDPG algorithm provided in this embodiment of the invention and DQN and the traditional DDPG algorithm. Figure 5 As shown, the improved DDPG algorithm of this invention exhibits significant advantages in all indicators: the total load power restoration rate reaches 98.7%, while the total load restoration rates of DDPG and DQN are 97.5% and 96.8%, respectively.

[0171] In terms of the critical first-level load recovery rate, the improved DDPG algorithm of this invention significantly outperforms DDPG (93.1%) and DQN (92.4%) with a performance of 96.8%, directly demonstrating the effectiveness of the load prioritization design. By constructing a multi-objective reward function that considers load levels, the improved DDPG algorithm of this invention automatically strengthens its ability to guarantee important loads during training.

[0172] Of particular note is that the improved DDPG algorithm of this invention not only improves the primary load recovery rate, but also achieves secondary and tertiary load recovery rates of 95.4% and 94.2% respectively, realizing coordinated optimization of loads at all levels and demonstrating significant advantages in power supply reliability, providing a better solution for fault recovery in smart distribution networks.

[0173] It is worth noting that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. The user information in the embodiments of this application was obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have been authorized and agreed upon by the client.

[0174] It is worth noting that the information collected in this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation portals are provided for users to choose to authorize or refuse.

[0175] It is worth noting that the technical solution provided in this application provides users with a corresponding operation entry point, allowing users to choose to agree to or reject the automated decision-making result; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0176] The technical solution of the load transfer method for multiple distribution substations based on improved deep reinforcement learning provided in this invention embodiment constructs a fault switching and load transfer model for multiple distribution substations under fault scenarios based on the structure of the multiple distribution substations and the state data and unit cost of the multiple distribution substations. Combining Markov decision processes, the fault switching and load transfer model for multiple distribution substations under fault scenarios is converted into an intelligent agent model. The intelligent agent model is trained based on an improved deep reinforcement learning algorithm to construct a load transfer model. Through the load transfer model, adaptive fault switching is performed on the acquired real-time state variables and random noise values ​​of the multiple distribution substations to generate the optimal load transfer action. This enables rapid adaptive decision-making and precise control of the multiple distribution substations under fault scenarios, effectively improving adaptability to high-dimensional dynamic operating environments. By intelligently generating the optimal load transfer strategy, priority can be given to ensuring power supply to important loads under source-load power imbalance, and the three-phase balance state of the system can be optimized simultaneously, thereby significantly enhancing the power supply reliability and system safety stability during islanded operation.

[0177] Figure 6 This is a schematic diagram of a multi-distribution area load transfer device based on improved deep reinforcement learning, provided in an embodiment of the present invention. This device is used to execute the aforementioned multi-distribution area load transfer method based on improved deep reinforcement learning, such as... Figure 6 As shown, the device includes: a basic model building unit 11, an agent model conversion unit 12, a load transfer model training unit 13, and an optimal load transfer action generation unit 14.

[0178] The basic model building unit 11 is used to build a fault switching and load transfer model for multiple distribution substations under fault scenarios, based on the structure of multiple distribution substations and the status data and unit cost of the multiple distribution substations.

[0179] The intelligent agent model conversion unit 12 is used to combine the Markov decision process to convert the power distribution multi-area fault switching and load transfer model under the fault scenario into an intelligent agent model.

[0180] The load transfer model training unit 13 is used to train the agent model based on the improved deep reinforcement learning algorithm to construct the load transfer model.

[0181] The optimal load transfer action generation unit 14 is used to adaptively switch faults based on the real-time status variables and random noise values ​​of the acquired power distribution multi-station area through the load transfer model, and generate the optimal load transfer action.

[0182] In this embodiment of the invention, the fault switching and load transfer model for multiple distribution substations under fault scenarios includes an objective function and constraints. The basic model construction unit 11 is specifically used to establish an objective function based on the structure of multiple distribution substations, according to the state data and unit cost of the multiple distribution substations, and to establish constraints based on the state data. The constraints include power flow constraints, distributed power output constraints, energy storage constraints, and node voltage constraints.

[0183] In this embodiment of the invention, the intelligent agent model includes a state space, an action space, and a reward function. The intelligent agent model conversion unit 12 is specifically used to generate the state space based on the real-time power output, real-time load power demand, sectionalizing switch status, tie switch status, node voltage, energy storage charge status, and branch power in the power distribution multi-station area; generate the action space based on the active power of the disconnected load and the binary variables of the opening and closing status of the sectionalizing switch and tie switch; and generate the reward function based on the objective function and a preset penalty value.

[0184] In this embodiment of the invention, the load transfer model training unit 13 is specifically used to cluster samples in a preset experience pool and determine the target cluster based on the observed samples; perform priority calculation on each sample in the target cluster to determine the sample to be extracted; and iteratively update the network parameters of the agent model according to the extracted samples through a dynamic soft update strategy to construct the load transfer model.

[0185] In this embodiment of the invention, the load transfer model training unit 13 is specifically used to calculate the error value and sampling frequency priority of each sample in the target cluster; generate a comprehensive priority for each sample based on the error value and sampling frequency priority of each sample; generate the sampling probability of each sample based on the comprehensive priority; and determine the sample to be extracted according to the sampling probability based on the preset sampling conditions.

[0186] In this embodiment of the invention, the optimal load transfer action generation unit 14 is specifically used to input the real-time status variables and random noise values ​​of multiple distribution areas into the load transfer model for adaptive fault switching, generate the optimal load transfer action, execute the optimal load transfer action, and complete the seamless load transfer operation.

[0187] In this embodiment of the invention, based on the multi-distribution area structure, a fault switching and load transfer model for the multi-distribution area under fault scenarios is constructed according to the status data and unit cost of the multi-distribution area. Combining a Markov decision process, the fault switching and load transfer model is converted into an intelligent agent model. An improved deep reinforcement learning algorithm is used to train the intelligent agent model, constructing a load transfer model. Through the load transfer model, adaptive fault switching is performed on the acquired real-time status variables and random noise values ​​of the multi-distribution area, generating the optimal load transfer action. This enables rapid adaptive decision-making and precise control of the multi-distribution area under fault scenarios, effectively improving its adaptability to high-dimensional dynamic operating environments. By intelligently generating the optimal load transfer strategy, priority can be given to ensuring power supply to important loads under source-load power imbalance, and the three-phase balance state of the system can be optimized simultaneously, thereby significantly enhancing the power supply reliability and system safety stability during islanded operation.

[0188] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer device, specifically, a computer device can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0189] This invention provides a computer device including a memory and a processor. The memory stores information including program instructions, and the processor controls the execution of the program instructions. When the program instructions are loaded and executed by the processor, they implement the steps of the above-described embodiment of the power distribution multi-area load transfer method based on improved deep reinforcement learning. For a detailed description, please refer to the above-described embodiment of the power distribution multi-area load transfer method based on improved deep reinforcement learning.

[0190] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer device 600 suitable for implementing the embodiments of this application.

[0191] like Figure 7As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate tasks and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer device 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0192] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal feedback (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed in storage section 608 as needed.

[0193] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611.

[0194] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0195] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0196] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0197] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0198] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0199] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0200] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0201] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0202] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0203] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0204] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0205] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for load transfer across multiple distribution areas based on improved deep reinforcement learning, characterized in that, The method includes: Based on the multi-distribution area structure, and according to the status data and unit cost of the multi-distribution area, a fault switching and load transfer model for the multi-distribution area under fault scenarios is constructed. By incorporating Markov decision processes, the fault switching and load transfer model for multiple distribution areas under the aforementioned fault scenario is transformed into an intelligent agent model. The agent model is trained based on an improved deep reinforcement learning algorithm to construct a load transfer model. The load transfer model is used to adaptively switch faults based on the real-time status variables and random noise values ​​of the acquired power distribution zones, thereby generating the optimal load transfer action.

2. The method for load transfer between multiple distribution areas based on improved deep reinforcement learning according to claim 1, characterized in that, The fault switching and load transfer model for multiple distribution areas under the fault scenario includes the objective function and constraints. Based on the multi-distribution area structure, and according to the status data and unit cost of the multi-distribution area, a fault switching and load transfer model for the multi-distribution area under fault scenarios is constructed, including: Based on the multi-distribution area structure, an objective function is established according to the status data and unit cost of the multi-distribution area. Based on the state data, constraints are established, including power flow constraints, distributed generation output constraints, energy storage constraints, and node voltage constraints.

3. The method for load transfer between multiple distribution areas based on improved deep reinforcement learning according to claim 2, characterized in that, The agent model includes a state space, an action space, and a reward function; The process of combining Markov decision processes to transform the multi-distribution area fault switching and load transfer model under the fault scenario into an intelligent agent model includes: A state space is generated based on the real-time power output, real-time load power demand, sectional switch status, tie switch status, node voltage, energy storage charge status, and branch power within the multiple distribution areas. The action space is generated based on the active power of the disconnected load and the binary variables of the opening and closing states of the sectionalizing switch and the tie switch. A reward function is generated based on the objective function and the preset penalty value.

4. The method for load transfer between multiple distribution areas based on improved deep reinforcement learning according to claim 1, characterized in that, The process of training the agent model using an improved deep reinforcement learning algorithm to construct a load shifting model includes: Cluster the samples in the pre-defined experience pool and determine the target cluster based on the observed samples; Priority calculation is performed on each sample in the target cluster to determine the samples to be extracted; By employing a dynamic soft update strategy, the network parameters of the agent model are iteratively updated based on the extracted samples to construct the load transfer model.

5. The method for load transfer between multiple distribution areas based on improved deep reinforcement learning according to claim 4, characterized in that, The step of prioritizing each sample in the target cluster to determine the samples to be extracted includes: Calculate the error value and sampling frequency priority for each sample in the target cluster; A comprehensive priority for each sample is generated based on the error value and sampling frequency priority of each sample. Based on the comprehensive priority, the probability of sampling for each sample is generated; According to the preset extraction conditions and the extraction probability, the extracted samples are determined.

6. The method for load transfer between multiple distribution areas based on improved deep reinforcement learning according to claim 1, characterized in that, The process of adaptively switching faults based on the real-time status variables and random noise values ​​of multiple distribution areas obtained through the load transfer model, and generating the optimal load transfer action, includes: The real-time status values ​​and random noise values ​​of the multiple distribution zones are input into the load transfer model to perform adaptive fault switching and generate the optimal load transfer action. Execute the optimal load transfer action to complete the seamless load transfer operation.

7. A power distribution multi-area load transfer device based on improved deep reinforcement learning, characterized in that, The device includes: The basic model building unit is used to build a fault switching and load transfer model for multiple distribution substations under fault scenarios based on the structure of multiple distribution substations and the status data and unit cost of the multiple distribution substations. The agent model conversion unit is used to convert the power distribution multi-area fault switching and load transfer model under the fault scenario into an agent model by combining the Markov decision process. The load transfer model training unit is used to train the agent model based on an improved deep reinforcement learning algorithm to construct a load transfer model. The optimal load transfer action generation unit is used to adaptively switch faults based on the real-time status variables and random noise values ​​of the acquired power distribution multi-station areas through the load transfer model, and generate the optimal load transfer action.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the power distribution multi-area load transfer method based on improved deep reinforcement learning as described in any one of claims 1 to 6.

9. A computer device comprising a memory and a processor, the memory for storing information including program instructions, and the processor for controlling the execution of the program instructions, characterized in that, When the program instructions are loaded and executed by the processor, they implement the power distribution multi-area load transfer method based on improved deep reinforcement learning as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the power distribution multi-area load transfer method based on improved deep reinforcement learning as described in any one of claims 1 to 6.

Images