A distribution network action plan auxiliary decision-making method, system, device and storage medium

Abstract

The present invention discloses a distribution network action plan auxiliary decision method, system, device and storage medium, the method includes the steps of: obtaining online operation data of the distribution network; inputting the online operation data of the distribution network into a preset distribution network action plan auxiliary decision model; the distribution network action plan auxiliary decision model outputs a network structure that enables the distribution network to operate safely and uncertainly; the distribution network action plan auxiliary decision model is formed by training based on a DQN reinforcement learning algorithm. For an online distribution network, only the corresponding state needs to be input to reconstruct the topological structure of the distribution network. Because the judgment process does not require the calculation of all reconstruction schemes of the entire distribution network, the amount of calculation is small and the time consumed is short. Therefore, the distribution network action plan auxiliary decision method based on the reinforcement learning method has the advantages of fast calculation speed and high efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of power grid operation safety, and specifically relates to a distribution network action plan auxiliary decision-making method, system, device and storage medium. Background Art

[0002] With the rapid development of urban distribution network technology, distribution network construction has gradually entered a high reliability stage. According to statistics, more than three quarters of power outages are caused by failures in the distribution network. At the same time, with the popularization of distributed renewable energy (DRG), the distribution network is facing changes in supply and demand. How to ensure the safe and economical operation of the distribution network is becoming increasingly important. Distribution network reconfiguration (DNR) adjusts the topology of the distribution network by controlling the on / off state of the connecting switches, which can reduce network losses, improve the voltage quality of the distribution network, and ensure the safe and stable operation of the power grid.

[0003] Traditional methods for optimizing the dispatching of distribution networks using network reconstruction mainly include heuristic algorithms such as brute force search algorithms and genetic algorithms. However, the network topology of the distribution network is complex, the distribution is relatively dispersed, the equipment types are numerous, and the equipment operation status is easily affected by external factors. Therefore, the traditional method of network reconstruction requires a large amount of calculation, which results in a long time consumption and low final reconstruction efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide a distribution network action plan auxiliary decision-making method, system, device and storage medium to solve the problems in the prior art that traditional heuristic methods such as genetic algorithms have low decision efficiency and untimely decision-making due to long calculation time.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] In a first aspect of the present invention, a method for assisting decision making of a power distribution network action plan based on a deep neural network is provided, comprising the following steps:

[0007] Obtain online operation data of distribution network;

[0008] Inputting the online operation data of the distribution network into a preset distribution network action plan auxiliary decision model; the distribution network action plan auxiliary decision model outputs a network structure that enables the distribution network to operate safely and uncertainly;

[0009] Among them, the distribution network action plan auxiliary decision-making model is formed based on DQN reinforcement learning algorithm training.

[0010] Optionally, the online operation data of the distribution network includes real-time or predicted switch status, wind turbine power generation, photovoltaic power generation and load size.

[0011] Optionally, the distribution network action plan auxiliary decision model is obtained in the following manner:

[0012] Obtain historical operation data of distribution network;

[0013] Set the ε-greedy strategy to select actions;

[0014] Establishing a distribution network system environment model based on the historical operation data of the distribution network, and constructing a deep reinforcement learning model of the intelligent agent;

[0015] Based on the preset ε-greedy strategy, the distribution network system environment model and deep reinforcement learning model are used for offline training and learning to obtain a distribution network action plan auxiliary decision-making model that meets the error requirements.

[0016] Optionally, after acquiring the historical operation data of the distribution network, the historical operation data of the distribution network is preprocessed to convert the historical operation data of the distribution network into an original sample set suitable for the reinforcement learning algorithm.

[0017] Optionally, offline training and learning are performed using the distribution network system environment model and deep reinforcement learning model, including:

[0018] Each time the distribution network system environment model executes an action given by the deep reinforcement learning model, it returns a new system state and calculates the corresponding reward value; and the deep reinforcement learning model, based on the current state, aims to output a control action that can maximize the expected value of the reward, and continuously learns and improves the action strategy during the interaction with the distribution network system environment model.

[0019] Optionally, establishing the distribution network system environment model includes: setting the agent state space, action space and agent reward / penalty mechanism.

[0020] Optionally, when building a deep reinforcement learning model for an agent, use two neural networks, one real network that generates the current Q value and the other a target network, Q target. The two neural networks have the same initial weights and parameters but different parameter update speeds.

[0021] A second aspect of the present invention provides a system for the distribution network action plan auxiliary decision-making method based on a deep neural network, comprising:

[0022] Data acquisition module, used to obtain online operation data of distribution network;

[0023] The prediction module is used to input the online operation data of the distribution network into a preset distribution network action plan auxiliary decision model; the distribution network action plan auxiliary decision model outputs a network structure that enables the distribution network to operate safely and uncertainly.

[0024] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for assisting decision-making of distribution network action plans based on a deep neural network is implemented.

[0025] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the distribution network action plan auxiliary decision-making method based on a deep neural network is implemented.

[0026] The beneficial effects of the present invention are as follows:

[0027] (A) The neural network-based distribution network action plan auxiliary decision-making method provided by the present invention uses an artificial intelligence algorithm to solve the idea of ​​​​distribution network action plan auxiliary decision-making: first, the distribution network risk warning problem is converted into a Markov decision process, the network structure, power generation and load of the distribution network are selected as states, the opening and closing of all available line switches are taken as actions, and ensuring the radiation and safe and stable operation of the power grid is taken as a reward, and then the distribution network auxiliary decision-making model is trained using the DON reinforcement learning algorithm. In this way, for the online distribution network, only the corresponding state needs to be input to reconstruct the topological structure of the distribution network. Because the judgment process does not need to calculate all the reconstruction plans of the entire distribution network, the calculation amount is small and the time consumed is short. Therefore, the distribution network action plan auxiliary decision-making method based on the reinforcement learning method has the advantages of fast calculation speed and high efficiency.

[0028] (B) The DQN algorithm used in the present invention is an algorithm that is widely used in the field of reinforcement learning and has excellent performance. It combines deep neural networks and Q-learning reinforcement learning algorithms. Compared with traditional Q-learning algorithms, it has a deep neural network and stores more actions. It can solve the problem of "dimensionality curse" and is suitable for making decisions on various discrete actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0030] Figure 1 This is a flow chart of a method for assisting decision-making in power distribution network operation according to an embodiment of the present invention;

[0031] Figure 2 A structural diagram of the process of interaction between a reinforcement learning agent and an environment in an embodiment of the present invention;

[0032] Figure 3This is a structural diagram of the DQN reinforcement learning algorithm training process in an embodiment of the present invention;

[0033] Figure 4 A flowchart of a Markov decision process (MDP) in an embodiment of the present invention;

[0034] Figure 5 Schematic diagram of the neural network structure of the DQN reinforcement learning algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0036] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention.

[0037] The embodiments of the present invention provide a distribution network action plan auxiliary decision-making method, system, device and storage medium to solve the problems of low decision-making efficiency and untimely decision-making caused by long calculation time of current traditional heuristic methods such as genetic algorithms.

[0038] like Figure 1 As shown, in the first aspect of the present invention, a distribution network action plan auxiliary decision method based on a deep neural network is provided, which uses a reinforcement learning algorithm model to autonomously learn the distribution network auxiliary decision process. The trained model can make decisions immediately according to the current distribution network state to maintain the stable operation of the distribution network, including the following steps:

[0039] Step 1: Data acquisition and preprocessing. The historical operation data of the distribution network in a certain area is selected as the data source, and the historical operation data of the distribution network involved in the distribution network auxiliary decision-making is converted into an original sample set suitable for the reinforcement learning algorithm.

[0040] Specifically, the historical operation data of the distribution network includes: data that can simulate the distribution system (corresponding to the "environment" in RL) and various input data (corresponding to the "state" in RL, such as the number of wind turbines, the number of photovoltaic power generation, the number of loads and the frequency of switch operation, etc.).

[0041] The data preprocessing process in the present invention mainly consists of the following steps:

[0042] Step 11: The number of WT, PV and load in the distribution system is denoted as N WT ,N PV and ND , the required data includes the distributed renewable energy equipment power generation data set:

[0043]

[0044]

[0045] Load dataset:

[0046]

[0047] Step 12: Simultaneously record the time interval between the last operation of each switch and the current moment:

[0048]

[0049] When the time interval is less than the preset value, the switch will not operate within the preset value range. SW After that, the value does not increase further, which is enough to indicate that the switch is not overused. tr It is not prepared in advance, but rather extracted from the agent's actions (after assigning it an initial value).

[0050] Step 13: D t The data are obtained from various online or offline sources, and the data are preprocessed, including data cleaning and normalization.

[0051] Specifically, data cleaning refers to eliminating problem data such as missing data and erroneous measurement (zero or negative value) data, and selecting valid data with the closest linear connection to perform.

[0052] Specifically, data normalization is to convert the original data into D t Normalized to X WT ,X PV ,X D , remove all negative data, and divide all data by their maximum value. Before entering DQN, SW tr Normalized to X tr , divide each element by T SW .

[0053] Step 14: The normalized data is divided into training and testing sets.

[0054] Specifically, in ordinary reinforcement learning, the state is updated through the interaction between the agent and the environment, and does not depend on special external data. In this embodiment, the state depends on the power distribution system and distributed renewable energy system power generation corresponding to the environment, and the load data will change according to external conditions such as weather. If data separation is not performed, DQN may be overused in the training data set, which may cause configuration failure. Therefore, a test set is specified in this embodiment to verify whether DQN learning is performed correctly under relatively harsh conditions.

[0055] Step 2: Set the ε-greedy strategy to select actions;

[0056] Specifically, in reinforcement learning, the action selection of the agent is called a strategy. The purpose of reinforcement learning is to determine the best strategy to obtain the highest reward. Before training, DQN randomly initializes the weights and bias values, so it is not initially certain that an action with the highest reward will be output. Therefore, the agent takes random actions to "explore" which one can obtain a high reward, rather than relying on the output of DQN from the beginning. Therefore, this embodiment uses the intuitive and simple ε-greedy as an exploration strategy.

[0057] The agent selects the action with the largest Q value in the corresponding state with probability ε, and randomly selects action a with probability 1-ε t , to ensure that it can explore as much as possible in the action space, the ε-greedy formula is as follows:

[0058]

[0059] In the ε-greedy strategy, the larger the ε value, the faster the convergence, but also the easier it is to fall into the local optimum. Therefore, in this embodiment, in the early stage of exploration, the agent still lacks effective information and explores in the action space with a greater probability. With continuous learning, its accumulated experience and knowledge tend to be more accurate, and ε is gradually reduced.

[0060] Step 3: Use the preprocessed data to establish a distribution network system environment model and build a deep reinforcement learning model of the intelligent agent. Set the corresponding state space, action space and reward (penalty) function, use the DQN reinforcement learning model and simulation environment for offline training and learning, and obtain a distribution network action plan auxiliary decision model that meets the error requirements.

[0061] Figure 2 It is a reinforcement learning process for the interaction between the agent and the system environment. Each time the system environment executes an action given by the agent, it returns a new system state and calculates the corresponding reward value; and the agent, based on the current state, aims to output a control action that can maximize the expected value of the reward, and continuously learns and improves its action strategy in the process of interacting with the actual environment.

[0062] As an example applied to the present invention, a deep Q network (DQN) fits the action-value function through a neural network, learns the Q values ​​of all state-action pairs in the process of interaction between a limited number of states and the environment, and thus learns the optimal strategy.

[0063] Figure 3 For the DQN algorithm training process, step 3 specifically includes the following steps:

[0064] Step 31: Construct a distribution network system environment model;

[0065] The distribution network system environment model is the power system environment that interacts with the intelligent agent. That is, for each action given by the reinforcement learning agent, it can be calculated whether there is a risk under this strategy, and the reward function can be fed back to update the strategy, and it is continuously iterated until the optimal strategy is learned.

[0066] In this example, the environment is formalized as a Markov decision process (MDP). MDP can be defined as a tuple (, A, P, R, γ), which represents the state space, action space, state transition probability, reward function and discount factor respectively. The agent observes the state s from the environment. t ∈S, and takes action a at time step t t ∈A, the agent with probability P(s t+1 |s t ,a t ) arrives at a new state s t+1 , and receive reward r(s t ,a t ,s t+1 ). Its state transformation process is as follows Figure 4 shown.

[0067] Step 32: Agent Action Space

[0068] The random action consists of an array containing 0 (turn on the switch) and 1 (turn off the switch). When the action is extracted from the DQN, the output layer is arranged in an array form, and then the switches that are desired to be turned off are replaced with 1, and the remaining elements are replaced with 0. The determined action is then input to the test system to turn each switch on or off, and the SW is updated by comparing the switch state with the state of the previous action stored. tr .

[0069] Step 33: Agent Reward (Punishment) Mechanism

[0070] The agent maximizes its cumulative reward through continuous learning over a time period, and the main purpose of the agent in this application is to maintain the radiality of the network. In addition, the reconfigured network satisfies the general power network constraints;

[0071] As an example of the present invention, for a given amount of power generation and load, the line flow or bus voltage is kept within a certain range. The reward (penalty) mechanism of the DQN model is:

[0072] Given a state s and an action a, the total reward rt in each time step can be expressed as follows:

[0073]

[0074]

[0075]

[0076]

[0077] Where L = the number of lines in the network; I i = the current carrying capacity of the ith line (percentage); α = line overload penalty weight; B = total number of buses in the network; V j = per unit value of voltage of the jth line; β = penalty weight of bus voltage; γ = penalty weight of excessive switch usage.

[0078] If the reconfigured network is radial, then each time step gives the agent a specific reward: r init ; Otherwise, it will receive a negative reward (i.e., punishment), and fail r fail , and the simulation episode will terminate immediately.

[0079] If the reconfigured network is radioactive, when the network violates the line capacity or bus voltage constraints, the agent will respond to p according to the degree of violation and the weight factor corresponding to each violation. line and p bus To punish.

[0080] p line The calculation method is to assume that the capacity percentage of the distribution network line is 1 and multiply the weight coefficient α by each illegal line i. Similarly, p bus The calculation assumes that the stable bus voltage is in the range of 0.95–1.05 and multiplies the weight coefficient β by each violating bus j. In addition, the penalty period p sw This prevents the frequent operation of the segment switch. To identify how much time has passed since the switch was last actuated, tr,k <T SW ,psw Multiply sw by the weight factor γ tr,k and T SW In some other embodiments, anyone using the model can adjust the penalty weight factor according to the applied distribution network environment.

[0081] Step 34: DQN algorithm process;

[0082] In this embodiment, DQN is used to transform the Q-table update into a function fitting problem. A function is fitted to replace the Q-table to generate Q values, so that similar states get similar output actions. Deep neural networks have a good effect on extracting complex features, so DeepLearning is combined with Reinforcement Learning to generate the DQN algorithm. The DQN reinforcement learning algorithm neural network is composed of Figure 5 shown.

[0083] Another innovation of the DQN algorithm is to solve the correlation and non-static distribution problems through the experience replay method. t ,a t ,r t ,s t+1 ) is stored in a replay memory D of size N. Then samples are randomly drawn to form a small batch of a certain size and parameter learning is performed. The size of the experience pool is limited due to memory limitations and the need to use the latest data for training. Using this method, the number of interactions with the environment can be reduced, data efficiency can be improved, the deviation caused by the correlation between training samples can be eliminated, and generalization performance can be improved; two neural networks are used, one real network generates the current Q value, and the other target network Qtarget. The initial weights and parameters of the two networks are the same, but the parameter update speed is different, which is used to reduce the correlation of data.

[0084] The specific algorithm flow is as follows:

[0085] (1) Initialize the playback memory unit D, which can accommodate N data items;

[0086] (2) Initialize the real Q network and randomly generate weights ω;

[0087] (3) Generate the target Q network with the same structure and parameters, with weight ω′=ω;

[0088] (4) Loop through, each episode = 1, 2, ..., M (where M is the total number of days);

[0089] (5) Initialize the first state after preprocessing

[0090] (6) Loop through each step of each event step = 1, 2, ..., T; (divide a day into 96 points at 15-minute intervals, T = 96)

[0091] (7) Generate action a using ∈-greedy strategy t : Choose a random action a with probability ∈ t ; If the low-probability event does not occur, use the greedy strategy to select the action with the largest current value function;

[0092]

[0093] (8) Execute switching actions in the distribution network simulation environment;

[0094] (9) If the reconfigured network is radioactive;

[0095] (10) Solve the power flow and save the current carrying capacity of all lines All bus voltages And update the record, and calculate the reward r according to the first formula in step 33 t ;

[0096] (11) Otherwise r t =r fail ;

[0097] (12) Receive reward r t and the new state s t+1 ;

[0098] (13) The conversion result (s t ,a t ,r t ,s t+1 ) is stored in D;

[0099] (14) Uniformly randomly sample a transformation sample data (s) from D j ,a j ,r j ,s j+1 );

[0100] (15) Determine whether it is the terminal state of an event. If it is the terminal state, the reward is r j ; Otherwise, the TD return is calculated using the TD target network parameter ω: j +γmax a′ Q(s′,a′,ω′);

[0101] (16) Execute the gradient descent algorithm to update the network parameters;

[0102]

[0103] (17) Update the network parameters of the action value function approximation θ = θ + Δθ;

[0104] (18) Update the target Q network every C steps;

[0105] (19) End the cycle of each event;

[0106] (20) End the loop between events.

[0107] After the training and learning of the above steps, a distribution network action plan auxiliary decision-making model based on the reinforcement learning algorithm is formed, thereby realizing accurate regulation of the distribution network structure.

[0108] Step 4: Directly use the online operation data of the distribution network and use the distribution network action plan auxiliary decision model to make auxiliary decisions online for the distribution network. Specifically, it includes the following steps:

[0109] Step 41: inputting the online operation data of the distribution network as a state into the distribution network action plan auxiliary decision model;

[0110] As an example of the present invention, the online operation data of the distribution network are real-time or predicted switch status, wind turbine power generation, photovoltaic power generation and load size.

[0111] Step 42: The distribution network action plan auxiliary decision model directly outputs a network structure that enables the distribution network to operate safely and stably.

[0112] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0113] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0116] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. A distribution network action plan auxiliary decision-making method based on deep neural network, characterized in that: The steps include: Obtain online operation data of distribution network; Inputting the online operation data of the distribution network into a preset distribution network action plan auxiliary decision model; the distribution network action plan auxiliary decision model outputs a network structure that enables the distribution network to operate safely and uncertainly; The distribution network action plan auxiliary decision model is formed based on the DQN reinforcement learning algorithm training, and specifically includes: Step 1: Select the historical operation data of the distribution network as the data source, and convert the historical operation data of the distribution network involved in the auxiliary decision-making of the distribution network into an original sample set suitable for the reinforcement learning algorithm; the historical operation data of the distribution network includes: data that can simulate the distribution system and various input data; the various input data include the number of wind turbines, the number of photovoltaic power generation, the number of loads and the frequency of switch operation; at the same time, record the time interval between the last operation of each switch and the current moment: When the time interval is less than the preset value, the switch will not operate within the preset value range; when the lead time reaches T SW After that, the value does not increase further, which is enough to indicate that the switch is not overused; therefore, SW tr is not prepared in advance, but extracted from the agent's actions; SW tr Indicates the switching record of the switch; represents the time interval of the mth switch; N sw Indicates the number of switches; Step 2: Set the ε-greedy strategy to select actions; Step 3DQN algorithm training process, including: Step 31: Construct a distribution network system environment model; Step 32: Construct the agent action space; Step 33: Agent reward mechanism; The agent maximizes its cumulative reward through continuous learning over a time period. The agent's goal is to maintain the network's radiance; for a given amount of power generation and load, keep the line flow or bus voltage within a certain range; The reward mechanism of the DQN model is: Given a state s and an action a, the total reward r in each time step is t (s,a) is expressed as follows: Where L is the number of lines in the network; I i is the current carrying capacity of the ith line; α is the line overload penalty weight; B is the total number of buses in the network; V j is the voltage per unit value of the jth line; β is the penalty weight of the bus voltage; γ is the penalty weight of excessive switch use; p line,i represents the current carrying capacity penalty value of the i-th line; p bus,j represents the voltage penalty value of the jth bus; p sw,k represents the overuse penalty value of the kth switch; sw tr,k represents the time interval of the kth switch; If the reconfigured network is radial, then at each time step the agent is given a reward r init ; Otherwise, give the agent a negative reward r fail , and the simulation episode will terminate immediately; If the reconfigured network is radioactive, when the network violates the line capacity or bus voltage constraints, the agent will respond to p according to the degree of violation and the weight factor corresponding to each violation. line and p bus to impose punishment; Step 34: DQN algorithm process; (1) Initialize the playback memory unit D, which can accommodate N data items; (2) Initialize the real Q network and randomly generate weights ω; (3) Generate the target Q network with the same structure and parameters, with weight ω′=ω; (4) Loop through, each event episode = 1, 2, ..., M; where M is the total number of days; episode represents an event; (5) Initialize the first state after preprocessing x1 represents the complete data set in state S1; represents the power generation data set of the wind turbine at the first time step; represents the photovoltaic power generation data set at the first time step; D 1 represents the load demand dataset in the distribution network at the first time step; represents the set of time intervals that have passed since the last operation of each switch in the first time step; (6) Loop through each step of each event step = 1, 2, ..., T; divide a day into 96 points at intervals of 15 minutes, T = 96; (7) Generate action a using ∈-greedy strategy t : Choose a random action a with probability ∈ t ; If the low-probability event does not occur, use the greedy strategy to select the action with the largest current value function; Among them, a represents the action taken by the agent at a certain moment; ω represents the weight parameter in the deep neural network; s t represents the state of the distribution network at time t; (8) Execute switching actions in the distribution network simulation environment; (9) If the reconfigured network is radioactive; (10) Solve the power flow and save the current carrying capacity of all lines All bus voltages And update the record, and calculate the reward r according to the first formula in step 33 t (s,a); (11) Otherwise r t (s,a)=r fail ; (12) Receive reward r t (s,a) and the new state s t+1 ;s t+1 Indicates the state of the distribution network at time t+1; (13) The conversion result (s t ,a t ,r t (s,a),s t+1 ) is stored in D; D represents the playback memory unit; (14) Uniformly randomly sample a transformation sample data (s) from D j ,a j ,r j ,s j+1 );s j represents the state of the distribution network at a certain time step j; a j Indicates that in state s j The action taken by the agent; r j Indicates that in state s j Take action a j The reward obtained by the agent; s j+1 Indicates that in state s j Take action a j The new state to which the post-environment shifts; (15) Determine whether it is the terminal state of an event. If it is the terminal state, the reward is r j ; Otherwise, the TD return is calculated using the TD target network parameter ω: j +γmax a′ Q(s′,a′,ω′); TD represents time difference; s′, a′, ω′ represent the next state, the next optimal action, and the weight of the target Q network respectively; (16) Execute the gradient descent algorithm to update the network parameters; (17) Update the network parameters of the action value function approximation θ = θ + Δθ; θ represents the parameters of the deep neural network; r represents the reward obtained by the agent from the environment after performing the action; s represents the state of the environment in which the agent is located at the current moment; a represents the action selected by the agent in the current state; ω represents the weight parameter of the Q network; (18) Update the target Q network every C steps; (19) End the cycle of each event; (20) End the cycle between events; After training and learning, a distribution network action plan auxiliary decision-making model based on reinforcement learning algorithm is formed.

2. A system for the distribution network action plan auxiliary decision-making method based on deep neural network as claimed in claim 1, characterized in that: include: Data acquisition module, used to obtain online operation data of distribution network; A prediction module, used for inputting the online operation data of the distribution network into a preset distribution network action plan auxiliary decision model; The distribution network action plan auxiliary decision model outputs a network structure that enables the distribution network to operate safely and uncertainly.

3. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the distribution network action plan auxiliary decision-making method based on deep neural network as described in claim 1 is implemented.

4. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the distribution network action plan auxiliary decision method based on deep neural network as claimed in claim 1 is implemented.

Images