Active power distribution network multi-time scale rolling optimization method driven by multi-agent graph reinforcement learning

Abstract

The invention provides an active power distribution network multi-time scale rolling optimization method driven by multi-agent graph reinforcement learning. The method comprises the following steps: dividing slow time scale equipment and fast time scale equipment; the slow time scale is optimized and modeled into a multi-agent Markov game, and a global state space and a combined action space corresponding to switch actions, reactive compensation equipment gears and voltage regulation equipment gears are defined; switching action candidate vectors are generated based on a basic loop of the power distribution network, and action space dimensions are compressed through a scheme of excluding conflicting actions of the same common branch; fitting an action value function of each agent by adopting a graph neural network, generating a dimensionality-reduced discrete action strategy according to the global state space, and carrying out collaborative optimization through a distributed decision and a global reward mechanism; and on the basis of the slow time scale optimization result, the fast time scale optimization problem is converted into a mixed integer second-order cone programming model, and an instruction is solved in a rolling manner.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of active power distribution network operation optimization, and particularly relates to a multi-agent graph reinforcement learning driven active power distribution network multi-time scale rolling optimization method. BACKGROUND

[0002] With the continuous improvement of the grid penetration rate of large-scale distributed power, the traditional power distribution network is experiencing intelligent transformation to an active power distribution network. It is worth noting that the large-scale access of distributed power will lead to significant bidirectional power flow characteristics of the power distribution network operating conditions, which may cause prominent problems such as reverse power back feeding and bus voltage out-of-limit, and thus pose a serious challenge to system network loss optimization and voltage stability. If a rough distributed power generation reduction strategy is adopted, not only will it cause marginal diminishing returns of renewable energy utilization efficiency, but also will have a negative inhibitory effect on the strategic goal of low-carbon energy transformation. Existing researches generally focus on single-dimensional solutions such as network reconfiguration, reactive power compensation and active power regulation, which can achieve the technical targets of network loss suppression and voltage regulation in specific scenarios. However, in the high penetration rate of renewable energy scenarios, a single optimization method may have technical bottlenecks and be difficult to fully meet the system multi-objective optimization requirements. Due to the significant difference in dynamic response speed of different types of devices, in the construction of the active power distribution network topology-reactive power-active power collaborative optimization and regulation system, a multi-time scale hierarchical decision mechanism needs to be established to match the difference in device response characteristics.

[0003] Existing researches mainly adopt mathematical optimization algorithms or heuristic algorithms to solve the slow-time scale optimization problem. However, the traditional methods have significant limitations: firstly, mixed integer programming models face computational efficiency bottlenecks when dealing with large-scale discrete variables, while heuristic algorithms have the risk of premature convergence; secondly, the above methods rely on accurate power distribution network parameters for model construction, making it difficult to adapt to the parameter uncertainty of complex power distribution networks. As a model-free optimization method under the deep reinforcement learning framework, the deep Q-network algorithm exhibits unique optimization potential in the discrete decision space through the interactive learning mechanism between the agent and the environment. Existing researches have verified the effectiveness of the deep Q-network algorithm in the discrete reactive power compensation device gear adjustment and the reconfiguration switch action decision, but its technical limitations are also significant: 1) the overestimation problem caused by the main network assuming the functions of action selection and Q-value evaluation has not been solved; 2) the fully connected network has the defect of insufficient topological perception ability when processing power distribution network graph structure data. In view of the inherent defects of the deep Q-network, the double deep Q-network selects the optimal action through the main network and calculates the Q-value corresponding to the action using the target network, thereby suppressing the overestimation problem of the Q-value. Recent researches have proven the optimization performance advantage of the double deep Q-network in network reconfiguration and discrete reactive power compensation device gear adjustment. At the same time, the graph reinforcement learning algorithm uses a graph neural network to fit the action value function, which exhibits significant advantages in processing graph structure data, and recent researches have also verified its effectiveness in network reconfiguration and reactive power compensation tasks. It is worth noting that although existing researches use the double deep Q-network algorithm and the graph reinforcement learning algorithm to achieve breakthroughs in alleviating the overestimation bias and enhancing the topological perception ability, respectively, they have not realized the collaborative decision of the reconfiguration switch and the discrete reactive power device. Therefore, an efficient and accurate method is needed to solve the multi-time scale topological-reactive-active collaborative optimization problem of the active power distribution network under high proportion of renewable energy access. SUMMARY

[0004] In view of the defects and deficiencies of the prior art, the present application provides a multi-agent graph reinforcement learning driven active power distribution network multi-time scale rolling optimization method, which systematically solves the collaborative optimization problem of topological reconfiguration, reactive power compensation and active dispatching under high proportion of renewable energy access through a time-space decoupling architecture and a hybrid solving engine. The core innovative design is as follows:

[0005] 1. Multi-time scale device hierarchical collaborative mechanism

[0006] Slow-time scale devices (response speed lower than the preset threshold): covering tie switches, switchable capacitor banks and on-load voltage regulating transformers, their discrete action decisions are optimized in 24 hours, and topological reconfiguration and voltage regulation are realized through global scheduling.

[0007] Fast-time scale devices (response speed higher than a preset threshold): including static var generators, micro gas turbines, energy storage systems, and soft switches, to generate fine active-reactive instructions for the next 8 time periods with a fixed step of 15 minutes, to suppress renewable energy fluctuations.

[0008] 2. Slow-scale discrete decision efficient solution engine

[0009] Action space dimension reduction: based on the basic loop of the distribution network, generate switch candidate vectors, and significantly compress the combination decision dimension by excluding conflicting action combinations of the same public branch.

[0010] Graph reinforcement learning decision:

[0011] Construct a global state space that integrates the topology adjacency matrix, node power-voltage state, and device action frequency;

[0012] Dynamically capture topology-operation coupling features using graph convolution networks, and combine double network architecture (main network + target network) to suppress Q value overestimation;

[0013] Design a reward-punishment balance mechanism: reward negative operation cost when constraints are met, and impose a penalty term when constraints are violated, to drive multi-agent collaborative optimization.

[0014] 3. Fast-scale continuous optimization real-time solution framework

[0015] Mixed integer second-order cone programming:

[0016] Voltage offset target linearization: introduce non-negative auxiliary variables to convert absolute value constraints;

[0017] Nonlinear power flow convex relaxation: convert the branch power-voltage relationship into a second-order cone form;

[0018] Topology constraint processing: use the big M method to ensure the feasibility of the radial network structure.

[0019] Rolling execution mechanism: based on intraday load and renewable energy prediction values, only execute the first time period optimization instruction to reduce computational complexity.

[0020] This scheme realizes:

[0021] Network loss reduction: topology reconstruction and reactive power compensation cooperate to reduce resistance loss;

[0022] Voltage safety improvement: active-reactive joint regulation to suppress over-limit phenomenon;

[0023] Enhanced new energy consumption: fast-scale devices quickly respond to fluctuations and reduce curtailment of wind and light.

[0024] The scheme of the application decomposes complex high-dimensional decision-making into graph reinforcement learning optimization of discrete actions and mathematical programming solution of continuous actions, breaks through the limitations of traditional single methods in calculation efficiency and optimization accuracy, and provides a feasible collaborative decision support for active power distribution networks.

[0025] The technical scheme specifically adopted by the application to solve the technical problem is:

[0026] A multi-agent graph reinforcement learning driven active power distribution network multi-time scale rolling optimization method, comprising the following steps:

[0027] Divide slow time scale devices and fast time scale devices, the slow time scale devices include discrete action devices with a response speed lower than a preset threshold, and the fast time scale devices include continuous action devices with a response speed higher than the preset threshold;

[0028] Model the slow time scale optimization as a multi-agent Markov game, define a global state space including a power distribution network topology adjacency matrix, node active power, reactive power, voltage amplitude and cumulative action number of each slow time scale device, and a joint action space corresponding to switch action, reactive power compensation device gear and voltage regulation device gear;

[0029] Generate a switch action candidate vector based on a basic loop of the power distribution network, and compress the action space dimension by excluding schemes with conflicting actions in the same common branch;

[0030] Fit an action value function of each agent using a graph neural network, generate a reduced discrete action strategy according to the global state space, and cooperatively optimize through a distributed decision and a global reward mechanism;

[0031] Based on the slow time scale optimization result, convert the fast time scale optimization problem into a mixed integer second order cone programming model and rollingly solve the instruction.

[0032] Further, the slow time scale devices include tie-in switches, switchable capacitor banks and on-load voltage regulation transformers;

[0033] The fast time scale devices include static var generators, micro gas turbines, energy storage systems and soft switches;

[0034] The gear adjustment action number of the on-load voltage regulation transformer is constrained by a preset upper limit.

[0035] Further, the global state space includes:

[0036] A normalized topology adjacency matrix;

[0037] A node active power, reactive power and voltage amplitude set;

[0038] Cumulative number of actions of each slow time scale device.

[0039] Further, the switch action candidate vector is generated based on the basic loop of the power distribution network, and the action space dimension is compressed by excluding schemes with conflicting actions of the same common branch, specifically as follows:

[0040] Generating a basic loop candidate vector without soft switching, each vector element corresponding to the switch number to be disconnected in a basic loop;

[0041] Deleting schemes with multiple switch actions of the same common branch in the candidate vector.

[0042] Further, the graph neural network comprises:

[0043] Topology feature extraction layer: fusion of adjacency matrix and node operating state through graph convolution operation;

[0044] Action value output layer: dual network architecture of main network and target network is adopted to fit the Q value function.

[0045] Further, the rolling solution by the mixed integer second-order cone programming method specifically includes:

[0046] Introducing non-negative auxiliary variables to linearize the absolute value of voltage offset target;

[0047] Converting the nonlinear power flow constraint into a second-order cone form;

[0048] Using the big M method to handle the radial topology constraint and the integer constraint of discrete devices.

[0049] Further, the decision cycle of the slow time scale optimization is 24 hours;

[0050] The fast time scale optimization generates control instructions for the next 8 time periods with a fixed step size of 15 minutes.

[0051] Further, the global reward mechanism comprises:

[0052] When the action meets the constraint, the reward value is the negative value of the operating cost;

[0053] When the action violates the constraint, the reward value is the negative value of the preset penalty term;

[0054] Wherein the operating cost includes network loss cost and voltage offset penalty.

[0055] Further, it further comprises:

[0056] Storing the state before optimization, action, reward value and new state to a shared experience replay pool;

[0057] Training the graph neural network parameters by batch sampling data.

[0058] And a computer device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method as described above when executing the computer program.

[0059] A non-transitory computer-readable storage medium having stored thereon a computer program, the computer program implementing the method as described above when executed by a processor.

[0060] Compared with the prior art, the present application and its preferred schemes at least include the following beneficial effects:

[0061] 1. Breakthrough high-dimensional discrete decision bottleneck

[0062] Through the basic loop coding and the action space dimension reduction strategy driven by the topological rule, the switch-capacitor collaborative decision problem of combinatorial explosion is compressed to a computable scale, solving the calculation failure problem of traditional mixed integer programming in device collaborative optimization.

[0063] The graph reinforcement learning fuses the topological adjacency relationship and the operating state feature, significantly improves the physical feasibility of the discrete action strategy, and avoids invalid oscillation operation.

[0064] 2. Realize multi-time scale collaborative optimization

[0065] Establish a "slow scale global optimization + fast scale rolling correction" double loop architecture to match the response speed difference of devices (slow scale mechanical action vs. fast scale power electronic regulation), and eliminate the response lag defect of single time scale optimization.

[0066] Based on the mixed integer second-order cone programming of the intra-day rolling optimization, while ensuring the efficiency of convex optimization solution, the voltage stability margin is maintained through voltage offset linearization and power flow constraint relaxation.

[0067] 3. Enhance new energy consumption and safety control ability

[0068] Under the distributed decision-making framework, the global reward function integrates network loss cost, voltage offset penalty and constraint violation penalty to drive multi-agent autonomous balance of economy and safety.

[0069] The fast scale device responds to renewable energy fluctuations at a minute level, reducing the curtailment of wind and light caused by prediction errors; the slow scale topology reconfiguration optimizes the power flow distribution dynamically, and suppresses the risk of voltage out-of-limit caused by reverse power reverse sending.

[0070] 4. Provide an extensible technical framework

[0071] The graph neural network architecture is compatible with the change of power distribution network topology, and the double network mechanism suppresses the overestimation deviation of Q value, providing an adaptive optimization basis for high penetration rate renewable energy scenarios.

[0072] Modular design of action space dimension reduction rules, mixed integer convex optimization, etc. support the collaborative control demand of accessing new power equipment (such as hydrogen energy storage and flexible DC).

[0073] The application and its preferred schemes deeply integrate the environment perception capability of reinforcement learning and the accurate solution capability of mathematical programming, form a “data-driven + model-driven” complementary optimization paradigm, and provide decision support with real-time and global performance for active distribution networks. BRIEF DESCRIPTION OF DRAWINGS

[0074] The application will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0075] Figure 1 The application is an embodiment of the overall flowchart. DETAILED DESCRIPTION

[0076] To make the features and advantages of the application more obvious and easy to understand, the following examples are described in detail as follows:

[0077] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise indicated, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0078] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms “comprise” and / or “include” are used in this specification, they refer to the presence of a feature, step, operation, device, component and / or combination thereof.

[0079] To solve the problems in the prior art, the application realizes the collaborative optimization of topology reconfiguration, reactive power compensation and active power scheduling by Markov game modeling of slow time scale equipment and mixed integer second-order cone programming rolling solution of fast time scale equipment; adopts a basic loop-based action space dimension reduction strategy and a distributed decision mechanism to compress the discrete device action space, and designs a multi-agent graph convolution network to integrate topology features and operating state, suppresses the Q value overestimation problem, and finally forms a “spatial and temporal decoupling-hybrid solution” rolling optimization framework. Application of the technical solution can effectively reduce the network loss of the distribution network, suppress the voltage out-of-limit phenomenon, improve the distributed energy consumption capacity, and realize the collaborative optimization of topology reconfiguration, reactive power compensation and active power scheduling.

[0080] As Figure 1As shown, it is the specific implementation process of the active power distribution network multi-time scale rolling optimization method driven by the multi-agent graph reinforcement learning of the embodiment of the application, including the following steps:

[0081] Step S1: initialize the day-ahead slow time scale active power distribution network dynamic reconstruction and reactive power compensation collaborative optimization model, and initialize the day-ahead fast time scale reactive power compensation and active regulation collaborative optimization model;

[0082] Step S2: divide the slow time scale equipment and the fast time scale equipment;

[0083] Step S3: model the day-ahead optimization as a Markov game ;

[0084] Step S4: adopt an action space dimension reduction strategy based on a basic loop to reduce the dimension of the switch action space;

[0085] Step S5: based on the day-ahead load and renewable energy prediction value, generate slow time scale equipment actions through a multi-agent graph reinforcement learning algorithm;

[0086] Step S6: determine whether the day-ahead optimization termination time T End1 is reached, if not, execute step S5; if yes, execute step S7;

[0087] Step S7: relax the day-ahead fast time scale into a mixed integer second order cone programming model;

[0088] Step S8: based on the day-ahead load and renewable energy prediction value, rollingly solve the fast time scale actions;

[0089] Step S9: determine whether the day-ahead optimization termination time T End2 is reached, if not, execute step S8; if yes, output the day-ahead slow time scale actions and the day-ahead fast time scale actions.

[0090] The following will specifically expand the introduction of the embodiment scheme:

[0091] I. Initialize the day-ahead slow time scale active power distribution network dynamic reconstruction and reactive power compensation collaborative optimization model, and initialize the day-ahead fast time scale reactive power compensation and active regulation collaborative optimization model.

[0092] (1) Day-ahead stage optimization target initialization: including: system operation cost and voltage deviation. Its expression is as follows:

[0093]

[0094]

[0095]

[0096] Where c loss is the unit network loss cost; P loss,t and are the network loss and voltage deviation of the distribution network at time t; T is the simulation time; C m,t is the operating cost of equipment m at time t; is the set of controllable devices in the day-ahead phase; N b is the number of branches in the distribution network; X b,t Indicates the state of the b-th branch at time t. When X b,t =0 means the branch is disconnected, X b,t =1 means closed; R b Represents the resistance of branch b; I b,t represents the current of branch b at time t; P b,t With Q b,t They represent the active and reactive power of branch b at time t; U b,t represents the terminal node voltage of branch b at time t; N is the number of nodes in the network; U i,t is the voltage of node i at time t; U N is the rated voltage.

[0097] (2) Initialization of the day-ahead constraints, including: power flow balance constraints, node voltage constraints, branch current constraints, radial constraints, switch operation times constraints, distributed generation output constraints, switchable capacitor constraints, and on-load tap-changing transformer constraints. The switch operation times constraints, switchable capacitor constraints, and on-load tap-changing transformer constraints are shown in Equations (4) to (6), respectively:

[0098]

[0099]

[0100]

[0101] Where, N represents the upper limit of the number of times all the reconfiguration switches involved in the distribution network reconfiguration can operate; IS is the number of reconfigurable switches in the distribution network; The upper limit of the number of actions for a single reconfiguration switch; is the reactive compensation amount of a single capacitor in the switchable capacitor bank at node j; is the maximum reactive compensation amount of the switchable capacitor bank at node j; The number of groups put into the SCB of node j at time t; The maximum number of actions of the switchable capacitor group of node j. In the formula, U j,t And The voltage of node j before and after voltage regulation at time t, respectively; The tap position of the on-load voltage regulation transformer of node j at time t, The voltage compensation value of a single tap; The maximum number of actions of the on-load voltage regulation transformer.

[0102] (3) Initialization of the intra-day stage optimization target: including the operation cost of the distribution network and the voltage deviation:

[0103]

[0104]

[0105] In the formula, k is the number of prediction time domain windows; The optimization step length; c DG The unit new energy reduction cost; The new energy reduction amount in the distribution network at time t; The set of controllable devices in the intra-day stage; The maximum active power output of the renewable energy of node i at time t.

[0106] (4) Initialization of the intra-day stage constraint condition: including the distributed power supply constraint, the static var generator constraint, the energy storage system constraint, the micro gas turbine constraint and the soft switch constraint. Among them, the energy storage system constraint, the micro gas turbine constraint and the soft switch constraint are shown in formulas (9)-(11), respectively:

[0107]

[0108]

[0109]

[0110] In the formula, And Indicate the charging and discharging states of the energy storage system of node j at time t, respectively; And Indicate the charging and discharging power of the energy storage system of node j at time t, respectively; And Indicate the upper limits of the charging power and discharging power of the energy storage system of node j, respectively;E j ,t Soj(t) represents the state of charge of the energy storage system of node j at time t; E j,min and E j,max represent the lower and upper limits of the state of charge of the energy storage system of node j, respectively; 、 represent the charging and discharging efficiency coefficients of the energy storage system of node j, respectively; Δt is the simulation time at time t; is the maximum charging and discharging times of the energy storage system of node j. In the formula, and represent the lower and upper limits of the active power output of the micro gas turbine of node j, respectively; is the upper limit of the active power ramp of the micro gas turbine of node j. In the formula, and are the active power injected by the soft switch on branch ij to nodes i and j at time t, respectively; and are the active power losses of the soft switch on branch ij at nodes i and j at time t, respectively; and are the power loss coefficients of the soft switch at nodes i and j, respectively; and are the active power injected by the soft switch on branch ij to nodes i and j at time t, respectively; is the soft switch capacity of branch ij.

[0111] II. Divide slow time scale devices and fast time scale devices.

[0112] (1) For discrete action devices with slow response speed, such as tie switches, switchable capacitor banks, and on-load tap-changing transformers, since their regulation process involves mechanical action or longer stabilization time, they are classified as slow time scale devices;

[0113] (2) For continuous action devices with fast response speed, such as static var generators, renewable energy inverters, micro gas turbines, energy storage systems, and soft switches, since they can quickly respond to system changes and achieve fine control, they are classified as fast time scale devices.

[0114] III. Model day-ahead optimization as a Markov game .

[0115] (1) Construct the global state space: In the Markov game of the slow time scale optimization problem, the global state s t observed by the agent at time t includes the topology-operation joint characteristics of the distribution network: 1) Topology characteristics: the normalized adjacency matrix at time t; 2) Node operation characteristics: the set of node active power P t, reactive power aggregate Q t And the node voltage set U t ;3) Equipment action times: the set of switch action times , Operation times of on-load tap-changing transformers And the set of operation times of all switchable capacitor banks All states t The set of is the global state space S of the agent:

[0116]

[0117] (2) Constructing joint action space:

[0118]

[0119] Where N SCB is the number of switchable capacitors; is the number of selectable gears for the i-th switchable capacitor; n OLTC The number of selectable gears of the on-load tap-changing transformer.

[0120] (3) Construct a global shared reward function. The global shared reward function r of each agent at time t is t It mainly includes three parts: minimizing the distribution network operation cost, minimizing the voltage deviation, and the penalty for violating the constraint conditions:

[0121]

[0122] Where m is a binary variable. When the action selected by the agent at time t satisfies the constraint conditions, its value is 1, otherwise its value is 0; p is the penalty term given to the agent when the selected action does not meet the constraint conditions.

[0123] 4. The switching action space is reduced in dimension by adopting the action space dimensionality reduction strategy based on basic loops.

[0124] (1) List all common branches CB and tie switch groups excluding soft switches;

[0125] (2) Generate an n-dimensional vector X={x1,x2,...,x n}, stores the action space of the switch, n is the number of basic loops without soft switches, and X in this paper is a three-dimensional vector, where the variable x k represents the open switch in the kth basic circuit;

[0126] (3) Select a tie switch b in each basic circuit without a soft switch i, generate a three-dimensional vector X, a total of 6*3*4=72 three-dimensional vectors are generated;

[0127] (4) For each three-dimensional vector X, the variable b i , b j (i≠j) is judged, if there is b i ∈CB ij and b j ∈CB ij , delete the scheme, otherwise keep the scheme.

[0128] Five, based on the day-ahead load and renewable energy prediction value, through multi-agent graph reinforcement learning algorithm to generate slow time scale equipment action.

[0129] (1) Initialize the main network parameters and target network parameters of each agent respectively, and determine the maximum iteration number E and the maximum random exploration number M;

[0130] (2) Initialize the power distribution network, get the normalized adjacency matrix, each node voltage, each node active power, each node reactive power and the initial action number of each discrete device, and construct the initial state space;

[0131] (3) If the iteration number is less than the maximum random exploration number M, each agent generates a random strategy, if the iteration number is greater than the maximum random exploration number M, each agent selects the optimal action a t according to the observed power distribution network state s t ;

[0132] (4) The power distribution network according to the switch action selected by each agent and the switchable capacitor and the on-load regulation transformer position;

[0133] (5) The power flow calculation is carried out on the actioned power distribution network, the state space s t+1 after reconstruction and reactive power compensation is obtained, and the reward function value r t is calculated;

[0134] (6) The state s t before this optimization, the optimization action a t , the optimization corresponding reward function value r t and the optimized state are stored to the shared experience replay pool D;

[0135] (7) Batch sample m samples {s t , a t , r t , s t+1} from the experience replay pool;

[0136] (8) If the iteration number reaches the maximum iteration number, end the training; otherwise, update the iteration number and execute step S52.

[0137] Six, determine whether to reach the day-ahead optimization termination time T End1 , if not, execute step five; if yes, execute seven.

[0138] Seven, relax the intraday fast time to a mixed integer second-order cone programming model.

[0139] (1) Relax the voltage offset:

[0140]

[0141] In the formula, b vol,i,t , c vol,i,t Non-negative auxiliary variable; M is a large enough positive number.

[0142] (2) Relax the power flow balance constraint:

[0143] (16)

[0144]

[0145]

[0146]

[0147] In the formula, P ij,t , Q ij,t respectively represent the active power flow and the reactive power flow of branch ij at time t; z ij =r ij +jx ij Indicates the impedance of branch ij; , respectively represent the parent node and the child node set of node j.

[0148] (3) Relax the node voltage constraint and current constraint:

[0149]

[0150]

[0151] In the formula, ; .

[0152] (4) Relax the equation (18) to equation (22) using the big M method:

[0153]

[0154] In the formula, M2 is a large positive number.

[0155] Eight, based on the intraday load and renewable energy prediction value, the fast time scale action is solved.

[0156] (1) Initialize the time identifier t;

[0157] (2) According to the load and renewable energy output prediction value in the prediction time domain and the slow time scale regulation resource action determined in advance, the fast time scale regulation resource optimization instruction in the future 8 time periods is generated with 15 min as the optimization step;

[0158] (3) Execute the first fast time scale regulation resource optimization instruction and update the time identifier t.

[0159] Nine, judge whether the intraday optimization termination time T is reached End2 , if not, step eight is executed; if yes, the slow time scale action in advance and the fast time scale action in the day are output.

[0160] The above scheme provided by the embodiment of the application provides a multi-agent driven active power distribution network multi-time scale rolling optimization method. First, a joint optimization mathematical model of topology reconfiguration, reactive power compensation and active power scheduling under multi-time scale is constructed with the optimization objectives of minimizing network loss, inhibiting voltage deviation and improving new energy consumption rate. Secondly, the slow time scale optimization problem in advance is modeled as a Markov game; at the same time, the dimension of the action space of discrete devices is reduced by using a coding method based on a loop and a distributed decision mechanism; further, a multi-agent graph reinforcement learning algorithm is used for solving. Finally, the fast time scale optimization problem in the day is relaxed into a mixed integer second order cone programming model. In summary, the application can break through the collaborative decision barrier of slow / fast time scale devices through a space-time decoupling architecture, form a complementary optimization engine by using the topological perception advantage of graph reinforcement learning and the real-time solving ability of mathematical programming, inhibit the dimension disaster of combined decision by using the action space dimension reduction strategy, ensure the feasibility of the algorithm in the complex power distribution network scene, and guarantee the compatibility of the scheduling instruction and the dynamic response of the device through the integrated design of detailed constraints, and provide reliable technical support for the high proportion of new energy access scene.

[0161] The application is used to solve the active power distribution network multi-time scale topology-reactive power-active power collaborative optimization problem under high proportion of renewable energy access. Compared with the prior art, the specific advantages are as follows:

[0162] Firstly, the application proposes a hierarchical coordination mechanism of slow / fast time scale devices, which connects the global topology optimization of tie switches, capacitor banks and on-load voltage regulating transformers through day-ahead scheduling, and the fluctuation suppression of static var generators, micro gas turbines, energy storage systems and soft switches through intra-day rolling optimization, breaking through the limitations of single optimization method.

[0163] Secondly, the application innovatively combines basic loop encoding mechanism and distributed decision architecture, significantly reduces the dimension scale of tie switch and switchable capacitor combination decision through multi-dimensional action space compression strategy, effectively avoids the dimension disaster problem; at the same time, the graph convolution network is used to dynamically capture the coupling characteristics of power distribution network topology structure and operating state, which improves the fitting accuracy of action value function and suppresses the overestimation deviation of Q value, and realizes the efficient solution of discrete device coordination strategy.

[0164] Finally, the application proposes a double-loop control mechanism of "day-ahead global optimization + intra-day error correction", the outer ring generates slow device scheduling strategy based on multi-agent graph reinforcement learning, and the inner ring rolls up the fast device output instruction according to the load and distributed energy prediction error, realizing the coordinated improvement of voltage safety margin and new energy consumption capacity.

[0165] In summary, through the hierarchical coordination optimization mechanism of slow / fast time scale devices, the action space dimension reduction strategy based on graph reinforcement learning and the topology-operation state perception ability enhancement technology, the application systematically solves the multi-dimensional regulation and control resource coordination optimization problem of active distribution network, significantly improves the overall optimization performance of network loss suppression, voltage safety control and new energy consumption capacity, and provides a feasible technical path for intelligent decision of distribution network under high proportion of renewable energy access environment.

[0166] Based on the same inventive concept, the application also provides a computer device, which comprises one or more processors and a memory for storing one or more computer programs; the program comprises program instructions, and the processor is used to execute the program instructions stored in the memory. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are used to implement one or more instructions, and are specifically used to load and execute one or more instructions in the computer storage medium to realize the above-mentioned method.

[0167] It should be further noted that based on the same inventive concept, the present application also provides a computer storage medium, which stores a computer program, and the computer program is run by a processor to execute the above method. The storage medium can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus.

[0168] It should be noted that unless otherwise defined, technical or scientific terms used in the present application should be understood as their ordinary meaning to those skilled in the art. The terms "first", "second" and similar words of comparison used in the present application do not indicate any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, and do not exclude other elements or objects. The terms "connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0169] The above is only the preferred embodiment of the present application, and is not intended to limit the other forms of the present application. Any skilled person in the art can modify or change the above disclosed technical content to equivalent embodiments. However, any simple modification, equivalent change and modification of the above embodiments made according to the technical essence of the present application without departing from the technical solution content of the present application still falls within the protection scope of the present application.

[0170] The application is not limited to the above best mode, and anyone can derive other various forms of multi-agent graph reinforcement learning driven active power distribution network multi-time scale rolling optimization method under the inspiration of the application, and any equivalent changes and modifications made in the application patent range shall be within the scope of the application.

Claims

1. A multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning, characterized in that: The following steps are involved: Dividing slow time scale devices into fast time scale devices, wherein the slow time scale devices include discrete action devices with a response speed lower than a preset threshold, and the fast time scale devices include continuous action devices with a response speed higher than a preset threshold; The slow-time-scale optimization is modeled as a multi-agent Markov game. A global state space is defined, which includes the distribution network topology adjacency matrix, node active power, reactive power, voltage amplitude, and the cumulative number of actions of each slow-time-scale device, as well as the joint action space corresponding to the switch actions, the gear positions of the reactive compensation device, and the gear positions of the voltage regulation device. Generate switching action candidate vectors based on the basic circuit of the distribution network, and compress the action space dimension by excluding conflicting actions on the same common branch; A graph neural network is used to fit the action value function of each agent, and a discrete action strategy with reduced dimensionality is generated based on the global state space. This strategy is then optimized collaboratively through distributed decision-making and a global reward mechanism. Based on the slow time scale optimization results, the fast time scale optimization problem is transformed into a mixed integer second-order cone programming model and the solution instruction is rolled.

2. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized by: The slow time scale equipment includes a tie switch, a switchable capacitor bank and an on-load tap-changing transformer; The fast time scale equipment includes a static VAR generator, a micro gas turbine, an energy storage system and a soft switch; Among them, the number of gear adjustment actions of the on-load tap-changing transformer is subject to a preset upper limit.

3. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized by: The global state space includes: Normalized topological adjacency matrix; The set of node active power, reactive power and voltage amplitude; The cumulative number of actions of each slow time scale device.

4. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized by: The method of generating the switching action candidate vector based on the basic circuit of the distribution network and compressing the action space dimension by excluding conflicting actions on the same common branch is specifically as follows: Generate a basic loop candidate vector without soft switches, where each vector element corresponds to the number of a switch to be disconnected in the basic loop; Delete the solutions with multiple switching actions on the same common branch in the candidate vector.

5. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized by: The graph neural network includes: Topological feature extraction layer: fuses the adjacency matrix and node operation status through graph convolution operation; Action value output layer: A dual network architecture of the main network and the target network is used to fit the Q-value function.

6. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized by: The rolling solution using the mixed integer second-order cone programming method specifically includes: A non-negative auxiliary variable is introduced to linearize the absolute value target of voltage offset; Convert nonlinear power flow constraints into second-order cone form; The big-M method is used to handle radial topology constraints and discrete device integer constraints.

7. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized by: The decision cycle of the slow time scale optimization is 24 hours; The fast time scale optimization generates control instructions for the next 8 time periods in a rolling manner with a fixed step length of 15 minutes.

8. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized by: The global reward mechanism includes: When the action satisfies the constraint, the reward value is the negative of the running cost; When an action violates a constraint, the reward value is the negative value of the preset penalty term; The operating costs include network loss costs and voltage deviation penalties.

9. The multi-timescale rolling optimization method for active distribution networks driven by multi-agent graph reinforcement learning according to claim 1 is characterized in that: Also includes: Store the state, action, reward value before optimization and the new state into the shared experience replay pool; Train graph neural network parameters by batch sampling data.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images