Source network load storage collaborative partition collaborative optimization method and related device

Abstract

The invention belongs to the technical field of new energy power generation, and discloses a source network load storage collaborative partition collaborative optimization method and a related device. The source network load storage collaborative partition collaborative optimization method comprises the following steps: acquiring historical data of a power grid load based on a selected power distribution network; and based on the obtained historical data of the power grid load, performing prediction by using the trained reinforcement learning model, and obtaining a regulation and control strategy prediction result of the regulation and control device in the selected power distribution network. According to the technical scheme disclosed by the invention, in the training process of the reinforcement learning model, a multi-agent reinforcement learning algorithm is adopted, and compared with reinforcement learning of a single agent architecture, the method has better efficiency and performance; besides, the reinforcement learning model adopts a topological change graph convolutional network model, and the trained reinforcement learning model can be suitable for different topological structures, so that the training cost required for model transplantation can be reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of renewable energy power generation, and in particular relates to a method and related device for collaborative optimization of source, grid, load and storage coordinated partitioning. Background Art

[0002] As the global pursuit of sustainable energy continues to heat up, the installed capacity of new energy in the distribution network has shown a significant upward trend. In the context of today's energy transformation, new energy such as solar energy and wind energy have become an important choice to meet the growing energy demand with their clean and renewable characteristics. As the proportion of renewable energy in the energy structure increases, its volatility and intermittent characteristics have brought huge challenges to the stable operation of the power grid; at the same time, the peak-to-valley difference of power load continues to expand and users' requirements for power supply reliability and power quality have increased, making the coordinated optimization between source, grid, load and storage inevitable. Explanatory, "source" mainly refers to various types of power generation sources, including traditional thermal power generation, hydropower generation, and new energy power generation (such as photovoltaic power generation, wind power generation, etc.); "grid" represents the transmission and distribution network, which is responsible for the transmission and distribution of electric energy; "load" refers to the power load, which refers to the equipment or system that consumes electric energy at the user end; "storage" refers to the energy storage system, such as battery energy storage, heat storage and cold storage, and other energy storage methods.

[0003] The topology of the power grid is complex, especially at the distribution network level. The access of distributed power sources and energy storage systems makes the power flow distribution more complicated. Excessive injection of active power may cause voltage fluctuations to exceed the standard threshold of the power grid. Voltage fluctuations can be mitigated by utilizing the control flexibility of the photovoltaic inverter itself and other controllable devices (for example, static VAR compensators (SVCs) and on-load tap changers (OLTCs)). However, a well-designed scheme is required to coordinate these devices on a large scale so that the voltage of the entire network can be adjusted with limited local information. However, the computational time cost of the optimal power flow algorithm (OPF) is very high, and it is difficult to effectively respond to voltage fluctuations in a short time.

[0004] At present, some studies have tried to use a single-agent deep reinforcement learning (DRL) model to model the reactive power optimization problem of the distribution network as a Markov decision process (MDP). For example, the actor-critic (AC) algorithm is used to minimize the network loss and the action cost of the equipment as the optimization goal, and the switching instructions of the discrete reactive power regulation equipment are used as the control variable for solving. In addition, some studies use the deterministic policy gradient (DPG) algorithm and use the reactive power of the inverter or static VAR compensator as the control action. However, the above existing studies only use the grid node data as features of the same dimension for reinforcement learning training, ignoring the impact of the grid topology modeling on the model. The reinforcement learning model trained by the existing method is difficult to achieve good results, and it is also difficult to apply to other grid scenarios with different topologies. Summary of the invention

[0005] The purpose of the present invention is to provide a method and related device for collaborative optimization of source-grid-load-storage synergy partitioning to solve one or more of the above-mentioned technical problems. In the technical solution provided by the present invention, a trained reinforcement learning model is used for prediction to obtain the prediction results of the control strategy of the control device in the selected distribution network; wherein, in the training process of the reinforcement learning model, a multi-agent reinforcement learning algorithm is used, which has better efficiency and performance than the reinforcement learning of a single agent architecture; in addition, the reinforcement learning model adopts a topology-changing graph convolutional network model, and the trained reinforcement learning model can be applied to different topological structures, which can reduce the training cost required for model transplantation.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for collaborative optimization of source-grid-load-storage collaborative partitioning, comprising the following steps: Based on the selected distribution network, obtain historical data of grid load; Based on the historical data of power grid load, the trained reinforcement learning model is used to make predictions to obtain the prediction results of the control strategy of the control device in the selected distribution network; Among them, the training steps of the reinforcement learning model include: based on the distribution network sample, obtaining the distribution network operation data and topology information, and constructing a mathematical model for the voltage and reactive power optimization of the distribution network; converting the constructed mathematical model into a Markov decision process model based on a multi-agent; constructing a topology change graph convolutional network model according to the distribution network operation data and topology information; based on the multi-agent Markov decision process model, using a multi-agent reinforcement learning algorithm to train the topology change graph convolutional network model to obtain a reinforcement learning model that can dynamically adapt to the topology changes of the distribution network and optimize the voltage and reactive power control; using the power grid data actually obtained based on the distribution network sample to perform offline training and verification on the obtained reinforcement learning model to obtain a trained reinforcement learning model.

[0007] A further improvement of the present invention is that The topology change graph convolutional network model includes: Input layer, used to input distribution network operation data and topology information; The learnable spectral graph convolution module with residual graphs is repeatedly stacked for multiple times, and is used to extract features of input distribution network operation data and topology information to obtain high-dimensional features of the distribution network operation data and topology information; wherein each learnable spectral graph convolution module with residual graph includes a learnable spectral graph convolution layer with residual graph, a first normalization layer and a pooling layer; The second normalization layer is used to normalize the obtained high-dimensional features to obtain distribution network optimization prediction information; The output layer is used to output the obtained distribution network optimization prediction information.

[0008] A further improvement of the present invention is that The topology change graph convolutional network model also includes: A bilateral filter layer is provided between a learnable spectral convolution module with a residual graph that is repeatedly stacked multiple times and a second normalization layer, and is used for filtering the high-dimensional features of the obtained distribution network operation data and topology information, obtaining the filtered high-dimensional features and inputting them into the second normalization layer.

[0009] A further improvement of the present invention is that The learnable spectral convolutional layer with residual map is expressed as: ; In the formula, is the output of the learnable spectral convolutional layer with residual map; is the activation function; is a learnable spectral convolution kernel with a residual map, The result of the operation is a learnable spectral convolution matrix with a residual map; is the eigenvector corresponding to each vertex on the graph; is a trainable weight matrix; is a trainable weight vector; ; In the formula, is a learnable Laplacian matrix with residual; About of k polynomial of order; k Iterate over the sequence numbers for the sum function; K is the total number of nodes in the graph; for Parameters of the polynomial in ; ; ; ; In the formula, L is the normalized graph Laplacian matrix; is a given constant value; is the identity matrix; is the distance metric matrix using generalized Mahalanobis distance; A is the adjacency matrix; , They are the randomly initialized distance metric matrix and adjacency matrix respectively; is the residual Laplacian matrix obtained from training when the distance metric and adjacency relations are randomly initialized.

[0010] A further improvement of the present invention is that The calculation expression of the distance metric matrix using generalized Mahalanobis distance is: ; In the formula, is the vertex on the graph corresponding to The eigenvector of , are trainable weights.

[0011] A further improvement of the present invention is that In the multi-agent based Markov decision process model, The control devices with independent action space in the distribution network are defined as intelligent agents; The state observation space is defined as: ; In the formula, is a set of active and reactive power loads; It is a group of active powers generated by power generation equipment; It is a group of reactive powers generated by power generation equipment; is a set of voltage amplitudes and the voltage phase in radians ; They are the active power, reactive power and phase angle of the load respectively; They are respectively the active power and reactive power generated by the power generation equipment; The action space of the agent is defined as: For a continuously controlled device, the agent is equipped with a continuous set of actions, represented as: ; In the formula, To regulate the output of the device; They represent the lower and upper output limits of the continuous control equipment respectively; For discrete control devices, the agent is equipped with a discrete action set, expressed as: ; In the formula, To regulate the output of the device; Represents all actions that the device can perform; Reward Function for: ; In the formula, This is the network loss penalty item; is the voltage deviation penalty term; ; In the formula, is a positive weight coefficient; Indicates network loss; ; In the formula, is the number of nodes in the distribution network; is a positive weight coefficient; Representation Node The voltage amplitude; Representation Node The reference voltage amplitude.

[0012] A further improvement of the present invention is that the control device in the distribution network includes a switch, an inverter and an energy storage device.

[0013] In a second aspect, the present invention provides a system for collaborative optimization of source, grid, load and storage, comprising: A data acquisition module, used for acquiring historical data of power grid load based on the selected distribution network; A prediction module is used to make predictions based on the acquired historical data of power grid loads using a trained reinforcement learning model to obtain prediction results of the control strategy of the control device in the selected distribution network; Among them, the training steps of the reinforcement learning model include: based on the distribution network sample, obtaining the distribution network operation data and topology information, and constructing a mathematical model for the voltage and reactive power optimization of the distribution network; converting the constructed mathematical model into a Markov decision process model based on a multi-agent; constructing a topology change graph convolutional network model according to the distribution network operation data and topology information; based on the multi-agent Markov decision process model, using a multi-agent reinforcement learning algorithm to train the topology change graph convolutional network model to obtain a reinforcement learning model that can dynamically adapt to the topology changes of the distribution network and optimize the voltage and reactive power control; using the power grid data actually obtained based on the distribution network sample to perform offline training and verification on the obtained reinforcement learning model to obtain a trained reinforcement learning model.

[0014] A further improvement of the present invention is that The topology change graph convolutional network model includes: Input layer, used to input distribution network operation data and topology information; The learnable spectral graph convolution module with residual graphs is repeatedly stacked for multiple times, and is used to extract features of input distribution network operation data and topology information to obtain high-dimensional features of the distribution network operation data and topology information; wherein each learnable spectral graph convolution module with residual graph includes a learnable spectral graph convolution layer with residual graph, a first normalization layer and a pooling layer; The second normalization layer is used to normalize the obtained high-dimensional features to obtain distribution network optimization prediction information; The output layer is used to output the obtained distribution network optimization prediction information.

[0015] A further improvement of the present invention is that The topology change graph convolutional network model also includes: A bilateral filter layer is provided between a learnable spectral convolution module with a residual graph that is repeatedly stacked multiple times and a second normalization layer, and is used for filtering the high-dimensional features of the obtained distribution network operation data and topology information, obtaining the filtered high-dimensional features and inputting them into the second normalization layer.

[0016] A further improvement of the present invention is that The learnable spectral convolutional layer with residual map is expressed as: ; In the formula, is the output of the learnable spectral convolutional layer with residual map; is the activation function; is a learnable spectral convolution kernel with a residual map, The result of the operation is a learnable spectral convolution matrix with a residual map; is the eigenvector corresponding to each vertex on the graph; is a trainable weight matrix; is a trainable weight vector; ; In the formula, is a learnable Laplacian matrix with residual; About of k polynomial of order; k Iterate over the sequence numbers for the sum function; K is the total number of nodes in the graph; for Parameters of the polynomial in ; ; ; ; In the formula, L is the normalized graph Laplacian matrix; is a given constant value; is the identity matrix; is the distance metric matrix using generalized Mahalanobis distance; A is the adjacency matrix; , They are the randomly initialized distance metric matrix and adjacency matrix respectively; is the residual Laplacian matrix obtained from training when the distance metric and adjacency relations are randomly initialized.

[0017] A further improvement of the present invention is that The calculation expression of the distance metric matrix using generalized Mahalanobis distance is: ; In the formula, is the vertex on the graph corresponding to The eigenvector of , are trainable weights.

[0018] A further improvement of the present invention is that In the multi-agent based Markov decision process model, The control devices with independent action space in the distribution network are defined as intelligent agents; The state observation space is defined as: ; In the formula, is a set of active and reactive power loads; It is a group of active powers generated by power generation equipment; It is a group of reactive powers generated by power generation equipment; is a set of voltage amplitudes and the voltage phase in radians ; They are the active power, reactive power and phase angle of the load respectively; They are respectively the active power and reactive power generated by the power generation equipment; The action space of the agent is defined as: For a continuously controlled device, the agent is equipped with a continuous set of actions, represented as: ; In the formula, To regulate the output of the device; They represent the lower and upper output limits of the continuous control equipment respectively; For discrete control devices, the agent is equipped with a discrete set of actions, expressed as: ; In the formula, To regulate the output of the device; Represents all actions that the device can perform; Reward Function for: ; In the formula, This is the network loss penalty item; is the voltage deviation penalty term; ; In the formula, is a positive weight coefficient; Indicates network loss; ; In the formula, is the number of nodes in the distribution network; is a positive weight coefficient; Representation Node The voltage amplitude of Representation Node The reference voltage amplitude.

[0019] A further improvement of the present invention is that the control device in the distribution network includes a switch, an inverter and an energy storage device.

[0020] A third aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for collaborative optimization of source-grid-load-storage collaborative partitioning as described in any one of the first aspect of the present invention is implemented.

[0021] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for collaborative optimization of source-grid-load-storage collaborative partitioning as described in any one of the first aspect of the present invention is implemented.

[0022] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method for collaborative optimization of source-grid-load-storage collaborative partitioning, which uses a trained reinforcement learning model for prediction to obtain the prediction results of the control strategy of the control device in the selected distribution network; wherein, in the training process of the reinforcement learning model, a multi-agent reinforcement learning algorithm is used, which has better efficiency and performance than the reinforcement learning of a single agent architecture; in addition, the reinforcement learning model adopts a topology-changing graph convolutional network model, and the trained reinforcement learning model can be applied to different topological structures, which can reduce the training cost required for model transplantation. Further explanatory, the multi-agent reinforcement learning algorithm (MARL) performs very well when the centralized control mechanism cannot well predict the next behavior of each individual. There are many problems involving the collaborative or competitive behavior of a large number of agents in the distribution network power network, and it is suitable to use MARL as the training framework of the power network optimization model.

[0023] The present invention improves the spectral convolution kernel in the reinforcement learning model and adopts a learnable Laplace matrix, so that the spectral convolution kernel can be applied to data with diverse graph topological structures, improves the generalization of the topological structure of the network model, and the trained model can be applied to different topological structures, reducing the training cost required for model transplantation. The present invention proposes a residual Laplace matrix to create a graph structure in an unsupervised manner, improves the model's perception of the inline relationship of nodes in the graph, and makes dynamic adjustments to adapt to changes in the topological structure of the power grid (such as switches and load changes).

[0024] In the reinforcement learning model proposed in the present invention, when measuring the relationship between nodes, the generalized Mahalanobis distance is used instead of the Euclidean distance as the distance metric, and the generalized Mahalanobis distance is used as a parameter for training. It can be adaptive according to the topological structure and the relationship between nodes, so that the constructed Laplace matrix can more reasonably predict the loss in an environment with a changing topological structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below; obviously, the drawings described below are some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 It is a flow chart of a method for collaborative optimization of source-grid-load-storage collaborative partitioning in an embodiment of the present invention; Figure 2 It is a flow chart of a method for collaborative optimization of source-grid-load-storage collaborative partitioning in a specific embodiment of the present invention; Figure 3 It is a schematic diagram of a topology change graph convolutional network model in an embodiment of the present invention; Figure 4 It is a schematic diagram of a system for collaborative optimization of source, grid, load and storage partitions in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments and technical solutions are only part of the embodiments of the present invention, not all of the embodiments.

[0028] All other embodiments obtained by those of ordinary skill in the art without creative work based on the technical solutions disclosed in the embodiments of the present invention belong to the scope of protection of the present invention. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] See also Figure 1 , a method for collaborative optimization of source-grid-load-storage collaborative partitioning provided by an embodiment of the present invention comprises the following steps: Step 1, based on the selected distribution network, obtaining historical data of power grid load; Step 2: Based on the historical data of the power grid load obtained in step 1, a trained reinforcement learning model is used to perform prediction to obtain a prediction result of the control strategy of the control device in the selected distribution network; in the exemplary technical solution, the control device in the distribution network may include a switch, an inverter, and an energy storage device, etc.; The training steps of the reinforcement learning model include: Step (1), based on the distribution network sample, obtain the distribution network operation data and topology information, and construct a mathematical model for voltage and reactive power optimization of the distribution network; Step (2), converting the mathematical model constructed in step (1) into a Markov decision process model based on multi-agents; Step (3), constructing a topology change graph convolutional network model based on the distribution network operation data and topology information; Step (4), based on the multi-agent Markov decision process model obtained in step (2), a multi-agent reinforcement learning algorithm (for example, such as Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments, Group Relative Policy Optimization, etc.) is used to train the topology change graph convolutional network model obtained in step (3), so as to obtain a reinforcement learning model that can dynamically adapt to the topology change of the distribution network and optimize the voltage and reactive power control; In step (5), the reinforcement learning model obtained in step (4) is offline trained and verified using the actual (i.e., real) power grid data obtained based on the distribution network sample to obtain a trained reinforcement learning model; further explanatory, the trained reinforcement learning model can be saved and then its effectiveness and scalability when the distribution network topology changes are verified.

[0030] In the technical solution provided by the embodiment of the present invention, by constructing a Markov decision process model based on a multi-agent and training it with a multi-agent reinforcement learning algorithm, the intelligent decision-making of the voltage and reactive power control of the distribution network is realized. The reinforcement learning model can automatically predict and optimize the control strategy of the control device according to the real-time status and historical data of the distribution network without manual intervention; in addition, by using the trained reinforcement learning model, the voltage and reactive power in the distribution network can be automatically optimized and controlled, which improves the timeliness and accuracy of the control and reduces the errors and delays caused by human factors. In the present invention, by designing a topology change graph convolutional network model and combining it with a multi-agent reinforcement learning algorithm for training, the reinforcement learning model can dynamically adapt to the changes in the topological structure of the distribution network. When the line, load or power generation facility in the distribution network changes, the reinforcement learning model can quickly adjust its strategy and maintain the optimization performance; in addition, the technical solution of the present invention is not only applicable to a specific distribution network, but can also be expanded to other similar distribution network environments by adjusting the model parameters and structure. This scalability makes the technical solution of the present invention have a wide range of application prospects. The present invention can significantly reduce the voltage deviation and network loss of the distribution network and improve the power supply quality through the prediction and optimization of the reinforcement learning model, which not only helps to improve the stability and reliability of the power system, but also can reduce energy waste and reduce operating costs. The present invention obtains a trained reinforcement learning model through offline training, which can be quickly deployed and used in practical applications. This offline training and real-time application method improves the training efficiency of the model and reduces the complexity of real-time calculation. In summary, the technical solution of the embodiment of the present invention applies improved technical means such as multi-agent reinforcement learning algorithm and topology change graph convolution network model to the field of voltage and reactive power optimization of distribution network, which not only improves the intelligence and automation level of voltage and reactive power control of distribution network, but also enhances dynamic adaptability and scalability, and provides new ideas and methods for the optimization control of distribution network.

[0031] See also Figure 2 and Figure 3 In a specific embodiment of the present invention, the training step of the reinforcement learning model specifically includes the following steps: Step S101, constructing a mathematical model for voltage and reactive power optimization of a distribution network; wherein the mathematical model includes: an optimization target, equality constraints and inequality constraints for power flow calculation; In a specific exemplary technical solution, medium voltage (10-24 kV) and low voltage (0.23-1 kV) distribution networks are considered, and the distribution network is modeled as , and ) represent the node (bus) set and the edge (branch) set respectively; among them, node (bus) 0 is regarded as the connection point with the main power grid, balancing the active and reactive power in the distribution network; By coordinating a variety of control devices, the voltage of the entire distribution network is adjusted. The voltage quality of the distribution network is an important indicator for measuring the quality of power supply. Reactive power has a significant impact on the voltage amplitude. Reasonable reactive power optimization can maintain the node (bus) voltage within the specified range, and the voltage deviation can be used to measure the voltage quality.

[0032] For safety and optimal operation, a 5% voltage deviation is usually allowed, and the target voltage formula is: ; in, For Node The voltage per unit value; It represents per unit value, dimensionless unit; In reactive power optimization of distribution networks, minimizing network losses is an important goal; explanatory speaking, network losses are mainly due to power losses caused by line resistance, and reasonable configuration of reactive power can effectively reduce network losses.

[0033] The total active power loss of the distribution network can be expressed by the following formula: ; in, is the total active power loss of the distribution network; is the number of lines; It is The resistance of the line; It is The voltage across the line, It is Active power and reactive power on the line; The optimization goal is to make Minimum, that is .

[0034] For each bus ,make and are the magnitude and phase angle of the complex voltage, For complex power injection, the active and reactive power injection can be defined as follows: ; ; in, Is connected to the bus The index set of the busbars; and It is a branch road Conductance and susceptance on ; It is a busbar and The phase difference between and It is a busbar The active and reactive power of the upper generating facility (interpretatively, if there is none on the bus, it is zero); and It is a busbar The active and reactive powers of the loads on the bus (interpretatively, they are zero if there is no load on the bus); this formula can represent the power system dynamics and is essential for solving power flow problems and active voltage control problems.

[0035] Step S102, converting the mathematical model constructed in step S101 into a Markov decision process model based on multi-agents; In the exemplary technical solution, the mathematical model is converted into a Markov decision process (MDP) model to describe the specific solution process of reinforcement learning; wherein a partially observable multi-agent Markov game is considered, and it is assumed that communication between all agents is available; at the time step , Agent Observe your own observations , which is the global environment state Based on my own observations and its previous time step message Plus the messages received from other agents , select action and information ; Assume that the agent News is sent to all other agents and is sent to them at the next time step (i.e., time step ), which is available in other agents; The combined effect of , and according to the transition probability And the reward function respectively, and They are the environment state space and the joint action space respectively. The goal is to find a strategy Maximize its discount benefits Therefore, the agent The objective function is defined as ,in and are the common strategy and the discount factor respectively.

[0036] Step S103, designing a topology change graph convolutional network model according to the topological characteristics of the distribution network; wherein, for example, Figure 3 As shown, the structure of the topology change graph convolutional network model includes: Input layer, used to input distribution network operation data and topology information; Repeat stacking N The spectral graph convolution module with residual graph is used to extract features of input distribution network operation data and topology information to obtain high-dimensional features of distribution network operation data and topology information; wherein each spectral graph convolution module with residual graph includes a spectral graph convolution layer with residual graph, a normalization layer and a pooling layer; wherein, N is an adjustable parameter; In a preferred solution, a bilateral filter layer may be included to filter the obtained high-dimensional features of the distribution network operation data and topology information to obtain filtered high-dimensional features; A normalization layer is used to normalize the high-dimensional features of the obtained distribution network operation data and topology information or the filtered high-dimensional features to obtain distribution network optimization prediction information; The output layer is used to output the obtained distribution network optimization prediction information.

[0037] Step S104, designing a distribution network optimization reinforcement learning framework based on multi-agent reinforcement learning, and training a topology change graph convolutional network model based on a multi-agent reinforcement learning MARL algorithm.

[0038] Step S105, using real distribution network data to perform offline training on the reinforcement learning model, saving the trained reinforcement learning model, and then verifying its effectiveness and scalability when the distribution network topology changes.

[0039] The application scenario of the technical solution of the embodiment of the present invention is to predict, based on the historical data of the power grid load, how to regulate the control strategies of control devices such as switches, inverters and energy storage devices in the next period of time, so as to maximize the reduction of active power loss in the power grid and meet safety requirements.

[0040] The core improvement means of the technical solution of the embodiment of the present invention is to transform the topological structure of the power grid into a graph network, conform to the constraints of the power flow calculation equation, describe a Markov decision process, and adopt the actor-critic (AC) algorithm based on the multi-agent reinforcement learning (MARL) algorithm to minimize the network loss and the action cost of the equipment as the optimization goal, design a reward function to evaluate the operation status of the power grid during operation, and predict the operation training reinforcement learning model for the next period through the optimized graph convolution network. Further improvements based on this are that the main module of the graph convolution network is the spectral convolution layer with residuals, the generalized Mahalanobis distance is used as the distance metric, and the learnable Laplace matrix is ​​used as the topological representation matrix. At the same time, the Laplace matrix with residuals is trained to perceive the intrinsic connection between nodes, and the intra-node and inter-node mapping is constructed through feature transformation reparameterization, thereby forming the integral part of the general convolution layer with residuals.

[0041] In the embodiment of the present invention, a learnable Laplace matrix is ​​used to make the spectral convolution kernel applicable to data with diverse graph topological structures, improve the generalization of the topological structure of the network model, and the trained model can be applicable to different topological structures, reducing the training cost required for model transplantation. The reinforcement learning model proposed in the present invention uses the generalized Mahalanobis distance instead of the Euclidean distance as the distance metric when measuring the relationship between nodes, and the generalized Mahalanobis distance is used as a parameter for training. It can be adaptive according to the relationship between the topological structure and the nodes, so that the constructed Laplace matrix can more reasonably predict the loss in the environment where the topological structure changes. The influence relationship between power grid nodes is related to the distance. The closer the distance, the greater the influence of the node. The present invention re-parameterizes the feature transformation in the convolutional network to construct a mapping of the features within and between nodes in the topological structure to improve the model accuracy. The present invention proposes a residual Laplace matrix to create a graph structure in an unsupervised manner, improve the model's perception of the inline relationship between nodes in the graph, and make dynamic adjustments to adapt to changes in the topological structure of the power grid (such as switches and load changes).

[0042] In a specific embodiment of the present invention, a spectral graph convolution layer with a residual graph is proposed, which is improved from the spectral convolution layer; wherein, The present invention improves the spectral convolution kernel in the reinforcement learning model, and adds a learnable Laplace matrix to the spectral convolution kernel parameter learning process, so that the model can adapt to more diverse power grid topology structures; in addition, the present invention proposes a learnable Laplace matrix with residuals. Considering the existence of discrete variables such as switches in the power grid topology, the spectral graph convolution layer with residual graph can explore networks with similar features to the original topology graph by training the residual graph; furthermore, the present invention adopts the generalized Mahalanobis distance as the distance metric in the Laplace matrix, which can be adaptive according to the relationship between the topological structure and the nodes, and can more reasonably characterize the topological structure of the power grid nodes; finally, the present invention introduces a transformation matrix and a deviation vector applied to the output features in the spectral convolution layer after topology optimization to construct mapping within and between nodes; Specifically, the formula of the spectral convolution layer is: ; In the formula, is the output of the spectral convolutional layer; is the activation function; is the spectral convolution kernel; The result of the operation is the spectral convolution matrix; is the eigenvector corresponding to each vertex on the graph; Spectral convolution kernel It can be expressed as: ; In the formula, It is an arbitrary trainable parameter, which is initialized and then adjusted using error back propagation; is the characteristic matrix of the Laplace matrix, that is .

[0043] In the embodiment of the present invention, the description of the spectral convolution layer with residual graph is: given a graph and its adjacency matrix , distance metric matrix , a learnable Laplacian matrix with residual The formula is as follows: ; ; ; In the formula, is the specified value, usually 0.1; symmetric semi-positive definite matrix , is a trainable weight parameter, is the eigenvector corresponding to each vertex in the graph.

[0044] In the embodiment of the present invention, the distance measurement matrix of the generalized Mahalanobis distance is used The calculation method is as follows: ; In the formula, is the vertex on the graph corresponding to The feature vector of .

[0045] In the embodiment of the present invention, the formula of the spectral convolution layer with residual is: ; In the formula, is the output of the learnable spectral convolutional layer with residual map; is the activation function; is a learnable spectral convolution kernel with a residual map, The result of the operation is a learnable spectral convolution matrix with a residual map; is the eigenvector corresponding to each vertex on the graph; is a trainable weight matrix; is a trainable weight vector; Given the initial Laplacian matrix ,feature and parameters ,function Output a learnable Laplacian matrix Spectrum, convolution kernel It is expressed as: ; In the formula, is a learnable Laplacian matrix with residual, is the eigenvector corresponding to each vertex in the graph.

[0046] In the embodiment of the present invention, given a diagram and its adjacency matrix and the distance metric matrix , the normalized graph Laplacian matrix It can be obtained through the following methods: ; In the formula, the matrix is the distance measure between nodes, here it is the distance measure of generalized Mahalanobis distance; matrix A is the adjacency matrix.

[0047] Interpretatively, the Laplacian matrix Determines the connectivity between nodes and the degree of vertices, the Laplacian matrix The only expression This study proposes a learnable Laplacian matrix that converts the Laplacian matrix Parameterize, given the initial Laplacian matrix ,feature and parameters ,function Output a learnable Laplacian matrix 's score.

[0048] The convolution kernel will be represented as: ; The main statement of the improved spectral convolution layer is: ; Since dense matrix multiplication , the complexity of this formula is ;if Approximated as A polynomial function of and can be calculated recursively, then due to the Laplace matrix The sparsity of , using Chebyshev expansion to calculate Polynomial .

[0049] This paper proposes a learnable Laplacian matrix with residuals. Considering the existence of discrete variables such as switches in the power grid topology, the spectral convolution layer with residual graph can explore networks with similar features to the original topology graph by training the residual graph. Since there is no prior knowledge about the distance metric, the metric is randomly initialized, so it may take a long time to converge. To speed up training and increase the stability of the learned graph topology, a learnable graph Laplacian with residuals is used. is the Laplace matrix A small offset is made based on: ; In the formula, It is a configurable parameter, usually 0.1; is the Laplacian matrix obtained by training when the metric M is randomly initialized; Laplacian Matrix A large amount of useful graph structure information has been revealed, except for those substructures consisting of virtual vertex connections that cannot be directly learned on the intrinsic graph; therefore, the residual Laplacian matrix The impact on the final graph topology is determined by control.

[0050] The present invention adopts the generalized Mahalanobis distance as the distance metric in the Laplace matrix, which can be adaptive according to the relationship between the topological structure and the nodes, and can more reasonably characterize the topological structure of the power grid nodes. For the power grid topological structure, the discretized data of the graph structure is established, and the Euclidean distance is no longer a good metric for measuring the similarity of vertices. Therefore, during the training process, the distance metric needs to change adaptively with the task and features. In metric learning, the algorithm is divided into supervised learning and unsupervised learning. The optimal metric obtained in an unsupervised manner minimizes the intra-class distance and maximizes the inter-class distance at the same time. For a labeled data set, the learning goal is to find a metric that minimizes the loss. and The generalized Mahalanobis distance formula between is: ; if , the formula degenerates into the Euclidean distance. In the model of the present invention, the symmetric semi-positive definite matrix ,in are trainable weights.

[0051] Is to measure and This distance is then used to compute the Gaussian kernel: ; right After normalization, we get a dense adjacency matrix In the model of the present invention, the optimal metric is to construct the graph Laplacian set that minimizes the prediction loss The measure of .

[0052] The present invention introduces a transformation matrix and a deviation vector applied to the output features in the spectral convolution layer after topology optimization to construct mapping within and between nodes.

[0053] In a classic convolutional neural network, the output features of a convolutional layer are the sum of all feature maps of the previous layer, which means that new features are not only constructed based on adjacent vertices, but also depend on other intra-vertex features. In a power grid topology environment, it is difficult to explain the creation and training of separate topologies for different vertex features on the same graph. In order to construct intra-node and inter-node mappings, a transformation matrix and bias vector applied to the output features are introduced in the spectral convolution layer after topology optimization. The reparameterization formula for the output features is: ; In the Layer, transformation matrix ,deviation and measurement Train together, is the feature dimension. In each spectral convolution layer, we have a total of Parameters of learning complexity , which is independent of the size or degree of the input graph. In the next spectral convolution layer, the spectral convolution kernel will be constructed in another feature domain with a different metric.

[0054] In the embodiment of the present invention, further explanation of the multi-agent reinforcement learning algorithm used is as follows: 1) Problem description The multi-agent reinforcement learning algorithm proposed in this paper can be used in large power networks with inverters, energy storage devices, switches and other control devices. Each device can be controlled in a distributed manner, so it can be regarded as an agent. In order to ensure the safety of the distribution network, all agents in a region share observations of the region.

[0055] 2) Mathematical modeling Since each agent can only observe part of the information of the entire power grid, and ensuring the security of the power grid is a common goal among the agents, the problem is modeled as a distributed partially observable Markov decision process (Dec-POMDP) ​​that can be described as a 10-tuple, where is a collection of agents, is a state set, is a collection of actions. is the reward function, is the set of observations, is the state transition function, is the observation probability, is the initial state probability distribution, is the discount factor.

[0056] Agents: There is a set of agents that control a set of control devices (i.e., control devices with independent action spaces are defined as agents), denoted as Each agent is located at a node in the graph (i.e., the power network). In this embodiment of the present invention, a function is defined To indicate the node where each agent is located.

[0057] Area: The entire power network is divided into regions, and their union is recorded as ,in and (if ). We define a function To map nodes to their regions.

[0058] The state observation space of the multi-agent Markov decision process model is defined as:

[0059] in, is a set of active and reactive power loads; It is a group of active powers generated by power generation equipment; It is a group of reactive powers generated by power generation equipment; is a set of voltage amplitudes and the voltage phase in radians ; They are the active power, reactive power and phase angle of the load respectively; They are respectively the active power and reactive power generated by the power generation equipment;

[0060] In the embodiment of the present invention, a function is defined ; The function defined above will A subset of is mapped to its associated metric, where represents the power set; An observation is defined as ,in Indicates in the agent The measurement within the region.

[0061] The action space of the agent: For continuously controlled devices (such as inverters), each The agent is equipped with a continuous action set: ; In the formula, To regulate the output of the device; They represent the lower and upper output limits of the continuous control equipment respectively; Explanatory, continuous action means the maximum reactive power it produces, i.e. The reactive power generated by each inverter is ,in It is The maximum apparent power that a device can produce based on its physical capacity. , which means that reactive power is exported to the distribution grid. , which means absorbing reactive power from the distribution grid. The value of is usually selected according to the capacity of the distribution network for the sake of safe operation.

[0062] For discrete control devices (such as switches), each The agent is equipped with a discrete action set: ; In the formula, To regulate the output of the device; Represents all actions that the device can perform. Joint actions are defined as .

[0063] State transition probability function: Since the state includes the last action and the load change is random (theoretically it can be modeled as any probability distribution), the state transition probability function can be defined as: ; It follows a Markov process, specifically, ; in, and ; Define the understanding space, and Describes the load variation. is a very short time interval, much shorter than the time interval between two controls (i.e., a time step). .

[0064] Observation probability function: In the context of power networks, it describes the measurement errors that can occur in sensors. Mathematically, it can be defined as Specifically, ;in, is a multivariate isotropic Gaussian distribution, Depends on the physical properties of the sensor.

[0065] Reward function for the multi-agent Markov decision process model: First, the network loss penalty item is calculated. The greater the network loss, the greater the penalty. Define network loss penalty: ; in, is a positive weight coefficient used to adjust the importance of network loss in the reward function.

[0066] For voltage stability, calculate the voltage deviation penalty term; Voltage deviation can be measured using the root mean square error (RMSE): ; in, is another positive weight coefficient; Finally, the reward function is designed as: ; in, represents the number of nodes in the distribution network, Indicates network loss, Representation Node The voltage amplitude, Representation Node The reference voltage amplitude.

[0067] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0068] See also Figure 4 In an embodiment of the present invention, a system for collaborative optimization of source-grid-load-storage collaborative partition is provided, including: A data acquisition module, used for acquiring historical data of power grid load based on the selected distribution network; A prediction module is used to make predictions based on the acquired historical data of power grid loads using a trained reinforcement learning model to obtain prediction results of the control strategy of the control device in the selected distribution network; Among them, the training steps of the reinforcement learning model include: based on the distribution network sample, obtaining the distribution network operation data and topology information, and constructing a mathematical model for the voltage and reactive power optimization of the distribution network; converting the constructed mathematical model into a Markov decision process model based on a multi-agent; constructing a topology change graph convolutional network model according to the distribution network operation data and topology information; based on the multi-agent Markov decision process model, using a multi-agent reinforcement learning algorithm to train the topology change graph convolutional network model to obtain a reinforcement learning model that can dynamically adapt to the topology changes of the distribution network and optimize the voltage and reactive power control; using the power grid data actually obtained based on the distribution network sample to perform offline training and verification on the obtained reinforcement learning model to obtain a trained reinforcement learning model.

[0069] In one embodiment of the present invention, a computer device is provided, the computer device comprising a processor and a memory, the memory being used to store a computer program, the computer program comprising program instructions, and the processor being used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in a computer storage medium to implement a corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used to perform the operation of the method for collaborative optimization of source-grid-load-storage collaborative partitioning.

[0070] In one embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM (Random Access Memory) memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the method for collaborative optimization of source-grid-load-storage collaborative partition in the above embodiment.

[0071] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program codes.

[0072] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for collaborative optimization of source-grid-load-storage collaborative partitioning, characterized in that: The following steps are involved: Based on the selected distribution network, obtain historical data of grid load; Based on the historical data of power grid load, the trained reinforcement learning model is used to make predictions to obtain the prediction results of the control strategy of the control device in the selected distribution network; Among them, the training steps of the reinforcement learning model include: based on the distribution network sample, obtaining the distribution network operation data and topology information, and constructing a mathematical model for the voltage and reactive power optimization of the distribution network; converting the constructed mathematical model into a Markov decision process model based on a multi-agent; constructing a topology change graph convolutional network model according to the distribution network operation data and topology information; based on the multi-agent Markov decision process model, using a multi-agent reinforcement learning algorithm to train the topology change graph convolutional network model to obtain a reinforcement learning model that can dynamically adapt to the topology changes of the distribution network and optimize the voltage and reactive power control; using the power grid data actually obtained based on the distribution network sample to perform offline training and verification on the obtained reinforcement learning model to obtain a trained reinforcement learning model.

2. The method for collaborative optimization of source-grid-load-storage collaborative partitioning according to claim 1 is characterized in that: The topology change graph convolutional network model includes: Input layer, used to input distribution network operation data and topology information; The learnable spectral graph convolution module with residual graphs is repeatedly stacked for multiple times, and is used to extract features of input distribution network operation data and topology information to obtain high-dimensional features of the distribution network operation data and topology information; wherein each learnable spectral graph convolution module with residual graph includes a learnable spectral graph convolution layer with residual graph, a first normalization layer and a pooling layer; The second normalization layer is used to normalize the obtained high-dimensional features to obtain distribution network optimization prediction information; The output layer is used to output the obtained distribution network optimization prediction information.

3. The method for collaborative optimization of source-grid-load-storage collaborative partitioning according to claim 2 is characterized in that: The topology change graph convolutional network model also includes: A bilateral filter layer is provided between a learnable spectral convolution module with a residual graph that is repeatedly stacked multiple times and a second normalization layer, and is used for filtering the high-dimensional features of the obtained distribution network operation data and topology information, obtaining the filtered high-dimensional features and inputting them into the second normalization layer.

4. The method for collaborative optimization of source-grid-load-storage collaborative partitioning according to claim 2 is characterized in that: The learnable spectral convolutional layer with residual map is expressed as: ； In the formula, is the output of the learnable spectral convolutional layer with residual map; is the activation function; is a learnable spectral convolution kernel with a residual map, The result of the operation is a learnable spectral convolution matrix with a residual map; is the eigenvector corresponding to each vertex on the graph; is a trainable weight matrix; is a trainable weight vector; ； In the formula, is a learnable Laplacian matrix with residual; About of k polynomial of order; k Iterate over the sequence numbers for the sum function; K is the total number of nodes in the graph; for Parameters of the polynomial in ; ； ； ； In the formula, L is the normalized graph Laplacian matrix; is a given constant value; is the identity matrix; is the distance metric matrix using generalized Mahalanobis distance; A is the adjacency matrix; , They are the randomly initialized distance metric matrix and adjacency matrix respectively; is the residual Laplacian matrix obtained from training when the distance metric and adjacency relations are randomly initialized.

5. The method for collaborative optimization of source-grid-load-storage collaborative partitioning according to claim 4 is characterized in that: The calculation expression of the distance metric matrix using generalized Mahalanobis distance is: ； In the formula, is the vertex on the graph corresponding to The eigenvector of , are trainable weights.

6. The method for collaborative optimization of source-grid-load-storage collaborative partitioning according to claim 1 is characterized in that: In the multi-agent based Markov decision process model, The control devices with independent action space in the distribution network are defined as intelligent agents; The state observation space is defined as: ; In the formula, is a set of active and reactive power loads; It is a group of active powers generated by power generation equipment; It is a group of reactive powers generated by power generation equipment; is a set of voltage amplitudes and the voltage phase in radians ; They are the active power, reactive power and phase angle of the load respectively; They are respectively the active power and reactive power generated by the power generation equipment; The action space of the agent is defined as: For a continuously controlled device, the agent is equipped with a continuous set of actions, represented as: ； In the formula, To regulate the output of the device; They represent the lower and upper output limits of the continuous control equipment respectively; For discrete control devices, the agent is equipped with a discrete set of actions, expressed as: ； In the formula, To regulate the output of the device; Represents all actions that the device can perform; Reward Function for: ； In the formula, This is the network loss penalty item; is the voltage deviation penalty term; ； In the formula, is a positive weight coefficient; Indicates network loss; ； In the formula, is the number of nodes in the distribution network; is a positive weight coefficient; Representation Node The voltage amplitude; Representation Node The reference voltage amplitude.

7. The method for collaborative optimization of source-grid-load-storage collaborative partitioning according to claim 1 is characterized in that: The control devices in the distribution network include switches, inverters and energy storage devices.

8. A system for collaborative optimization of source, grid, load and storage, characterized in that: include: A data acquisition module, used for acquiring historical data of power grid load based on the selected distribution network; A prediction module is used to make predictions based on the acquired historical data of power grid loads using a trained reinforcement learning model to obtain prediction results of the control strategy of the control device in the selected distribution network; Among them, the training steps of the reinforcement learning model include: based on the distribution network sample, obtaining the distribution network operation data and topology information, and constructing a mathematical model for the voltage and reactive power optimization of the distribution network; converting the constructed mathematical model into a Markov decision process model based on a multi-agent; constructing a topology change graph convolutional network model according to the distribution network operation data and topology information; based on the multi-agent Markov decision process model, using a multi-agent reinforcement learning algorithm to train the topology change graph convolutional network model to obtain a reinforcement learning model that can dynamically adapt to the topology changes of the distribution network and optimize the voltage and reactive power control; using the power grid data actually obtained based on the distribution network sample to perform offline training and verification on the obtained reinforcement learning model to obtain a trained reinforcement learning model.

9. The system for collaborative optimization of source-grid-load-storage coordination and partitioning according to claim 8 is characterized in that: The topology change graph convolutional network model includes: Input layer, used to input distribution network operation data and topology information; The learnable spectral graph convolution module with residual graphs is repeatedly stacked for multiple times, and is used to extract features of input distribution network operation data and topology information to obtain high-dimensional features of the distribution network operation data and topology information; wherein each learnable spectral graph convolution module with residual graph includes a learnable spectral graph convolution layer with residual graph, a first normalization layer and a pooling layer; The second normalization layer is used to normalize the obtained high-dimensional features to obtain distribution network optimization prediction information; The output layer is used to output the obtained distribution network optimization prediction information.

10. The system for collaborative optimization of source, grid, load and storage according to claim 9, characterized in that: The topology change graph convolutional network model also includes: A bilateral filter layer is provided between a learnable spectral convolution module with a residual graph that is repeatedly stacked multiple times and a second normalization layer, and is used for filtering the high-dimensional features of the obtained distribution network operation data and topology information, obtaining the filtered high-dimensional features and inputting them into the second normalization layer.

11. The system for collaborative optimization of source, grid, load and storage according to claim 9, characterized in that: The learnable spectral convolutional layer with residual map is expressed as: ； In the formula, is the output of the learnable spectral convolutional layer with residual map; is the activation function; is a learnable spectral convolution kernel with a residual map, The result of the operation is a learnable spectral convolution matrix with a residual map; is the eigenvector corresponding to each vertex on the graph; is a trainable weight matrix; is a trainable weight vector; ； In the formula, is a learnable Laplacian matrix with residual; About of k polynomial of order; k Iterate over the sequence numbers for the sum function; K is the total number of nodes in the graph; for Parameters of the polynomial in ; ； ； ； In the formula, L is the normalized graph Laplacian matrix; is a given constant value; is the identity matrix; is the distance metric matrix using generalized Mahalanobis distance; A is the adjacency matrix; , They are the randomly initialized distance metric matrix and adjacency matrix respectively; is the residual Laplacian matrix obtained from training when the distance metric and adjacency relations are randomly initialized.

12. The system for collaborative optimization of source, grid, load and storage according to claim 11, characterized in that: The calculation expression of the distance metric matrix using generalized Mahalanobis distance is: ； In the formula, is the vertex on the graph corresponding to The eigenvector of , are trainable weights.

13. The system for collaborative optimization of source, grid, load and storage according to claim 8, characterized in that: In the multi-agent based Markov decision process model, The control devices with independent action space in the distribution network are defined as intelligent agents; The state observation space is defined as: ; In the formula, is a set of active and reactive power loads; It is a group of active powers generated by power generation equipment; It is a group of reactive powers generated by power generation equipment; is a set of voltage amplitudes and the voltage phase in radians ; They are the active power, reactive power and phase angle of the load respectively; They are respectively the active power and reactive power generated by the power generation equipment; The action space of the agent is defined as: For a continuously controlled device, the agent is equipped with a continuous set of actions, represented as: ； In the formula, To regulate the output of the device; They represent the lower and upper output limits of the continuous control equipment respectively; For discrete control devices, the agent is equipped with a discrete set of actions, expressed as: ； In the formula, To regulate the output of the device; Represents all actions that the device can perform; Reward Function for: ； In the formula, This is the network loss penalty item; is the voltage deviation penalty term; ； In the formula, is a positive weight coefficient; Indicates network loss; ； In the formula, is the number of nodes in the distribution network; is a positive weight coefficient; Representation Node The voltage amplitude of Representation Node The reference voltage amplitude.

14. The system for collaborative optimization of source, grid, load and storage according to claim 8, characterized in that: The control devices in the distribution network include switches, inverters and energy storage devices.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the method for collaborative optimization of source-grid-load-storage collaborative partitioning according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for collaborative optimization of source-grid-load-storage collaborative partitioning according to any one of claims 1 to 7 is implemented.

Images