Distributed photovoltaic absorption capability improving method, system and device and medium

Abstract

The invention belongs to the technical field of electrical engineering control, and discloses a distributed photovoltaic consumption capability improving method, system and device and a medium, and the method comprises the steps: obtaining the annual load power supply and distributed photovoltaic output data of each optical storage transformer area, calculating the distributed photovoltaic consumption capability and the system active loss, and calculating the power consumption of each optical storage transformer area; determining an optimization target and a scheduling optimization object of the distributed optical storage transformer area, and establishing a target function and a constraint condition of multi-target optimization of the distributed optical storage transformer area; the operation state of the distributed optical storage transformer area is monitored in real time, an action set executed by equipment participating in scheduling control in the distributed optical storage transformer area is obtained and fed back in time, and a multi-target Markov decision model is constructed based on a multi-target deep reinforcement learning algorithm. According to the method, self-discipline balance of power and electric quantity of the power distribution area is realized, challenges brought by high-permeability distributed photovoltaic and energy storage access to the power distribution area can be effectively handled, and balance among economy, stability and sustainability of the power distribution area is realized.

Description

Technical Field

[0001] The present invention relates to the field of electrical engineering control technology, and in particular to a method, system, equipment and medium for improving distributed photovoltaic absorption capacity. Background Art

[0002] Distributed photovoltaics is an important means to achieve carbon peak and carbon neutrality in the energy and power industry. In recent years, the development of distributed energy and microgrids has attracted social attention. Among them, the application of photovoltaic systems and energy storage has become an important means to improve power supply reliability, promote the consumption of new energy and reduce environmental pollution. In the future, in the process of transitioning to a high proportion of renewable energy and large-scale development of photovoltaic power generation, the coordinated development of photovoltaics and energy storage will become a major model for the development of energy storage. Driven by the strategic development of new energy, distributed photovoltaics and energy storage have continued to develop rapidly, gradually becoming an important part of the energy Internet, and occupying an important position in energy production and consumption.

[0003] Due to the continuous access of high-proportion distributed photovoltaic and energy storage on the distribution side, the power generation and consumption in the substation area will be affected by the temporal and spatial changes of sunshine and power demand, which will lead to voltage fluctuations in the substation area grid, limit photovoltaic absorption, and cause power and electricity imbalance in the substation area. Therefore, in order to optimize the operation of distributed photovoltaic power generation and energy storage systems, solve the problem of power and electricity volatility faced by existing high-proportion distributed photovoltaic storage substations, and improve the economic benefits of the power grid, it is necessary to study the distributed photovoltaic absorption capacity improvement technology for the self-discipline balance of power and electricity in the substation area, ensure the stable operation of the power system, reduce the risk of overvoltage, improve the power quality, and enhance the user experience.

[0004] Therefore, how to provide a method, system, equipment and medium for improving the absorption capacity of distributed photovoltaic power generation is an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present invention provide a method, system, device and medium for improving the absorption capacity of distributed photovoltaic power to solve the above-mentioned technical problems existing in the prior art.

[0006] In order to have a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended to be a general review, nor is it intended to identify key / important components or to delineate the scope of protection of these embodiments. Its only purpose is to present some concepts in a simple form as a preface to the detailed description that follows.

[0007] According to a first aspect of an embodiment of the present invention, a method for improving a distributed photovoltaic power consumption capacity is provided.

[0008] In one embodiment, the method for improving the absorption capacity of distributed photovoltaic power generation includes:

[0009] Obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculate the distributed photovoltaic absorption capacity and system active power loss, determine the optimization objectives and scheduling optimization objects of the distributed photovoltaic storage area, and establish the objective function and constraint conditions of the multi-objective optimization of the distributed photovoltaic storage area;

[0010] Real-time monitoring of the operating status of the distributed optical storage area, obtaining the action set and timely feedback of the equipment involved in the dispatching control in the distributed optical storage area, and building a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm;

[0011] By introducing different preference weights to stimulate the multi-objective deep reinforcement learning algorithm, in each scheduling cycle, the multi-objective Markov decision model is solved according to the input substation monitoring data and preference weights to generate a Pareto optimal solution set that meets the multi-objective optimization requirements.

[0012] In one embodiment, the acquisition of annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculation of distributed photovoltaic absorption capacity and system active power loss, determination of optimization objectives and scheduling optimization objects of distributed photovoltaic storage areas, and establishment of objective functions and constraints for multi-objective optimization of distributed photovoltaic storage areas include:

[0013] Obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, set the ratio of the sum of the photovoltaic capacity of the connected distribution area to the load power consumption of the area as the distributed photovoltaic absorption capacity, and obtain the active power, reactive power and node line circuit of each branch in the distribution network, and calculate the active power loss of the system;

[0014] Maximizing the photovoltaic absorption capacity and minimizing the system active power loss are set as the optimization goals of the distributed photovoltaic storage area. The objective functions of maximizing the photovoltaic absorption capacity and minimizing the system active power loss are established respectively, and the dispatch optimization objects and equipment access area capacity of the distributed photovoltaic storage area participating in multiple optimization goals are determined;

[0015] Based on the node set, line load and equipment operating status within the distributed photovoltaic storage area, constraints that meet multiple optimization objectives are established, and the constraints include power balance constraints, distributed photovoltaic access capacity constraints, system safety constraints and equipment operation constraints.

[0016] In one embodiment, the objective function for maximizing photovoltaic absorption capacity is expressed as:

[0017]

[0018] Where f1 is the objective function for maximizing the photovoltaic absorption capacity; N PV is the total number of nodes connected to photovoltaic; N t is the scheduling period; P PV,i,tis the active power output of the photovoltaic power station at the ith node in time period t;

[0019] The expression of the objective function of minimizing the system active power loss is:

[0020]

[0021] Where f2 is the objective function for minimizing the system active power loss; {N B} represents the set of all nodes in the distribution network; P ij , Q ij are the active power and reactive power flowing through branch i and branch j respectively; U i represents the voltage at node i; R ij Represents the resistance between the lines at nodes i and j.

[0022] In one embodiment, the real-time monitoring of the operating status of the distributed optical storage area, obtaining the action set and timely feedback of the devices participating in the scheduling control in the distributed optical storage area, and constructing a multi-objective Markov decision model based on a multi-objective deep reinforcement learning algorithm include:

[0023] Obtain the real-time active load demand, reactive load demand, photovoltaic active output and charge state of the energy storage system of the nodes in the distributed photovoltaic storage area, establish a state space, and collect the execution action sets of the capacitor bank, static VAR compensator, photovoltaic inverter and energy storage system to establish an action space;

[0024] Based on the timely feedback obtained by the deep reinforcement learning agent after performing an action, a reward function for multi-optimization objective scheduling is established, and the state space and action space are integrated to construct a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm.

[0025] In one embodiment, the reward function is expressed as:

[0026]

[0027] In the formula, R t is the real-time reward in period t; f 1,t is the photovoltaic absorption capacity in time period t; f 2,t is the operating cost of period t; is a vector of Boolean variables; r is a penalty vector.

[0028] In one embodiment, the multi-objective deep reinforcement learning algorithm is stimulated by introducing different preference weights. In each scheduling cycle, the multi-objective Markov decision model is solved according to the input area monitoring data and preference weights to generate a Pareto optimal solution set that meets the multi-objective optimization requirements, including:

[0029] The distributed solar storage area is set to participate in the scheduling cycle of multi-objective optimization, and the area monitoring data collected regularly is input into the multi-objective Markov decision model, and the multi-objective Markov decision model formulates the control strategy of the scheduling optimization object according to the state space at the current moment;

[0030] The multi-objective Markov decision model is used to interact with the substation environment and perform power flow calculation to obtain the reward function at the current moment, introduce weight preference, take maximizing the model cumulative reward as the solution goal, and convert the maximized model cumulative reward into the state action value in the expected form;

[0031] A multi-objective deep Q-network model is used to process discrete variables, and a multi-objective soft actor-critic model is used to process continuous variables. By solving the multi-objective Markov decision model, the Pareto optimal solution set that meets the multi-objective optimization requirements is obtained.

[0032] In one embodiment, the expression of the state action value is:

[0033]

[0034] In the formula, Q π (s t ,a t ) is the state action value; is the mathematical expectation; S t is the state space; A t is the action space; s t is the state vector; a t For state s t The action vector taken by the deep reinforcement learning agent; γ is the decay factor; f ω (R t ) is a scalar-valued function.

[0035] According to a second aspect of an embodiment of the present invention, a distributed photovoltaic absorption capacity enhancement system is provided.

[0036] In one embodiment, the distributed photovoltaic absorption capacity enhancement system includes:

[0037] The parameter setting module is used to obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculate the distributed photovoltaic absorption capacity and system active power loss, determine the optimization objectives and scheduling optimization objects of the distributed photovoltaic storage area, and establish the objective function and constraint conditions of the multi-objective optimization of the distributed photovoltaic storage area;

[0038] The model building module is used to monitor the operating status of the distributed optical storage area in real time, obtain the action set and timely feedback of the equipment involved in the dispatching control in the distributed optical storage area, and build a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm;

[0039] The solution optimization module is used to stimulate the multi-objective deep reinforcement learning algorithm by introducing different preference weights. In each scheduling cycle, according to the input substation monitoring data and preference weights, the multi-objective Markov decision model is solved to generate a Pareto optimal solution set that meets the multi-objective optimization requirements.

[0040] In one embodiment, the parameter setting module includes: a data acquisition module, a target setting module and a constraint condition module, wherein:

[0041] The data acquisition module is used to obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, set the ratio of the sum of the photovoltaic capacity of the connected distribution area to the load power consumption of the area as the distributed photovoltaic absorption capacity, and obtain the active power, reactive power and node line circuit of each branch in the distribution network, and calculate the active power loss of the system;

[0042] The target setting module is used to set maximizing photovoltaic absorption capacity and minimizing system active power loss as the optimization goals of the distributed photovoltaic storage area, respectively establish the objective functions of maximizing photovoltaic absorption capacity and minimizing system active power loss, and determine the scheduling optimization objects and equipment access area capacity of the distributed photovoltaic storage area participating in multiple optimization goals;

[0043] The constraint condition module is used to establish constraint conditions that meet multiple optimization objectives based on the node set, line load and equipment operating status within the distributed photovoltaic storage area, and the constraint conditions include power balance constraints, distributed photovoltaic access capacity constraints, system safety constraints and equipment operation constraints.

[0044] In one embodiment, the objective function for maximizing photovoltaic absorption capacity is expressed as:

[0045]

[0046] Where f1 is the objective function for maximizing the photovoltaic absorption capacity; N PV is the total number of nodes connected to photovoltaic; N t is the scheduling period; P PV,i,t is the active power output of the photovoltaic power station at the ith node in time period t;

[0047] The expression of the objective function of minimizing the system active power loss is:

[0048]

[0049] Where f2 is the objective function for minimizing the system active power loss; {N B} represents the set of all nodes in the distribution network; P ij , Q ij are the active power and reactive power flowing through branch i and branch j respectively; U i represents the voltage at node i; R ij Represents the resistance between the lines at nodes i and j.

[0050] In one embodiment, the model building module includes: a space building module and a model generation module, wherein:

[0051] The space establishment module is used to obtain the real-time active load demand, reactive load demand, photovoltaic active output and charge state of the energy storage system of the nodes in the distributed photovoltaic storage area, establish the state space, and gather the execution action set of the capacitor bank, static VAR compensator, photovoltaic inverter and energy storage system to establish the action space;

[0052] The model generation module is used to establish a reward function for multi-optimization objective scheduling based on the timely feedback obtained by the deep reinforcement learning agent after performing an action, integrate the state space and action space, and construct a multi-objective Markov decision model based on a multi-objective deep reinforcement learning algorithm.

[0053] In one embodiment, the reward function is expressed as:

[0054]

[0055] In the formula, R t is the real-time reward in period t; f 1,t is the photovoltaic absorption capacity in time period t; f 2,t is the operating cost of period t; is a vector of Boolean variables; r is a penalty vector.

[0056] In one embodiment, the solution optimization module includes: a strategy generation module, a reward conversion module and a model solution module, wherein:

[0057] The strategy generation module is used to set the scheduling period of the distributed photovoltaic storage area participating in the multi-objective optimization, input the regularly collected area monitoring data into the multi-objective Markov decision model, and the multi-objective Markov decision model formulates the control strategy of the scheduling optimization object according to the state space at the current moment;

[0058] The reward conversion module is used to use the multi-objective Markov decision model to interact with the substation environment and perform power flow calculations to obtain a reward function at the current moment, introduce weight preferences, maximize the model cumulative reward as the solution goal, and convert the maximized model cumulative reward into a state action value in an expected form;

[0059] The model solving module is used to process discrete variables using a multi-objective deep Q network model and process continuous variables using a multi-objective soft actor critic model, and obtain a Pareto optimal solution set that meets multi-objective optimization requirements by solving a multi-objective Markov decision model.

[0060] In one embodiment, the expression of the state action value is:

[0061]

[0062] In the formula, Q π (s t ,a t ) is the state action value; is the mathematical expectation; S t is the state space; A t is the action space; s t is the state vector; a t For state s t The action vector taken by the deep reinforcement learning agent; γ is the decay factor; f ω (R t ) is a scalar-valued function.

[0063] According to a third aspect of an embodiment of the present invention, a computer device is provided.

[0064] In some embodiments, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0065] According to a fourth aspect of embodiments of the present invention, a computer-readable storage medium is provided.

[0066] In one embodiment, the computer-readable storage medium stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0067] The technical solution provided by the embodiment of the present invention may include the following beneficial effects: through the IMODRL algorithm under different preference weight incentives, in each scheduling cycle, according to the input real-time load, photovoltaic power generation data, energy storage charging and discharging conditions and weight preferences, a series of Pareto frontier optimal solution sets that meet the needs of decision makers are generated. The present invention adopts two different algorithms to solve the MDP, using the MODQN algorithm to process discrete variables and the MOSAC algorithm to process continuous variables, so as to achieve accurate optimization scheduling of different types of equipment in the photovoltaic storage system. The present invention reveals the competition and conflict relationship between photovoltaic absorption capacity and operation economy, and shows that multi-objective optimization scheduling through IMODRL not only ensures the safety and economy of the system, but also can dynamically adjust the energy storage system under different operating conditions, realize the self-discipline balance of power and electricity in the substation area, and can effectively cope with the challenges brought by the access of high-penetration distributed photovoltaic and energy storage to the distribution substation area, and achieve the trade-off between the economy, stability and sustainability of the distribution substation area, and further promote the efficient operation of large-scale distributed photovoltaic storage distribution substation areas. Provide a powerful reference.

[0068] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0070] Figure 1 is a flow chart of a method for improving the absorption capacity of distributed photovoltaic power generation according to an exemplary embodiment;

[0071] Figure 2 is a structural schematic diagram of a distributed photovoltaic absorption capacity improvement system according to an exemplary embodiment;

[0072] Figure 3 is a schematic diagram of the structure of a computer device according to an exemplary embodiment;

[0073] Figure 4 is a framework diagram of an improved multi-objective deep reinforcement learning method according to an exemplary embodiment;

[0074] Figure 5 is a DNQ algorithm neural network structure shown according to an exemplary embodiment;

[0075] Figure 6 is a SAC algorithm neural network structure shown according to an exemplary embodiment;

[0076] Figure 7The present invention is a graph showing an improved multi-objective deep reinforcement learning algorithm optimization performance according to an exemplary embodiment. DETAILED DESCRIPTION

[0077] The following description and accompanying drawings fully illustrate the specific embodiments of this article so that those skilled in the art can practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of this article includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such structure, device or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements. Each embodiment is described in a progressive manner herein, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0078] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. in this document indicate the orientation or position relationship based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing this document and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of this document, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal communication of two elements, it can be a direct connection, or it can be an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0079] As used herein, the term "plurality" means two or more than two, unless otherwise specified.

[0080] In this document, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0081] In this article, the term "and / or" is a description of the association relationship between objects, indicating that three relationships may exist. For example, A and / or B means: A or B, or, A and B.

[0082] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0083] Each module in the device or system of the present application can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above modules.

[0084] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0085] Figure 1 An embodiment of a method for improving the absorption capacity of distributed photovoltaic power generation according to the present invention is shown.

[0086] In this optional embodiment, the method for improving the absorption capacity of distributed photovoltaic power generation includes:

[0087] Step S101, obtaining the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculating the distributed photovoltaic absorption capacity and system active power loss, determining the optimization objectives and scheduling optimization objects of the distributed photovoltaic storage area, and establishing the objective function and constraint conditions of the multi-objective optimization of the distributed photovoltaic storage area;

[0088] Step S103: monitor the operating status of the distributed optical storage area in real time, obtain the action set and timely feedback of the equipment participating in the dispatching control in the distributed optical storage area, and build a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm;

[0089] Step S105, by introducing different preference weights to stimulate the multi-objective deep reinforcement learning algorithm, in each scheduling cycle, according to the input substation monitoring data and preference weights, solve the multi-objective Markov decision model to generate a Pareto optimal solution set that meets the multi-objective optimization requirements.

[0090] In this optional embodiment, when obtaining the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculating the distributed photovoltaic absorption capacity and system active power loss, determining the optimization goals and scheduling optimization objects of the distributed photovoltaic storage area, and establishing the objective function and constraint conditions of the multi-objective optimization of the distributed photovoltaic storage area, the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area can be obtained, the ratio of the sum of the photovoltaic capacity of the connected distribution area to the load power consumption of the area is set as the distributed photovoltaic absorption capacity, and the active power, reactive power and node line circuit of each branch in the distribution network are obtained to calculate the system active power loss; maximizing the photovoltaic absorption capacity and minimizing the system active power loss are set as the optimization goals of the distributed photovoltaic storage area, and the objective functions of maximizing the photovoltaic absorption capacity and minimizing the system active power loss are established respectively, and the scheduling optimization objects and equipment access area capacity of the distributed photovoltaic storage area participating in multiple optimization goals are determined; based on the node set, line load and equipment operation status in the distributed photovoltaic storage area, constraints that meet multiple optimization goals are established, and the constraints include power balance constraints, distributed photovoltaic access capacity constraints, system safety constraints and equipment operation constraints.

[0091] In this optional embodiment, the expression of the objective function for maximizing the photovoltaic absorption capacity is:

[0092]

[0093] Where f1 is the objective function for maximizing the photovoltaic absorption capacity; N PV is the total number of nodes connected to photovoltaic; N t is the scheduling period; P PV,i,t is the active power output of the photovoltaic power station at the ith node in time period t;

[0094] The expression of the objective function of minimizing the system active power loss is:

[0095]

[0096] Where f2 is the objective function for minimizing the system active power loss; {N B} represents the set of all nodes in the distribution network; P ij , Q ij are the active power and reactive power flowing through branch i and branch j respectively; U i represents the voltage at node i; R ij Represents the resistance between the lines at nodes i and j.

[0097] In this optional embodiment, when the operating status of the distributed photovoltaic storage area is monitored in real time, the action set and timely feedback executed by the equipment participating in the scheduling control in the distributed photovoltaic storage area are obtained, and a multi-objective Markov decision model is constructed based on a multi-objective deep reinforcement learning algorithm, the active load demand, reactive load demand, photovoltaic active output and charge state of the energy storage system of the real-time load of the nodes in the distributed photovoltaic storage area can be obtained to establish a state space, and the execution action set of the capacitor bank, static VAR compensator, photovoltaic inverter and energy storage system is combined to establish an action space; based on the timely feedback obtained by the deep reinforcement learning agent after executing the action, a reward function for multi-optimization target scheduling is established, and the state space and action space are integrated to construct a multi-objective Markov decision model based on a multi-objective deep reinforcement learning algorithm.

[0098] In this optional embodiment, the reward function is expressed as:

[0099]

[0100] In the formula, R t is the real-time reward in period t; f 1,t is the photovoltaic absorption capacity in time period t; f 2,t is the operating cost of period t; is a vector of Boolean variables; r is a penalty vector.

[0101] In this optional embodiment, when the multi-objective deep reinforcement learning algorithm is stimulated by introducing different preference weights, the multi-objective Markov decision model is solved in each scheduling cycle according to the input substation monitoring data and preference weights to generate a Pareto optimal solution set that meets the multi-objective optimization requirements. The scheduling cycle of the distributed photovoltaic storage substation participating in the multi-objective optimization can be set, and the substation monitoring data collected regularly is input into the multi-objective Markov decision model, and the multi-objective Markov decision model formulates a control strategy for the scheduling optimization object according to the state space at the current moment; the multi-objective Markov decision model is used to interact with the substation environment and perform power flow calculations to obtain the reward function at the current moment, introduce weight preferences, and maximize the model cumulative reward as the solution goal, and convert the maximized model cumulative reward into an expected form of state action value; a multi-objective deep Q network model is used to process discrete variables, and a multi-objective soft actor critic model is used to process continuous variables, and a Pareto optimal solution set that meets the multi-objective optimization requirements is obtained by solving the multi-objective Markov decision model.

[0102] In this optional embodiment, the expression of the state action value is:

[0103]

[0104] In the formula, Qπ (s t ,a t ) is the state action value; is the mathematical expectation; S t is the state space; A t is the action space; s t is the state vector; a t For state s t The action vector taken by the deep reinforcement learning agent; γ is the decay factor; f ω (R t ) is a scalar-valued function.

[0105] Figure 2 An embodiment of a distributed photovoltaic absorption capacity enhancement system of the present invention is shown.

[0106] In this optional embodiment, the distributed photovoltaic absorption capacity improvement system includes:

[0107] Parameter setting module 201, used to obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculate the distributed photovoltaic absorption capacity and system active power loss, determine the optimization objectives and scheduling optimization objects of the distributed photovoltaic storage area, and establish the objective function and constraint conditions of the multi-objective optimization of the distributed photovoltaic storage area;

[0108] Model building module 202, used to monitor the operating status of the distributed optical storage area in real time, obtain the action set and timely feedback of the equipment participating in the dispatching control in the distributed optical storage area, and build a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm;

[0109] The solution optimization module 203 is used to stimulate the multi-objective deep reinforcement learning algorithm by introducing different preference weights. In each scheduling cycle, according to the input substation monitoring data and preference weights, the multi-objective Markov decision model is solved to generate a Pareto optimal solution set that meets the multi-objective optimization requirements.

[0110] In this optional embodiment, the parameter setting module 201 includes: a data acquisition module (not shown in the figure), a target setting module (not shown in the figure) and a constraint condition module (not shown in the figure), wherein the data acquisition module is used to obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, set the ratio of the sum of the photovoltaic capacity of the access distribution area to the load power consumption of the area as the distributed photovoltaic absorption capacity, and obtain the active power, reactive power and node line circuit of each branch in the distribution network, and calculate the system active loss; the target setting module is used to set the maximization of photovoltaic absorption capacity and the minimization of system active loss as the optimization goals of the distributed photovoltaic storage area, establish the objective functions of maximizing photovoltaic absorption capacity and minimizing system active loss respectively, and determine the scheduling optimization objects and equipment access area capacity of the distributed photovoltaic storage area participating in multiple optimization goals; the constraint condition module is used to establish constraints that meet multiple optimization goals based on the node set, line load and equipment operation status in the distributed photovoltaic storage area, and the constraints include power balance constraints, distributed photovoltaic access capacity constraints, system safety constraints and equipment operation constraints.

[0111] In this optional embodiment, the model building module 203 includes: a space building module (not shown in the figure) and a model generation module (not shown in the figure), wherein the space building module is used to obtain the real-time active load demand, reactive load demand, photovoltaic active output and charge state of the energy storage system of the nodes in the distributed photovoltaic storage area, establish a state space, and collect the execution action sets of the capacitor bank, static VAR compensator, photovoltaic inverter and energy storage system to establish an action space; the model generation module is used to establish a reward function for multi-optimization target scheduling based on the timely feedback obtained by the deep reinforcement learning agent after executing the action, integrate the state space and the action space, and construct a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm.

[0112] In this optional embodiment, the solution optimization module 205 includes: a strategy generation module (not shown in the figure), a reward conversion module (not shown in the figure) and a model solution module (not shown in the figure), wherein the strategy generation module is used to set the scheduling period of the distributed optical storage area participating in the multi-objective optimization, and input the area monitoring data collected regularly into the multi-objective Markov decision model, and the multi-objective Markov decision model formulates the control strategy of the scheduling optimization object according to the state space at the current moment; the reward conversion module is used to use the multi-objective Markov decision model to interact with the area environment and perform power flow calculations to obtain the reward function at the current moment, introduce weight preferences, and maximize the model cumulative reward as the solution goal, and convert the maximized model cumulative reward into the expected form of state action value; the model solution module is used to use the multi-objective deep Q network model to process discrete variables, and use the multi-objective soft actor critic model to process continuous variables, and by solving the multi-objective Markov decision model, obtain the Pareto optimal solution set that meets the multi-objective optimization requirements.

[0113] The following is a detailed description of a method and system for improving the distributed photovoltaic absorption capacity constructed by the present invention in conjunction with specific embodiments and drawings.

[0114] In view of the contradiction between photovoltaic absorption and economic efficiency of high-proportion photovoltaic storage areas, if we only pursue high absorption capacity and ignore the operating economy, it will be difficult to develop sustainably in practical applications. If we only focus on reducing active power loss, insufficient photovoltaic absorption in the distribution network will lead to problems such as voltage over-limit. Therefore, for operators and distribution network power balance and operational safety, it is of great value to achieve a balance between improving photovoltaic absorption capacity and reducing operating costs. This distributed photovoltaic absorption capacity improvement method based on improved multi-objective deep reinforcement learning can balance the active power loss and photovoltaic absorption capacity of distributed photovoltaic storage areas. The present invention uses the IMODRL algorithm under different preference weight incentives to generate a series of Pareto frontier optimal solution sets that meet the needs of decision makers in each scheduling cycle based on the input of real-time load, photovoltaic power generation data, energy storage charging and discharging conditions, and weight preferences. The specific implementation examples of the present invention are as follows:

[0115] 1. Obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area.

[0116] First, in order to illustrate the load absorption capacity of distributed photovoltaic in the high-proportion distributed photovoltaic storage area, the distributed photovoltaic absorption capacity is defined as the ratio of the sum of the photovoltaic capacity of the connected distribution area to the load power consumption of the area, which can be expressed as:

[0117]

[0118] Where: N PVis the total number of nodes connected to photovoltaic in the system; PV,m is the photovoltaic access capacity of the mth node, P load,n is the load power of the nth node.

[0119] Secondly, define the economic performance index of the substation as the system active power loss:

[0120]

[0121] Where: {N B} represents the set of all nodes in the distribution network; P ij , Q ij are the active power and reactive power flowing through branch i and branch j respectively; U i represents the voltage at node i; R ij Represents the resistance between the lines at nodes i and j.

[0122] 2. Determine the objective function of the multi-objective optimization problem to maximize the photovoltaic absorption capacity and minimize the system active power loss. The expression for maximizing the photovoltaic absorption capacity is:

[0123]

[0124] The expression for minimizing the system active power loss is:

[0125]

[0126] 3. Determine the scheduling optimization objects of the multi-objective problem of distributed photovoltaic storage area dispatch as capacitor banks (capacitybanker), static var compensators (static var compensator, SVC), photovoltaic inverters and energy storage systems (energy storage system, ESS), and determine the capacity of each device connected to the area.

[0127] 4. Designing Constraints for Multi-Objective Optimization Problems

[0128] 1) Power balance constraint, the formula is:

[0129]

[0130] Where: U j is the voltage at node j; G ij , B ij are the conductance and susceptance of branch ij respectively; θ ij is the voltage phase angle between node i and node j; P ij , Q ij are the injected active power and reactive power of node i respectively.

[0131] 2) Distributed photovoltaic access capacity constraints, the formula is:

[0132] 0≤P pv,t ≤P pv,max

[0133] Where: P pv,max It is the maximum value of distributed photovoltaic access capacity.

[0134] 3) System safety constraints, node voltage U i and line current I ij It is an important indicator to measure the system operation level and must be guaranteed to be within the safe operation range. The formula is:

[0135]

[0136] Where: U i,max , U i,min are the minimum and maximum voltages allowed at node i, respectively. ij,min is the maximum allowable current limit of line ij.

[0137] 4) Equipment operation constraints. Since the PV inverter has reactive power support capability, if the remaining reactive power capacity of the inverter can be utilized, the voltage over-limit problem of the distribution network can be solved, thereby improving the PV absorption capacity. Therefore, the PV inverter, static VAR compensator, group switching capacitor and energy storage converter are considered as controllable resources. Its operating state needs to meet the following constraints, and the formula is:

[0138]

[0139] |Q SVC,i,t ∣≤Q SVC,i,max

[0140]

[0141]

[0142] Where: P PV,i,t , Q PV,i,t are respectively the active and reactive power generated by the PV at node i in period t; S PV,i is the PV capacity configured at node i; Q SVC,i,t is the maximum reactive power of the SVC configured at node i; Q SVC,i,max is the maximum reactive power of the SVC configured at node i; CB,i,t is the number of CB input groups at node i in time period t; x CB,i,max is the number of installed groups of CB at node i; Q CB,i,t is the reactive power generated by CB at node i in time period t; C iis the capacity of a single group of capacitors at node i; P ESS,i,t is the charging and discharging power of the ESS configured at node i in time period t; ESS,i,max is the maximum charge and discharge power of the ESS configured at node i; SOC i,t , ΔSOC i,t SOC is the charge state and change amount of the energy storage device configured at node i in time period t. i,max , SOC i,min are the upper and lower limits of the state of charge of the ESS configured at node i, S ESS,i is the capacity of the ESS configured at node i.

[0143] According to the above content, the model composed of the above constraints is a typical mixed integer nonlinear multi-objective optimization problem.

[0144] 5. Establishing a multi-objective Markov decision model

[0145] Aiming at the multi-objective optimization problem of improving the distributed photovoltaic absorption capacity in the distributed photovoltaic storage distribution station area, a multi-objective Markov decision process (MOMDP) ​​model based on the improved multi-objective deep reinforcement learning (IMODRL) method is constructed. It consists of six tuples and is expressed as:

[0146] <S,A,R,P,W,γ>

[0147] Where: S represents the system state space, and the system state at time t is S t ∈S; A is the action space; R is the reward function vector, which consists of the scalar reward signals of each target, that is, R = (R1, R2, ..., R N ) T , N is the number of targets; P is the state transition probability function, indicating that the system environment is in the current state S t Under this condition, the DRL agent executes action strategy A t Then enter the next state S t +1 probability distribution; W represents the preference space, which consists of a series of weight vectors ω m where m is the sequence number of the preference vector in the preference space; γ represents the discount factor, which ensures that the later rewards have a smaller impact on the reward function and includes the uncertainty of future rewards.

[0148] MOMDP contains state space, action space and reward function, which are generally in vector form for multi-objective problems. The state space refers to the set of all states that can be observed and perceived. For distribution substations with a high proportion of distributed photovoltaics and energy storage, it mainly includes the load power of the substation, the active and reactive output of photovoltaics, the charge state of the energy storage system, and the status of other related equipment. The action space refers to the set of all actions that can be taken, that is, the set of actions performed by all devices involved in dispatching and control in the substation. The reward function is used to describe the immediate feedback obtained after executing an action, reflecting the contribution of an action to the goal under a specific state.

[0149] 1) State space: The state space S is defined by using real-time load active / reactive demand and PV active output, so that the DRL agent can accurately understand and respond to the current status of the distribution network. It can be expressed as:

[0150] S={P L,t ,Q L,t ,P PV,t ,SOC ESS,t}

[0151] Where: P L,t , Q L,t are the node active and reactive load demand vectors in time period t respectively; P PV,t is the photovoltaic active power output vector in period t; SOC ESS,t is the ESS charge state vector at time period t.

[0152] 2) Action space, which specifies all possible actions that the DRL agent can perform to improve the PV absorption capacity and operation economy of the distribution network. This action space can be defined as a combination of CB, SVC, PV and ESS control strategies.

[0153] A={A CB,t ,A SVC,t ,A PV,t ,A ESS,t}

[0154] Where: A CB,t , A SVC,t , A PV,t , A ESS,t are the control strategy vectors of CB, SVC, PV, and ESS in period t. The corresponding control variables can be expressed as:

[0155] Q CB,i,t =|A CB,i,t |C i

[0156] Q SVC,i,t =|A SVC,i,t |QSVC,i,max ,-1≤|A SVC,t |≤1

[0157]

[0158] P ESS,i,t =|A ESS,i,t |P ESS,i,max ,-1≤|A ESS,i,t |≤1

[0159] Where: A CB,i,t , A SVC,i,t , A PV,i,t , A ESS,i,t are the control strategy vectors of CB, SVC, PV, and ESS configured at node i in time period t. According to the defined action space, it can be ensured that the control variables provided by all controllable devices meet the device operation constraints, which can reduce the infeasible solutions generated by the algorithm during the exploration process, thereby improving the learning efficiency of the algorithm.

[0160] 3) Reward function, the reward function is the core component of MOMDP, which defines the feedback obtained by the DRL agent after taking a specific action. For the proposed dual-objective scheduling problem, it can be expressed as:

[0161]

[0162] Where: f 1,t 、f 2,t are the PV absorption capacity and operating cost in period t; R t is the real-time reward for period t; is a vector of Boolean variables. If the action satisfies the system constraints, otherwise r is a penalty vector.

[0163] 6. Solution of MOMDP model

[0164] In the constructed MOMDP, the model makes a decision based on the current state S of the system in each decision cycle. t Develop control strategies for controllable devicesA t The model then interacts with the environment and performs power flow calculations to determine the reward function R t The system then enters the next time step t+1 and repeats the previous operation until it reaches the predetermined termination time point T. The goal of the model to take action is to maximize the cumulative reward of the model, and its expected form is called the state action value Q π (s t ,a t ), which can be expressed as:

[0165]

[0166] Where: is the mathematical expectation; s t is the state vector; a t For state s t The action vector taken by the DRL agent; γ is the decay factor; f ω (R t )=ω T R t is a key scalar-valued function, according to the preference vector ω m Generates a scalar value reward. m If we regard it as a fixed vector, the MOMDP can be transformed into the original traditional MDP and solved using the standard DRL algorithm. Considering all possible weight preference vectors and corresponding rewards of the MOMDP, we can obtain a set of non-dominated strategies, called the Pareto frontier. When there is no other strategy that can increase the expected reward of a certain goal without reducing the expected reward of any other goal, the resulting strategy π * This is the Pareto optimal strategy.

[0167] In addition, the applications of MODQN and MOSAC are as follows:

[0168] 1) Multi-objective deep Q network (MODQN) algorithm

[0169] like Figure 5 As shown in the figure, the Deep Q Network (DQN) algorithm is an algorithm in deep reinforcement learning. It combines traditional Q learning and deep neural networks to solve reinforcement learning problems with high-dimensional discrete state spaces. The DQN algorithm uses neural networks to approximate the Q value function, which can learn and make decisions in complex environments. The DQN algorithm is an off-policy deep reinforcement learning algorithm that learns by using past experience. The generation of sample data is independent of the current strategy. In order to improve the stability of the algorithm, learning efficiency and accelerate the convergence process, the DQN algorithm uses experience replay techniques to reduce the correlation between training data.

[0170] In order to reduce the dependency during the update process, the DQN algorithm also uses a target network whose parameters are more stable than those in the Q network to estimate the Q value, solving the instability problem of the Q function and improving the training stability of the algorithm. The Q value calculation of the DQN algorithm is based on the Bellman principle, as shown in the following formula:

[0171]

[0172]

[0173] Where: s t and t+1 They represent the states of the agent at time t and t+1 respectively, and a t and a t+1 Respectively represent the actions taken at time t and t+1; r t+1 represents the reward value at time t+1; Q(s t ,a t ) means in state s t Take action a t The obtained Q value, Q(s t+1 ,a t+1 ) means in state s t+1 Take action a t+1 The obtained Q value, Q((s t+1 ,a t+1 )|s t ,a t ) indicates that in the state-action pair (s t ,a t ) is predicted to be in state s t+1 Take action a t+1 Q value; γ is the discount factor, which ranges from [0,1] and is used to attenuate the influence of future rewards; λ is the learning rate, which affects the learning effect and training speed.

[0174] The MODQN algorithm is proposed based on the deep Q network (DQN) algorithm. When dealing with discrete decision variable problems, the agent selects the device action strategy based on probability.

[0175]

[0176] Where: A ep,t is the action decision of the discrete action device, ι is the exploration rate, s t is the state of the agent at time t, a t is the action at time t.

[0177] Neural Network Q k The network parameters are calculated by minimizing the loss function Optimize and update the two neural network Q parameters respectively through the following formula.

[0178]

[0179] Where: m is the preference weight vector, which is used to measure the degree of bias between targets; N d represents the number of dimensions, d represents the dimension of the action, is the target Q network; Represents the loss function The gradient of is the learning rate of the Q network, is the normalized reward function.

[0180] 2) Multi-objective soft actor critic (MOSAC) algorithm

[0181] like Figure 6 As shown in the figure, the SAC algorithm is a reinforcement learning algorithm based on the maximum entropy theory. Its main idea is to maximize the entropy of the strategy, so that the strategy has higher exploratory and robustness. The SAC algorithm uses two value functions to estimate the value of the state-action pair, one of which is the Q function based on the Bellman equation, and the other is the state value function based on the maximum entropy theory. The two value functions work together to improve the estimation accuracy of the value function. Unlike the goal of allowing the agent's strategy to accumulate the maximum reward in reinforcement learning, each output action in the SAC algorithm must comprehensively consider the influence of the reward function and entropy to take into account the maximum reward and the randomness of the strategy. This method can improve the exploration ability and robustness of the agent, and its optimization goal can be expressed as:

[0182]

[0183]

[0184]

[0185] Where: is the entropy value, α is the entropy coefficient, which is used to control the randomness of the optimal strategy. During the training process, its size will be automatically adjusted according to the degree of exploration of the strategy area.

[22] , when the α value is larger, the randomness of the exploration is stronger, on the contrary, when the α value is smaller, the randomness of the exploration is weaker. Its value is calculated by formula (10), Generally equal to -dim(A), A represents the dimension of the action space; r sf (s t ,a t ) is the reward value trained by the soft policy. In order to maximize the reward in the iteration process, the policy iteration process alternates between soft policy evaluation and soft policy improvement. By repeatedly using formula (26), soft policy evaluation can be performed and a higher Q value can be obtained.

[0186]

[0187]

[0188] Where: V function represents the state value function, and Q value represents the agent in the current state st Take action a t And consider the discounted future return value under the optimal action in the future.

[0189] When a set of sample data {S t ,A t ,R t ,S t+1 ,ω m}, the two policy network parameters are updated by the following formula:

[0190]

[0191] Where: J(θ) is the loss function of the evaluation network; y is the predicted Q value; Q′ is the target evaluation network; θ and θ′ are the parameters of the evaluation network and the target evaluation network respectively; τ is the soft update coefficient; is the normalized reward function vector; is the first-order derivative of the loss function J(θ) with respect to θ.

[0192] For the normalized reward function in the MODQN and MOSAC algorithms, since the reward value units of each target dimension are different for MOMDP with multiple targets, a normalization method is used to process the reward vector to make it additive. The specific formula is as follows:

[0193]

[0194] Where: R max,i,t , R max,i,t are the maximum and minimum values ​​of the i-th objective function in time period t, corresponding to ω m =[1,0,...,0] and ω m =[0,...,0,1] the objective function value in these two cases. R i,t are the reward values ​​of the i-th target at time t, which can be solved by the standard DRL method.

[0195] In addition, it is necessary to add that Figure 4 As shown, the IMODRL algorithm flow is as follows:

[0196] Step 1: Initialize the network and playback buffer. Initialize 2 Q networks, 2 policy networks, and 1 preference space. Create an empty experience buffer to store experience tuples. Step 2: Collect and store experience. Use the current policy to perform actions in the environment and collect experience data. t ,A t ,R t ,S t+1 ,ω m}Store in the playback buffer. Step 3: Randomly sample from the buffer. Randomly sample a batch of experience data from the playback buffer. Step 4: Calculate and update the target Q value. For each sampled experience, use the target Q network and the policy network to estimate the target Q value. Update the 2 Q networks using the mean square error loss of the target Q value and the actual Q value. Step 5: Update the policy network. Use the policy gradient method to update the discrete policy network and the continuous policy network. Step 6: Soft update the target network. Use the soft update method to update the weights of the target Q network to keep the target network stable. Step 7: Repeat 2 to 6 until the algorithm converges or the set number of iterations is reached.

[0197] In addition, the improved multi-objective deep reinforcement learning algorithm (IMODRL algorithm) optimizes the performance of the graph as shown in the figure Figure 7 shown.

[0198] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above method embodiment are implemented.

[0199] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0200] In addition, the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiment when executing the computer program.

[0201] In addition, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0202] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0203] The present invention is not limited to the structures which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for improving the absorption capacity of distributed photovoltaic power generation, characterized in that: include: Obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculate the distributed photovoltaic absorption capacity and system active power loss, determine the optimization objectives and scheduling optimization objects of the distributed photovoltaic storage area, and establish the objective function and constraint conditions of the multi-objective optimization of the distributed photovoltaic storage area; Real-time monitoring of the operating status of the distributed optical storage area, obtaining the action set and timely feedback of the equipment involved in the dispatching control in the distributed optical storage area, and building a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm; By introducing different preference weights to stimulate the multi-objective deep reinforcement learning algorithm, in each scheduling cycle, the multi-objective Markov decision model is solved according to the input substation monitoring data and preference weights to generate a Pareto optimal solution set that meets the multi-objective optimization requirements.

2. The method for improving the absorption capacity of distributed photovoltaic power generation according to claim 1, characterized in that: The acquisition of annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculation of distributed photovoltaic absorption capacity and system active power loss, determination of optimization objectives and scheduling optimization objects of distributed photovoltaic storage areas, and establishment of objective functions and constraints for multi-objective optimization of distributed photovoltaic storage areas include: Obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, set the ratio of the sum of the photovoltaic capacity of the connected distribution area to the load power consumption of the area as the distributed photovoltaic absorption capacity, and obtain the active power, reactive power and node line circuit of each branch in the distribution network, and calculate the active power loss of the system; Maximizing the photovoltaic absorption capacity and minimizing the system active power loss are set as the optimization goals of the distributed photovoltaic storage area. The objective functions of maximizing the photovoltaic absorption capacity and minimizing the system active power loss are established respectively, and the dispatch optimization objects and equipment access area capacity of the distributed photovoltaic storage area participating in multiple optimization goals are determined; Based on the node set, line load and equipment operating status within the distributed photovoltaic storage area, constraints that meet multiple optimization objectives are established, and the constraints include power balance constraints, distributed photovoltaic access capacity constraints, system safety constraints and equipment operation constraints.

3. The method for improving the absorption capacity of distributed photovoltaic power generation according to claim 2, characterized in that: The expression of the objective function for maximizing the photovoltaic absorption capacity is: Where f1 is the objective function for maximizing the photovoltaic absorption capacity; N PV is the total number of nodes connected to photovoltaic; N t is the scheduling period; P PV,i,t is the active power output of the photovoltaic power station at the ith node in time period t; The expression of the objective function of minimizing the system active power loss is: Where f2 is the objective function for minimizing the system active power loss; {N B } represents the set of all nodes in the distribution network; P ij , Q ij are the active power and reactive power flowing through branch i and branch j respectively; U i represents the voltage at node i; R ij Represents the resistance between the lines at nodes i and j.

4. The method for improving the absorption capacity of distributed photovoltaic power generation according to claim 1, characterized in that: The real-time monitoring of the operating status of the distributed optical storage area, obtaining the action set and timely feedback of the equipment participating in the dispatching control in the distributed optical storage area, and constructing a multi-objective Markov decision model based on a multi-objective deep reinforcement learning algorithm include: Obtain the real-time active load demand, reactive load demand, photovoltaic active output and charge state of the energy storage system of the nodes in the distributed photovoltaic storage area, establish a state space, and collect the execution action sets of the capacitor bank, static VAR compensator, photovoltaic inverter and energy storage system to establish an action space; Based on the timely feedback obtained by the deep reinforcement learning agent after performing an action, a reward function for multi-optimization objective scheduling is established, and the state space and action space are integrated to construct a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm.

5. The method for improving the absorption capacity of distributed photovoltaic power generation according to claim 4, characterized in that: The expression of the reward function is: In the formula, R t is the real-time reward in period t; f 1,t is the photovoltaic absorption capacity in time period t; f 2,t is the operating cost of period t; is a vector of Boolean variables; r is a penalty vector.

6. The method for improving the absorption capacity of distributed photovoltaic power generation according to claim 1, characterized in that: The multi-objective deep reinforcement learning algorithm is stimulated by introducing different preference weights. In each scheduling cycle, the multi-objective Markov decision model is solved according to the input substation monitoring data and preference weights to generate a Pareto optimal solution set that meets the multi-objective optimization requirements, including: The distributed solar storage area is set to participate in the scheduling cycle of multi-objective optimization, and the area monitoring data collected regularly is input into the multi-objective Markov decision model, and the multi-objective Markov decision model formulates the control strategy of the scheduling optimization object according to the state space at the current moment; The multi-objective Markov decision model is used to interact with the substation environment and perform power flow calculation to obtain the reward function at the current moment, introduce weight preference, take maximizing the model cumulative reward as the solution goal, and convert the maximizing model cumulative reward into the state action value in the expected form; A multi-objective deep Q-network model is used to process discrete variables, and a multi-objective soft actor-critic model is used to process continuous variables. By solving the multi-objective Markov decision model, the Pareto optimal solution set that meets the multi-objective optimization requirements is obtained.

7. The method for improving the absorption capacity of distributed photovoltaic power generation according to claim 6, characterized in that: The expression of the state action value is: In the formula, Q π (s t ,a t ) is the state action value; is the mathematical expectation; S t is the state space; A t is the action space; s t is the state vector; a t For state s t The action vector taken by the deep reinforcement learning agent; γ is the decay factor; f ω (R t ) is a scalar-valued function.

8. A distributed photovoltaic absorption capacity improvement system, characterized in that: include: The parameter setting module is used to obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, calculate the distributed photovoltaic absorption capacity and system active power loss, determine the optimization objectives and scheduling optimization objects of the distributed photovoltaic storage area, and establish the objective function and constraint conditions of the multi-objective optimization of the distributed photovoltaic storage area; The model building module is used to monitor the operating status of the distributed optical storage area in real time, obtain the action set and timely feedback of the equipment involved in the dispatching control in the distributed optical storage area, and build a multi-objective Markov decision model based on the multi-objective deep reinforcement learning algorithm; The solution optimization module is used to stimulate the multi-objective deep reinforcement learning algorithm by introducing different preference weights. In each scheduling cycle, according to the input substation monitoring data and preference weights, the multi-objective Markov decision model is solved to generate a Pareto optimal solution set that meets the multi-objective optimization requirements.

9. The distributed photovoltaic absorption capacity improvement system according to claim 8 is characterized in that: The parameter setting module includes: a data acquisition module, a target setting module and a constraint condition module, wherein: The data acquisition module is used to obtain the annual load power supply and distributed photovoltaic output data of each photovoltaic storage area, set the ratio of the sum of the photovoltaic capacity of the connected distribution area to the load power consumption of the area as the distributed photovoltaic absorption capacity, and obtain the active power, reactive power and node line circuit of each branch in the distribution network, and calculate the active power loss of the system; The target setting module is used to set maximizing photovoltaic absorption capacity and minimizing system active power loss as the optimization goals of the distributed photovoltaic storage area, respectively establish the objective functions of maximizing photovoltaic absorption capacity and minimizing system active power loss, and determine the scheduling optimization objects and equipment access area capacity of the distributed photovoltaic storage area participating in multiple optimization goals; The constraint condition module is used to establish constraint conditions that meet multiple optimization objectives based on the node set, line load and equipment operating status within the distributed photovoltaic storage area, and the constraint conditions include power balance constraints, distributed photovoltaic access capacity constraints, system safety constraints and equipment operation constraints.

10. The distributed photovoltaic absorption capacity improvement system according to claim 9 is characterized in that: The expression of the objective function for maximizing the photovoltaic absorption capacity is: Where f1 is the objective function for maximizing the photovoltaic absorption capacity; N PV is the total number of nodes connected to photovoltaic; N t is the scheduling period; P PV,i,t is the active power output of the photovoltaic power station at the ith node in time period t; The expression of the objective function of minimizing the system active power loss is: Where f2 is the objective function for minimizing the system active power loss; {N B } represents the set of all nodes in the distribution network; P ij , Q ij are the active power and reactive power flowing through branch i and branch j respectively; U i represents the voltage at node i; R ij Represents the resistance between the lines at nodes i and j.

11. The distributed photovoltaic absorption capacity enhancement system according to claim 8, characterized in that: The model building module includes: a space building module and a model generation module, wherein: The space establishment module is used to obtain the real-time active load demand, reactive load demand, photovoltaic active output and charge state of the energy storage system of the nodes in the distributed photovoltaic storage area, establish the state space, and gather the execution action set of the capacitor bank, static VAR compensator, photovoltaic inverter and energy storage system to establish the action space; The model generation module is used to establish a reward function for multi-optimization objective scheduling based on the timely feedback obtained by the deep reinforcement learning agent after performing an action, integrate the state space and action space, and construct a multi-objective Markov decision model based on a multi-objective deep reinforcement learning algorithm.

12. The distributed photovoltaic absorption capacity enhancement system according to claim 11, characterized in that: The expression of the reward function is: In the formula, R t is the real-time reward in period t; f 1,t is the photovoltaic absorption capacity in time period t; f 2,t is the operating cost of period t; is a vector of Boolean variables; r is a penalty vector.

13. The distributed photovoltaic absorption capacity improvement system according to claim 8, characterized in that: The solution optimization module includes: a strategy generation module, a reward conversion module and a model solution module, wherein: The strategy generation module is used to set the scheduling period of the distributed photovoltaic storage area participating in the multi-objective optimization, input the regularly collected area monitoring data into the multi-objective Markov decision model, and the multi-objective Markov decision model formulates the control strategy of the scheduling optimization object according to the state space at the current moment; The reward conversion module is used to use the multi-objective Markov decision model to interact with the substation environment and perform power flow calculations to obtain a reward function at the current moment, introduce weight preferences, maximize the model cumulative reward as the solution goal, and convert the maximized model cumulative reward into a state action value in an expected form; The model solving module is used to process discrete variables using a multi-objective deep Q network model and process continuous variables using a multi-objective soft actor critic model, and obtain a Pareto optimal solution set that meets multi-objective optimization requirements by solving a multi-objective Markov decision model.

14. The distributed photovoltaic absorption capacity enhancement system according to claim 13, characterized in that: The expression of the state action value is: In the formula, Q π (s t ,a t ) is the state action value; is the mathematical expectation; S t is the state space; A t is the action space; s t is the state vector; a t For state s t The action vector taken by the deep reinforcement learning agent; γ is the decay factor; f ω (R t ) is a scalar-valued function.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images