Voltage reactive power control method and device, control equipment and storage medium

Abstract

The invention relates to a voltage reactive power control method and device, control equipment and a storage medium, and the method comprises the steps: obtaining the real-time state data of a power distribution network, the real-time state data comprises the reactive power output, node voltage and load information of a photovoltaic inverter in each sub-region divided in advance; inputting the real-time state data into a pre-trained multi-agent cooperative control model, and outputting a reactive power control instruction through the multi-agent cooperative control model, the reactive power control instruction being used for indicating the reactive power output of the photovoltaic inverter of each sub-region; wherein the multi-agent cooperative control model is obtained by performing set training on a Markov game model constructed by taking a sub-region as an agent, and the sub-region is obtained by dividing the power distribution network through node modularity determined by an electrical distance between nodes of the power distribution network. Therefore, the real-time performance and the control precision of voltage reactive power control can be improved.

Description

Technical Field

[0001] The present application relates to the field of electrical control technology, and in particular to a voltage and reactive power control method, device, control equipment and storage medium. Background Art

[0002] The widespread integration of large-scale renewable distributed power sources into distribution networks effectively promotes the absorption of new energy and reduces carbon emissions. However, this also poses significant challenges to the operation and control of distribution networks. Renewable energy exhibits significant randomness, volatility, and intermittency. Their integration into distribution networks alters the original radial network structure, causing the power flow in the network to shift from unidirectional to bidirectional. This can lead to voltage over-limits, voltage fluctuations, and power loss, reducing power quality and reliability, and placing higher demands on the operation and control of distribution networks.

[0003] In traditional distribution networks, voltage regulation can be achieved by adjusting transformer taps, installing capacitor banks, and setting up static VAR compensators.

[0004] However, during the implementation process, the applicant discovered that the conventional technology has at least the problem of low real-time performance of voltage and reactive power control of the distribution network. Summary of the Invention

[0005] Based on this, the purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defect of low real-time performance of voltage and reactive power control in the existing technology. This application provides a voltage and reactive power control method, device, control equipment and storage medium.

[0006] In a first aspect, the present application provides a voltage reactive power control method, the method comprising:

[0007] Acquire real-time status data of the distribution network, including reactive power output, node voltage, and load information of PV inverters in each pre-divided sub-area;

[0008] The real-time status data is input into a pre-trained multi-agent collaborative control model, which then outputs reactive power control instructions. The reactive power control instructions are used to indicate the reactive power output of the photovoltaic inverters in each sub-area.

[0009] Among them, the multi-agent collaborative control model is obtained by collective training of a Markov game model constructed with sub-regions as agents, and the sub-regions are obtained by dividing the distribution network into sub-regions based on the node modularity determined by the electrical distance between the distribution network nodes.

[0010] In one embodiment, a training method for a multi-agent collaborative control model includes:

[0011] Get the electrical distance between nodes of the distribution network;

[0012] Determine node modularity based on electrical distance;

[0013] Using a preset partitioning algorithm, the node modularity is optimized and the distribution network is divided into multiple sub-areas with close electrical coupling;

[0014] A Markov game model is constructed with each sub-region as an intelligent agent. The state set of the Markov game model includes the historical reactive power output of the PV inverter and the node voltage, and the action set of the Markov game model includes the coordinated reactive power output of the PV inverter.

[0015] The Markov game models of each sub-region are trained centrally to obtain a multi-agent collaborative control model.

[0016] In one embodiment, obtaining the electrical distance between nodes of the power distribution network includes:

[0017] Obtain voltage-reactive electrical distance and voltage-active electrical distance;

[0018] The electrical distance between nodes is determined by the following expression:

[0019]

[0020] in, is the voltage-reactive electrical distance; is the voltage-active electrical distance.

[0021] In one embodiment, the voltage-reactive electrical distance is obtained by the following expression:

[0022]

[0023] The voltage-active electrical distance is obtained by the following expression:

[0024]

[0025] in, (i=1, 2, …, n; j=1, 2, …, n) represents the ratio of the voltage amplitude change at node vj to the voltage amplitude change at node vi when the reactive power of node vj changes, reflecting the voltage-reactive power influence of node vj on node vi. (i=1, 2, …, n; j=1, 2, …, n) represents the ratio of the voltage amplitude change of node vj to the voltage amplitude change of node vi when the active power of node vj changes, reflecting the voltage-active power influence of node vj on node vi.

[0026] In one embodiment, determining the node modularity based on the electrical distance includes:

[0027] Determine the weight matrix based on the electrical distance;

[0028] The node modularity is obtained according to the weight matrix and the preset modularity function.

[0029] In one embodiment, a preset partitioning algorithm is used to optimize node modularity and divide the distribution network into multiple sub-areas with close electrical coupling, including:

[0030] Determine the original community area of ​​each node;

[0031] For each node, assign the node to the community area of ​​the adjacent nodes connected to the node;

[0032] Iteratively optimize the node modularity through the following expression:

[0033]

[0034] in, is the sum of the weights of all edges in the original community area of ​​the node, is the sum of the weights of the edges connecting the nodes in the original community area of ​​the node, is the sum of the weights of the edges connected to the node, is the sum of the weights of the edges connecting the node to the original community area, and m is the sum of the weights of all edges in the network;

[0035] If the node modularity reaches the maximum modularity, the currently allocated multiple community areas are determined as multiple sub-areas with close electrical coupling.

[0036] In one embodiment, the Markov game model of each sub-region is centrally trained to obtain a multi-agent collaborative control model, including:

[0037] Receiving global state information of the Markov game models of the plurality of sub-regions through the evaluators of the Markov game models of the sub-regions;

[0038] For the Markov game model of each sub-region, local information of the Markov game models of multiple other sub-regions is received through the executor of the Markov game model of the sub-region;

[0039] For the actuator that receives local information, the parameters of the actuator are updated using the preset policy gradient;

[0040] For the evaluator that receives the global state information, the parameters of the evaluator are updated using the preset loss function;

[0041] Based on the updated parameters of the actuator and the updated parameters of the evaluator, a multi-agent collaborative control model is obtained.

[0042] In a second aspect, the present application provides a voltage and reactive power control device, the device comprising:

[0043] The status data module is used to obtain real-time status data of the distribution network. The real-time status data includes the reactive output, node voltage and load information of the photovoltaic inverters in each pre-divided sub-area;

[0044] The reactive power control module is used to input real-time status data into a pre-trained multi-agent collaborative control model, and output reactive power control instructions through the multi-agent collaborative control model. The reactive power control instructions are used to indicate the reactive power output of the photovoltaic inverters in each sub-area.

[0045] Among them, the multi-agent collaborative control model is obtained by collective training of a Markov game model constructed with sub-regions as agents, and the sub-regions are obtained by dividing the distribution network into sub-regions based on the node modularity determined by the electrical distance between the distribution network nodes.

[0046] In a third aspect, the present application provides a control device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0047] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0048] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0049] The voltage and reactive power control method, device, control equipment and storage medium provided by the present application divide the distribution network into sub-areas by the node modularity determined by the electrical distance between the distribution network nodes, and collectively train a multi-agent collaborative control model based on the Markov game model constructed with the sub-areas as agents. The multi-agent collaborative control model can be used to process real-time status data, and can effectively and accurately control the reactive output and voltage of the photovoltaic inverters in each sub-area. Compared with traditional technologies, the present application divides the distribution network into regions by the electrical distance between the distribution network nodes, which can significantly reduce the complexity of the voltage and reactive power control of the distribution network, and the Markov game model based on the partition emphasizes the global collaboration between multiple agents, thereby improving the real-time performance and control accuracy of the voltage and reactive power control. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0051] Figure 1 A flow chart of a voltage and reactive power control method provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of the training steps of a multi-agent collaborative control model provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a process for determining a node module according to an embodiment of the present application;

[0054] Figure 4 A schematic diagram of a centralized training-decentralized execution architecture of a MASAC algorithm provided in an embodiment of the present application;

[0055] Figure 5 A distribution network partition voltage-reactive power optimization control method based on deep reinforcement learning is provided in an embodiment of the present application;

[0056] Figure 6 A schematic structural diagram of a voltage and reactive power control device provided in an embodiment of the present application;

[0057] Figure 7 A schematic diagram of the internal structure of a control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0059] The widespread integration of large-scale renewable distributed power sources into distribution networks effectively promotes the absorption of new energy and reduces carbon emissions. However, this also poses significant challenges to the operation and control of distribution networks. Renewable energy exhibits significant randomness, volatility, and intermittency. Their integration into distribution networks alters the original radial network structure, causing the power flow in the network to shift from unidirectional to bidirectional. This can lead to voltage over-limits, voltage fluctuations, and power loss, reducing power quality and reliability, and placing higher demands on the operation and control of distribution networks.

[0060] In traditional distribution networks, voltage regulation can be achieved by adjusting transformer taps, installing capacitor banks, and setting up static VAR compensators.

[0061] However, as the penetration of renewable energy sources like photovoltaics in distribution networks continues to increase, traditional voltage regulation equipment, plagued by slow response, insufficient accuracy, and high installation costs, has become incapable of achieving rapid, flexible, and precise control of distribution networks. Furthermore, traditional centralized control relies on a central controller, which places high demands on communication conditions and data processing capabilities. This makes it unsuitable for the real-time optimization of large-scale distribution networks with high penetrations of renewable energy.

[0062] Based on this, the present application provides a voltage and reactive power control method, apparatus, control device and storage medium, which divides the distribution network into regions according to the electrical distance between the distribution network nodes, and can significantly reduce the complexity of the voltage and reactive power control of the distribution network. The partitioned Markov game model emphasizes the global collaboration among multiple intelligent agents, thereby improving the real-time performance and control accuracy of the voltage and reactive power control.

[0063] In an exemplary embodiment, Figure 1 A flow chart of a voltage reactive power control method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, a voltage and reactive power control method is provided, and the method is applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smart phones, and tablet computers. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. In this embodiment, the method includes the following S101 to S102. Among them:

[0064] S101. Acquire real-time status data of the distribution network, where the real-time status data includes reactive output, node voltage, and load information of photovoltaic inverters in each pre-divided sub-area.

[0065] The real-time status data may refer to the dynamic operation data of the distribution network, and the real-time status data may include reactive output, node voltage and load information of photovoltaic inverters in each pre-divided sub-area.

[0066] Reactive output can refer to the reactive power currently output by the PV inverter (unit: kVar). Node voltage can refer to the real-time voltage amplitude at key monitoring points in the distribution network (unit: pu or kV). Load information can refer to the active power (kW) and reactive power (kVar) demands of users in a sub-region.

[0067] For example, the server system can acquire real-time dynamic operating data from pre-divided sub-areas through monitoring equipment deployed in the distribution network (such as RTUs, smart meters, and PMUs) and supervisory control and data acquisition (SCADA) systems. This data is transmitted via a communications network to a central server for subsequent analysis and decision-making. Typical data flow cycles range from seconds to minutes.

[0068] S102: Input the real-time status data into a pre-trained multi-agent collaborative control model, and output reactive control instructions through the multi-agent collaborative control model. The reactive control instructions are used to indicate the reactive output of the photovoltaic inverters in each sub-area.

[0069] Among them, the multi-agent collaborative control model is obtained by collective training of a Markov game model constructed with sub-regions as agents, and the sub-regions are obtained by dividing the distribution network into sub-regions based on the node modularity determined by the electrical distance between the distribution network nodes.

[0070] A multi-agent collaborative control model can refer to a distributed decision-making system composed of multiple agents, each of which can learn independently but collaboratively optimize a global objective (e.g., grid-wide voltage stability) by sharing environmental states. A reactive power control command can refer to a command for issuing a real-time reactive power setpoint to a photovoltaic inverter. A Markov game model can refer to a mathematical framework for multi-agent reinforcement learning. Electrical distance can refer to a metric that quantifies the strength of electrical coupling between nodes. Node modularity can refer to a metric that measures the quality of subgraph partitioning in a complex network.

[0071] For example, a central server can input real-time status data into a pre-trained multi-agent collaborative control model, which uses a Markov game framework for distributed decision-making. Each agent can correspond to a sub-region and output reactive power regulation commands for the PV inverters in that region. After verification, these reactive power regulation commands can be issued to local controllers for execution, achieving voltage and reactive power control.

[0072] In this embodiment, the distribution network is divided into sub-regions using the node modularity determined by the electrical distance between distribution network nodes. A Markov game model constructed with the sub-regions as agents is used to collectively train a multi-agent collaborative control model. This multi-agent collaborative control model can be used to process real-time status data, enabling effective and accurate control of the reactive power output and voltage of the photovoltaic inverters in each sub-region. Compared with traditional technologies, this embodiment significantly reduces the complexity of voltage and reactive power control in the distribution network by dividing the distribution network into regions based on the electrical distance between distribution network nodes. Furthermore, the partitioned Markov game model emphasizes global collaboration among multiple agents, thereby improving the real-time performance and control accuracy of voltage and reactive power control.

[0073] In an exemplary embodiment, Figure 2 A flowchart of the training steps of a multi-agent collaborative control model provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the training method of the multi-agent collaborative control model may specifically include S201 to S205, wherein:

[0074] S201: Obtain electrical distances between nodes of a distribution network.

[0075] The electrical distance can be an indicator to quantify the electrical coupling strength between nodes, reflecting the degree of mutual influence of node voltage changes.

[0076] Exemplarily, the server system can obtain the sensitivity matrix of active / reactive power and voltage amplitude between nodes by calculating the sensitivity matrix, such as using the Newton-Raphson method for power flow calculation, and can obtain the electrical distance between nodes of the distribution network by decoupling the sensitivity matrix and calculating the voltage-reactive influence and the voltage-active influence.

[0077] S202: Determine the node modularity based on the electrical distance.

[0078] Among them, node modularity can reflect the tightness of the internal connection of the sub-region.

[0079] Exemplarily, the electrical distance may be converted into a connection weight, and the modularity parameter may be calculated by the connection weight. Furthermore, the node modularity may be optimized based on a pre-set modularity function in combination with the modularity parameter.

[0080] S203: Utilize a preset partitioning algorithm to optimize node modularity and divide the distribution network into multiple sub-areas with close electrical coupling.

[0081] The preset partitioning algorithm may be the Louvain algorithm, which may be used to initialize partitioning and iteratively move nodes to achieve a final partitioning result.

[0082] S204. Construct a Markov game model with each sub-region as an intelligent agent; wherein the state set of the Markov game model includes the historical reactive output of the photovoltaic inverter and the node voltage, and the action set of the Markov game model includes the coordinated reactive output of the photovoltaic inverter.

[0083] Among them, the Markov game model can refer to the mathematical framework of multi-agent reinforcement learning.

[0084] S205. Centrally train the Markov game model of each sub-region to obtain a multi-agent collaborative control model.

[0085] Exemplarily, the server can use the Newton-Raphson method to perform power flow calculations, obtain the sensitivity matrix of active / reactive power and voltage amplitude between nodes, and obtain the voltage-reactive influence matrix and the voltage-active influence matrix by decoupling the sensitivity matrix. Further, based on the voltage-reactive influence matrix and the voltage-active influence matrix, the electrical distance between the nodes of the distribution network can be obtained.

[0086] The server can convert the electrical distances between nodes in the distribution network into connection weights between nodes in the distribution network. For example, smaller electrical distances correspond to larger weights, indicating closer node connections. The server can also calculate the node modularity based on a predefined modularity function by inputting the electrical distances between nodes in the distribution network into the function.

[0087] The server can use a partitioning algorithm such as the Louvain algorithm to traverse each node, calculate the modularity gain of moving it to an adjacent sub-region, and move the node to the sub-region where the modularity gain is maximized and positive, until all nodes no longer move, that is, until the modularity of the network no longer changes after iteration.

[0088] Using a pre-set partitioning algorithm, the node modularity is optimized, dividing the distribution network into multiple tightly coupled sub-regions. Furthermore, a Markov game model can be constructed using each sub-region as an intelligent agent. The Markov game model includes a state set, an action set, state transitions, and a reward function. The state set of the Markov game model includes the historical reactive power output of the PV inverter and the node voltage, and the action set includes the coordinated reactive power output of the PV inverter.

[0089] The server can solve the MG (Markov Game) model based on the Multi-Agent Soft Actor-Critic (MASAC) algorithm. For example, a centralized training-decentralized execution architecture can be used to achieve coordinated voltage-reactive power control of photovoltaic inverters, effectively alleviating voltage fluctuations at distribution network nodes and reducing reactive power losses.

[0090] Optionally, during the centralized training phase, based on a network partitioning approach, each agent's critic network receives global state information from all agents, while each agent's corresponding actor network receives local information from its own subregion. The critic network leverages this global information to support the actor network in developing a coordinated control strategy for the multi-PV voltage system. During decentralized execution, the control strategy is determined by the actor network, requiring only the actor network to be retained. Because the actor network has already acquired a collaborative control strategy leveraging global information during training, relying solely on local information during execution can provide a more collaborative and flexible control strategy.

[0091] In this embodiment, based on the multi-agent deep reinforcement learning algorithm and voltage regulation through photovoltaic inverters, it can show better control effects on problems such as voltage imbalance, voltage fluctuation and reactive power loss.

[0092] In an exemplary embodiment, obtaining the electrical distance between nodes of a power distribution network includes:

[0093] Obtain voltage-reactive electrical distance and voltage-active electrical distance;

[0094] The electrical distance between nodes is determined by the following expression (1):

[0095]

[0096] in, is the voltage-reactive electrical distance; is the voltage-active electrical distance.

[0097] For example, The larger the value, the greater the electrical distance, which means the electrical connection between the two nodes is more distant, and the corresponding connecting edge has a smaller weight.

[0098] Alternatively, the connection weights between nodes can be determined based on the electrical distances between the nodes, as shown in the following expression (2):

[0099]

[0100] Furthermore, the node modularity in step S202 may be determined based on the connection weights between the nodes.

[0101] In an exemplary embodiment, the voltage-reactive electrical distance is obtained as follows (3):

[0102] (3);

[0103] The voltage-active electrical distance is obtained from the following expression (4):

[0104] (4);

[0105] in, (i=1, 2, ..., n; j=1, 2, ..., n) means when node v j When the reactive power changes, the node v j The voltage amplitude change and node v i The ratio of the voltage amplitude change reflects the node v j Ability to influence voltage-reactive power of node vi; (i=1, 2, ..., n; j=1, 2, ..., n) represents when node v j When the active power changes, the node v j The voltage amplitude change and node v i The ratio of the voltage amplitude change reflects the voltage-active power influence of node vj on node vi.

[0106] For example, when the Newton-Ray method is used to calculate the power flow of the AC distribution network, the correction equation is as shown in Expression (5):

[0107] (5);

[0108] in: is the active power imbalance injected into the node; is the reactive power unbalance injected into the node; is the change in node voltage phase angle; is the change in voltage amplitude; H is the sensitivity of the node's injected active power to the voltage phase angle; N is the sensitivity of the node's injected active power to the voltage amplitude; J is the sensitivity of the node's injected reactive power to the voltage phase angle; L is the sensitivity of the node's injected reactive power to the voltage amplitude.

[0109] Performing matrix transformation operations on the above formula (5) yields formula (6):

[0110] (6);

[0111] in, is the sensitivity matrix of active power and voltage amplitude between nodes; is the sensitivity matrix of reactive power and voltage amplitude between nodes; is the sensitivity matrix of active power and voltage phase angle between nodes; is the sensitivity matrix of reactive power and voltage phase angle between nodes.

[0112] From the above expression (6), we can see that in a distribution network with n nodes, the node voltage amplitude changes The relationship between the changes in active power and reactive power is as follows (7):

[0113] (7);

[0114] in, ; .

[0115] The relationship between voltage-reactive power and voltage-active power is decoupled and the voltage-reactive power influence matrix is ​​calculated as shown in the following expression (8):

[0116] (8);

[0117] in, is the element in row i and column j in the voltage-reactive sensitivity matrix, representing the node v j Reactive power change and node v i The relationship between the voltage changes; is the representative node v j When the reactive power changes, the node v i The change in its own voltage amplitude; For node v j When the reactive power changes, the node v i Voltage amplitude change; When node v j When the reactive power changes, the node v j The voltage amplitude change and node v i The ratio of the voltage amplitude change reflects the node v j For node v i Voltage-reactive power influence capability.

[0118] Considering the influence of all nodes, define node v i 、v j The voltage-reactive electrical distance between them can be expressed as the above expression (3).

[0119] Similarly, by decoupling the relationship between voltage-reactive power and voltage-active power, the voltage-active power influence matrix can be obtained as follows (9):

[0120] (9)

[0121] in, is the element in row i and column j in the voltage-active power sensitivity matrix, representing the node v j Active power change and node v i The relationship between the voltage changes; is the representative node v j When the active power changes, the node vi The change in its own voltage amplitude; For node v j When the active power changes, the node v i Voltage amplitude change; When node v j When the active power changes, the node v j The voltage amplitude change and node v i The ratio of the voltage amplitude change reflects the node v j For node v i Voltage-active power influence capability.

[0122] Taking into account the influence of active power and reactive power on voltage of distribution network, the electrical distance between nodes can be obtained as shown in the above expression (1).

[0123] In this embodiment, by determining the electrical distance between nodes as described above, a basis for dividing sub-areas can be effectively provided.

[0124] In an exemplary embodiment, Figure 3 A flowchart of a step for determining a node module is provided in an embodiment of the present application, such as Figure 3 As shown, based on the electrical distance, the node modularity is determined, including S301 to S302, wherein:

[0125] S301, determining a weight matrix according to the electrical distance;

[0126] S302: Obtain the node modularity according to the weight matrix and a preset modularity function.

[0127] For example, the connection weights between nodes may be determined according to the electrical distances between the nodes, as shown in the above expression (2).

[0128] The node modularity is expressed as follows (10):

[0129] (10)

[0130] Where m represents the sum of the edge weights of the network (the total number of network edges if there is no weight); is the sum of the weights of the edges connected to point i; Represents the weight of the edge connecting point i and point j; is the Kroklar function, which is 1 when nodes i and j belong to the same community, otherwise it is 0. The larger the node modularity Q, the better the community division result.

[0131] In this embodiment, through the above steps S301 to S302, the node modularity can be effectively and accurately determined by electrical distance, thereby providing a data basis for dividing sub-areas, which is conducive to improving the real-time performance and control accuracy of voltage and reactive power control.

[0132] In an exemplary embodiment, a preset partitioning algorithm is used to optimize node modularity and divide the distribution network into multiple sub-areas with close electrical coupling, including:

[0133] Determine the original community area of ​​each node;

[0134] For each node, assign the node to the community area of ​​the adjacent nodes connected to the node;

[0135] The node modularity is iteratively optimized by the following expression (11):

[0136] (11)

[0137] in, is the sum of the weights of all edges in the original community area of ​​the node, is the sum of the weights of the edges connecting the nodes in the original community area of ​​the node, is the sum of the weights of the edges connected to the node, is the sum of the weights of the edges connecting the node to the original community area, and m is the sum of the weights of all edges in the network;

[0138] If the node modularity reaches the maximum modularity, the currently allocated multiple community areas are determined as multiple sub-areas with close electrical coupling.

[0139] For example, the Louvain algorithm may be used to divide the sub-regions, and each node in the network may be regarded as a community (sub-region).

[0140] Select a node v i , and distribute it to each adjacent node v connected to it in turn j The community to which the module belongs is calculated after the end of the round of allocation. , The node v can be transformed into i Assign to the community with the largest neighboring node, and ensure Greater than zero, if the largest is also less than zero, then node v i Continue to stay in the original community. Repeat this process until all nodes stop moving, that is, the maximum modularity in the current state is achieved.

[0141] Optionally, the community at the end of the above steps is regarded as a supernode to form a new network, where the weight of the connection edge between two supernodes is the sum of the weights of all the connection edges of the two communities. Repeat the above "determine the original community area of ​​each node" step until the modularity of the network no longer changes, the community division is completed, and the number of distribution network areas (communities) is C. p .

[0142] In this embodiment, the above-mentioned iterative optimization steps can accurately divide the sub-areas, thereby facilitating improvement of the control accuracy of voltage and reactive power control.

[0143] In an exemplary embodiment, a Markov game model is constructed, and the basic components are summarized as follows:

[0144] State set S t :S t is the state set of all agents at time t, is the state of agent i at time t, , Consists of five parts Composition, of which is the reactive power output of the photovoltaic inverter in the previous state, is the voltage of the node to which the inverter is connected.

[0145] Action Set A t :A t is the set of actions of all agents at time t, ,The action of the agent is determined by the reactive output of the PV inverter.

[0146] State transition: Due to S t The reactive power output of the photovoltaic inverter in the previous state is included, and the load demand is determined by the user behavior, so , used to describe the state change after the agent takes action, w t is random environmental noise, in order to simulate the randomness of load demand fluctuations.

[0147] Reward function R t : , represents the reward obtained by the agent after performing the action. The reward function is as follows: (12) and (13):

[0148] (12)

[0149] (13)

[0150] in, is the average reactive power loss; It is a normal distribution with a standard deviation of 0.1; the purpose of voltage-reactive power optimization control is to maintain the voltage within the range of 0.95pu to 1.05pu as much as possible.

[0151] In this embodiment, a Markov game model can be effectively established, thereby providing a model basis for optimizing variables, which is conducive to improving the control accuracy of voltage and reactive power control.

[0152] In an exemplary embodiment, the Markov game model of each sub-region is centrally trained to obtain a multi-agent collaborative control model, including:

[0153] Receiving global state information of the Markov game models of the plurality of sub-regions through the evaluators of the Markov game models of the sub-regions;

[0154] For the Markov game model of each sub-region, local information of the Markov game models of multiple other sub-regions is received through the executor of the Markov game model of the sub-region;

[0155] For the actuator that receives local information, the parameters of the actuator are updated using the preset policy gradient;

[0156] For the evaluator that receives the global state information, the parameters of the evaluator are updated using the preset loss function;

[0157] Based on the updated parameters of the actuator and the updated parameters of the evaluator, a multi-agent collaborative control model is obtained.

[0158] For example, the MASAC algorithm can effectively solve the sequential decision-making challenges in Markov games, and adopts a centralized training-decentralized execution architecture, which includes a centralized training stage and a decentralized execution process.

[0159] During the centralized training phase, based on a network partitioning approach, each agent's critic network receives global state information from all agents, while each agent's corresponding actor network receives local information from its own subregion. The critic network leverages this global information to support the actor network in developing a coordinated control strategy for the multi-PV voltage system.

[0160] During decentralized execution, the control strategy is determined by the actor network, so only the actor network needs to be retained. Because the actor network has already learned a collaborative control strategy that utilizes global information during the training phase, relying solely on local information during execution can provide a more collaborative and flexible control strategy.

[0161] Alternatively, the strategy set of the Markov game (MG) can be expressed as , where policy π represents the action function of agent i. The optimization goal of the MASAC algorithm is to maximize the weighted sum of cumulative benefits and entropy, as shown in the following expression (14):

[0162] (14);

[0163] in, is entropy; α is the temperature coefficient; the MASAC algorithm improves the efficiency of exploring the action space by introducing entropy.

[0164] Optionally, the parameters of the Actor Network’s policy function can be updated using policy gradients. The following expressions (15), (16), and (17) are used:

[0165] (15);

[0166] (16);

[0167] (17);

[0168] in, for The value of the action; b represents the state S t The basis term of the average value of all possible actions of agent i; The advantage function represents the value of the current action and provides the agent with an estimate of the relative value of the action.

[0169] Evaluation Network Responsible for calculating the value of an action of the agent. The parameters of the evaluation network are optimized by minimizing the following loss function, as shown in expressions (18) and (19):

[0170] (18)

[0171] (19);

[0173] in, is the target value, and are the target action and evaluation function.

[0174] Optionally, Figure 4 A schematic diagram of a centralized training-distributed execution architecture of a MASAC algorithm provided in an embodiment of the present application is shown in FIG. Figure 4As shown in the figure, Agent is an intelligent agent; Critic is an evaluator used to evaluate the value function of a state or action; Target Critic is a target evaluator used for the stabilized copy of Critic to calculate the target value (reduce training fluctuations); Loss is a loss function used to update the Critic network by comparing the predicted value and the target value; Actor is an executor used to generate an action strategy; Target Actor is a target executor used for the stabilized copy of Actor to generate the target action.

[0175] Sub-region-1 to Sub-region-N are sub-regions, and an independent agent is deployed in each sub-region: it uses the distributed parameters to execute decisions locally.

[0176] The VVC Environment is an application scenario for VVC (Voltage and Var Control), which is used for power system optimization provided in the embodiments of this application. It maintains grid stability by regulating equipment such as transformers and capacitors. The distribution network can be divided into multiple areas, requiring distributed real-time control.

[0177] Optionally, centralized training involves the central agent integrating global information to train the Actor-Critic network and optimize the strategy. Decentralized execution involves distributing the trained model parameters to each sub-region agent. Each sub-region agent makes independent decisions based on local observations to achieve rapid response.

[0178] arrive is the local environmental observation value of each sub-region, arrive Each sub-region agent is based on the local observation value ( ) calculated control instructions or decision results.

[0179] In an exemplary embodiment, a control model for voltage-reactive power optimization of a photovoltaic distribution network with high penetration is constructed with minimization of voltage deviation and reactive power loss as the objective function.

[0180] The objective function is established to minimize the voltage deviation and reactive power loss, as shown in the following expression (20):

[0181] (20);

[0182] Where T represents the number of control cycles per day; C μ Represents the voltage deviation weight coefficient; It represents the average node voltage deviation of the system in the t-th control cycle; represents the network loss weight coefficient; is the system network loss in the t-th control cycle; is the voltage at node i; is the reference voltage; n is the number of distribution network nodes.

[0183] The constraints for building the model are as follows:

[0184] The model constraints are constructed as shown in expression (21):

[0185] (twenty one);

[0186] The node voltage constraint is as shown in expression (22):

[0187] (twenty two);

[0188] The photovoltaic output constraints are as shown in expressions (23) and (24):

[0189] (twenty three);

[0190] (twenty four);

[0191] in, is the active power injected into node i at time step t; is the reactive power injected into node i at time step t; is the active load of node i; is the reactive load of node i; is the voltage amplitude of node i at time step t; is the voltage amplitude of node j at time step t; is the conductance of the branch between node i and node j; is the susceptance of the branch between node i and node j; is the voltage phase difference between node i and node j at time step t; is the lower limit of the node voltage; is the upper limit of the node voltage; The upper limit of active power output of distributed photovoltaics; is the lower limit of active power output of distributed photovoltaics; is the reactive power output of the photovoltaic inverter at time step t; is the apparent power capacity of the PV inverter at time step t; is the active power output of the photovoltaic inverter at time step t.

[0192] In this way, the electrical distance can be obtained by calculating the power flow through the control model of the photovoltaic distribution network voltage-reactive power optimization.

[0193] In some exemplary embodiments, the voltage-reactive power control problem of a distribution network can be formulated as a mixed-integer nonlinear programming problem, often solved using traditional mathematical optimization methods and heuristic algorithms. While traditional mathematical optimization methods, such as nonlinear programming and dynamic programming, can provide highly accurate solutions, they are computationally intensive and time-consuming, making them inadequate for real-time voltage-reactive power control in distribution networks. Heuristic algorithms, such as genetic algorithms and particle swarm optimization, offer advantages such as ease of programming and operability, but are prone to local optimization and require lengthy computations, making them unsuitable for large-scale voltage-reactive power control problems.

[0194] Figure 5 The present invention provides a distribution network partition voltage-reactive power optimization control method based on deep reinforcement learning, such as Figure 5 As shown, Figure 1 The voltage-reactive power control method shown may be a distribution network partition voltage-reactive power optimization control method based on deep reinforcement learning. The distribution network partition voltage-reactive power optimization control method based on deep reinforcement learning includes S501 to S505, wherein:

[0195] S501. With minimizing voltage deviation and reactive power loss as the objective function, a reactive power-voltage optimization control model for the distribution network is constructed.

[0196] Among them, as shown in Expression (20) to Expression (24), the distribution network reactive power-voltage optimization control model is constructed.

[0197] S502 : Calculate the electrical distances between distribution network nodes based on the active / reactive-voltage sensitivity matrix to obtain a weight matrix of the distribution network.

[0198] Among them, as shown in Expression (1) to Expression (9), the weight matrix of the distribution network is obtained.

[0199] S503: Calculate the node modularity based on the distribution network weight matrix, and then use the Louvain partitioning algorithm to divide the distribution network area.

[0200] Among them, as in Expression (10) to Expression (11), the power grid area is divided.

[0201] S504. Model each sub-area of ​​the distribution network as an intelligent agent, so that the voltage-reactive power optimization control problem of the distribution network is transformed into a multi-agent Markov decision process, that is, construct a Markov game (MG) model.

[0202] Among them, as in Expression (12) to Expression (14), a Markov game (MG) model is constructed.

[0203] S505. The MG model is solved based on the Multi-Agent Soft Actor-Critic (MASAC) algorithm. A centralized training-decentralized execution architecture is adopted to achieve coordinated voltage-reactive power control of photovoltaic inverters, effectively alleviating voltage fluctuations at distribution network nodes and reducing reactive power losses.

[0204] Among them, as in Expression (15) to Expression (20), the MG model is solved.

[0205] Optionally, steps S501 to S504 may be performed as Figure 1 The example of the voltage reactive power control method shown can be applied to the steps of the corresponding embodiments above. Therefore, the specific definitions of the example steps S501 to S504 can refer to the definitions of the corresponding embodiments above and will not be repeated here.

[0206] In this embodiment, the present application provides a fast, flexible, and precise distribution network voltage-reactive power optimization control method that can address existing issues with voltage regulation equipment, control methods, and algorithm optimization. The proposed method utilizes photovoltaic grid-connected inverters to rapidly adjust reactive power output to achieve real-time control of the distribution network voltage, maintaining a stable system operating voltage level while reducing costs. Distributed control is employed to partition the distribution network, with control and regulation performed within each region to fully leverage its rapid response capabilities. The MASAC algorithm, based on a centralized training-distributed execution framework, reduces communication and computational burdens, enabling photovoltaic inverter regulation to minimize voltage deviation and network reactive power losses.

[0207] In summary, there is an urgent need for a fast, flexible, and precise distribution network voltage-reactive power optimization control method that can address existing problems in voltage regulation equipment, control methods, and algorithm optimization. In the method proposed in this invention, photovoltaic grid-connected inverters are used to quickly adjust reactive power output to achieve real-time control of distribution network voltage, maintaining the stability of the system operating voltage level while reducing costs. Distributed control is used to partition the distribution network, with control and regulation performed within each region to fully utilize its rapid response capabilities. The MASAC algorithm is used, based on a centralized training-distributed execution framework, to reduce communication burdens and computational pressure, so that the photovoltaic inverter can be adjusted to minimize voltage deviation and network reactive power loss.

[0208] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0209] The voltage reactive power control device provided in the embodiment of the present application is described below. The voltage reactive power control device has the same inventive concept as the above-mentioned voltage reactive power control method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method. Therefore, the specific limitations in one or more voltage reactive power control device embodiments provided below can refer to the limitations on the voltage reactive power control method above. The voltage reactive power control device described below and the voltage reactive power control method described above can be referenced to each other and will not be repeated here.

[0210] In an exemplary embodiment, Figure 6 A schematic diagram of the structure of a voltage reactive power control device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the voltage and reactive power control device 60 includes: a state data module 610 and a reactive power control module 620, wherein:

[0211] The status data module 610 is used to obtain real-time status data of the distribution network. The real-time status data includes reactive power output, node voltage and load information of the photovoltaic inverters in each pre-divided sub-area.

[0212] The reactive power control module 620 is used to input real-time status data into a pre-trained multi-agent collaborative control model, and output reactive power control instructions through the multi-agent collaborative control model. The reactive power control instructions are used to indicate the reactive power output of the photovoltaic inverters in each sub-area.

[0213] Among them, the multi-agent collaborative control model is obtained by collective training of a Markov game model constructed with sub-regions as agents, and the sub-regions are obtained by dividing the distribution network into sub-regions based on the node modularity determined by the electrical distance between the distribution network nodes.

[0214] In an exemplary embodiment, the device also includes a model training module, which is used to obtain the electrical distance between nodes of the distribution network; determine the node modularity based on the electrical distance; use a preset partitioning algorithm to optimize the node modularity and divide the distribution network into multiple sub-areas with tight electrical coupling; construct a Markov game model with each sub-area as an intelligent agent; wherein the state set of the Markov game model includes the historical reactive output of the photovoltaic inverter and the node voltage, and the action set of the Markov game model includes the coordinated reactive output of the photovoltaic inverter; the Markov game model of each sub-area is centrally trained to obtain a multi-agent collaborative control model.

[0215] In an exemplary embodiment, the model training module is used to obtain the voltage-reactive electrical distance and the voltage-active electrical distance; the electrical distance between nodes is determined by the following expression:

[0216]

[0217] in, is the voltage-reactive electrical distance; is the voltage-active electrical distance.

[0218] In an exemplary embodiment, the voltage-reactive electrical distance is obtained by the following expression:

[0219]

[0220] The voltage-active electrical distance is obtained by the following expression:

[0221]

[0222] in, (i=1, 2, …, n; j=1, 2, …, n) represents the ratio of the voltage amplitude change at node vj to the voltage amplitude change at node vi when the reactive power of node vj changes, reflecting the voltage-reactive power influence of node vj on node vi. (i=1, 2, …, n; j=1, 2, …, n) represents the ratio of the voltage amplitude change of node vj to the voltage amplitude change of node vi when the active power of node vj changes, reflecting the voltage-active power influence of node vj on node vi.

[0223] In an exemplary embodiment, the model training module is used to determine a weight matrix according to the electrical distance; and obtain the node modularity according to the weight matrix and a preset modularity function.

[0224] In an exemplary embodiment, the model training module is used to determine the original community area of ​​each node; for each node, the node is assigned to the community area of ​​the adjacent nodes connected to the node;

[0225] Iteratively optimize the node modularity through the following expression:

[0226]

[0227] in, is the sum of the weights of all edges in the original community area of ​​the node, is the sum of the weights of the edges connecting the nodes in the original community area of ​​the node, is the sum of the weights of the edges connected to the node, is the sum of the weights of the edges connecting the node to the original community area, and m is the sum of the weights of all edges in the network;

[0228] If the node modularity reaches the maximum modularity, the currently allocated multiple community areas are determined as multiple sub-areas with close electrical coupling.

[0229] In an exemplary embodiment, the model training module is used to receive the global state information of the Markov game models of multiple sub-regions through the evaluator of the Markov game model of each sub-region; for the Markov game model of each sub-region, the local information of the Markov game models of multiple other sub-regions is received through the executor of the Markov game model of the sub-region; for the executor receiving the local information, the parameters of the executor are updated using the preset policy gradient; for the evaluator receiving the global state information, the parameters of the evaluator are updated using the preset loss function; based on the updated parameters of the executor and the updated parameters of the evaluator, a multi-agent collaborative control model is obtained.

[0230] In an exemplary embodiment, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors execute the steps of the voltage reactive power control method as described in any one of the above embodiments.

[0231] In an exemplary embodiment, the present application also provides a control device having a computer program stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the voltage reactive power control method as described in any one of the above embodiments.

[0232] In an exemplary embodiment, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any voltage and reactive power control method in the above embodiments.

[0233] Schematically, as Figure 7 As shown, Figure 7 This is a schematic diagram of the internal structure of a control device provided in an embodiment of the present application. The control device 700 can be provided as a server. Figure 7 The control device 700 includes a processing component 702, which further includes one or more processors, and a memory resource represented by a memory 701 for storing instructions executable by the processing component 702, such as an application. The application stored in the memory 701 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 702 is configured to execute the instructions to perform the text recognition method of any of the above embodiments.

[0234] The control device 700 may further include a power supply component 703 configured to perform power management of the control device 700, a wired or wireless network interface 704 configured to connect the control device 700 to a network, and an input / output (I / O) interface 705. The control device 700 may operate based on an operating system stored in the memory 701, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0235] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the control device to which the scheme of the present application is applied. The specific control device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0236] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0237] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0238] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0239] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voltage reactive power control method, characterized in that: The method comprises: Acquiring real-time status data of the distribution network, including reactive power output, node voltage, and load information of photovoltaic inverters in each pre-divided sub-area; Inputting the real-time status data into a pre-trained multi-agent collaborative control model, and outputting a reactive control instruction through the multi-agent collaborative control model, wherein the reactive control instruction is used to indicate the reactive output of the photovoltaic inverter in each of the sub-areas; The multi-agent collaborative control model is obtained by collective training of a Markov game model constructed with sub-regions as agents, and the sub-regions are obtained by dividing the distribution network by the node modularity determined by the electrical distance between the distribution network nodes.

2. The method according to claim 1, characterized in that The training method of the multi-agent collaborative control model includes: Obtaining electrical distances between nodes of the distribution network; determining a node modularity based on the electrical distance; Utilizing a preset partitioning algorithm, the node modularity is optimized to divide the distribution network into a plurality of sub-areas with close electrical coupling; A Markov game model is constructed with each of the sub-regions as an intelligent agent; wherein the state set of the Markov game model includes the historical reactive output of the photovoltaic inverter and the node voltage, and the action set of the Markov game model includes the coordinated reactive output of the photovoltaic inverter; The Markov game model of each sub-region is centrally trained to obtain the multi-agent collaborative control model.

3. The method according to claim 2, characterized in that The obtaining of the electrical distance between nodes of the distribution network includes: Obtain voltage-reactive electrical distance and voltage-active electrical distance; The electrical distance between nodes is determined by the following expression: in, is the voltage-reactive electrical distance; is the voltage-active electrical distance.

4. The method according to claim 3, characterized in that The voltage-reactive electrical distance is obtained by the following expression: The voltage-active electrical distance is obtained by the following expression: in, (i=1, 2, ..., n; j=1, 2, ..., n) means when node v j When the reactive power changes, the node v j The voltage amplitude change and node v i The ratio of the voltage amplitude change reflects the node v j For node v i Voltage-reactive power influence capability; (i=1, 2, ..., n; j=1, 2, ..., n) represents when node v j When the active power changes, the node v j The voltage amplitude change and node v i The ratio of the voltage amplitude change reflects the node v j For node v i Voltage-active power influence capability.

5. The method according to claim 2, characterized in that The determining of the node modularity based on the electrical distance includes: determining a weight matrix according to the electrical distance; The node modularity is obtained according to the weight matrix and a preset modularity function.

6. The method according to claim 2, characterized in that The method of utilizing a preset partitioning algorithm to optimize the node modularity and divide the distribution network into a plurality of sub-areas with close electrical coupling includes: Determine the original community area where each of the nodes is located; For each of the nodes, assign the node to a community area where adjacent nodes connected to the node are located; Iteratively optimize the node modularity through the following expression: in, is the sum of the weights of all edges in the original community area of ​​the node, is the sum of the weights of the edges connecting the nodes in the original community area of ​​the node, is the sum of the weights of the edges connected to the node, is the sum of the weights of the edges connecting the node to the original community area, and m is the sum of the weights of all edges in the network; If the node modularity reaches the maximum modularity, the currently allocated multiple community areas are determined as the multiple sub-areas that are tightly electrically coupled.

7. The method according to claim 2, characterized in that The Markov game model of each sub-region is centrally trained to obtain the multi-agent collaborative control model, including: Receiving global state information of the Markov game models of the plurality of sub-regions through an evaluator of the Markov game model of each sub-region; For each of the Markov game models of the sub-region, receiving local information of the Markov game models of multiple other sub-regions through an executor of the Markov game model of the sub-region; For an actuator receiving local information, updating parameters of the actuator using a preset policy gradient; For an evaluator receiving the global state information, updating the parameters of the evaluator using a preset loss function; Based on the updated parameters of the actuator and the updated parameters of the evaluator, the multi-agent collaborative control model is obtained.

8. A voltage and reactive power control device, characterized in that: The device comprises: A status data module is used to obtain real-time status data of the distribution network, including reactive power output, node voltage and load information of photovoltaic inverters in each pre-divided sub-area; a reactive power control module, configured to input the real-time status data into a pre-trained multi-agent collaborative control model, and output reactive power control instructions through the multi-agent collaborative control model, wherein the reactive power control instructions are used to indicate the reactive power output of the photovoltaic inverters in each of the sub-areas; The multi-agent collaborative control model is obtained by collective training of a Markov game model constructed with sub-regions as agents, and the sub-regions are obtained by dividing the distribution network by the node modularity determined by the electrical distance between the distribution network nodes.

9. A control device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images