Power distribution network reactive voltage control method and device based on physical guidance multi-scale spatio-temporal fusion multi-agent deep reinforcement learning

Abstract

The application provides a power distribution network reactive voltage control method and device based on physical guidance multi-scale spatio-temporal fusion multi-agent deep reinforcement learning, and belongs to the field of power distribution network reactive voltage control; solves the problem that the wide access of heterogeneous distributed power sources leads to significant increase of power distribution network voltage fluctuation and operation uncertainty, easily causes voltage out-of-limit, network loss increase and reactive power coordination difficulty and the like; the method constructs a PGMF-MADRL model, firstly, a regional voltage-reactive sensitivity model is constructed, and the power distribution network partition reactive voltage control problem is modeled as a distributed partially observable Markov decision process; secondly, an adaptive sensitivity guided graph attention network is constructed; then, a fast-slow dual-channel multi-scale state space model is constructed, and feature fusion is realized through a sensitivity guided fusion gate; the application is applied to power distribution network reactive voltage control.

Description

Technical Field

[0001] This application relates to the field of reactive power and voltage control in distribution networks, and in particular to a method and device for reactive power and voltage control in distribution networks based on physical-guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning. Background Technology

[0002] The integration of heterogeneous distributed power sources such as distributed photovoltaic (PV) and decentralized wind power into distribution networks continues to increase. Due to the randomness, volatility, and intermittency of renewable energy output, the complexity of distribution network operation and the difficulty of voltage regulation are constantly increasing. The voltage regulation problem has gradually evolved from local regulation under traditional stable operating conditions to a dynamic and coordinated regulation problem dominated by the fluctuations of wind, solar, and load. In such complex dynamic operating scenarios, distribution networks are more prone to voltage exceeding limits and increased network losses, posing a severe challenge to the safe and economical operation of the system. Existing methods have limited adaptability in complex dynamic operating scenarios dominated by the fluctuations of wind, solar, and load, and are insufficient to meet the voltage regulation requirements of distribution networks. Therefore, how to balance voltage regulation response speed and voltage stabilization and loss reduction effects has become a key issue in distribution network voltage control research. Summary of the Invention

[0003] To address the significant increase in voltage fluctuations and operational uncertainties in distribution networks caused by the widespread integration of heterogeneous distributed power sources, which can easily lead to voltage exceeding limits, increased network losses, and difficulties in reactive power coordination, this application proposes a reactive power and voltage control method and device for distribution networks based on physical-guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning.

[0004] The technical solution adopted in this application is as follows: a reactive power and voltage control method for distribution networks based on physical-guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning is proposed. This method constructs and trains a PGMF-MADRL model, and uses the trained PGMF-MADRL model to perform reactive power and voltage control of the distribution network. The method includes the following steps:

[0005] Step 1: Model the zoned reactive power and voltage control, including:

[0006] Step 1.1: Construct a regional voltage-reactive power sensitivity model;

[0007] Step 1.2: Determine the voltage control target and the physical feasible region constraints of the equipment;

[0008] Step 1.3: Modeling of Zoned Reactive Voltage Control (Dec-POMDP);

[0009] Step 2: Construct the PGMF network, including:

[0010] Step 2.1: Construct an adaptive sensitivity-guided graph attention network;

[0011] Step 2.2: Construct a multi-scale selective Mamba state-space model;

[0012] Step 2.3: Construct a sensitivity-guided multi-scale fusion gating system;

[0013] Step 3: Train and execute the PGMF-MADRL model.

[0014] Furthermore, the regional voltage-reactive power sensitivity model in step 1.1 is built on the basis of the node-level sensitivity. First, the relationship between branch power and node voltage is characterized based on the radial distribution network branch power flow DistFlow model to obtain the node-level voltage-reactive power sensitivity matrix. Then, the mapping relationship between node reactive power injection and device reactive power command is established to obtain the device-level voltage-reactive power sensitivity matrix. Finally, the device-level voltage-reactive power sensitivity is mapped to the regional control scale to obtain the regional voltage-reactive power sensitivity matrix.

[0015] Furthermore, in step 1.2, a multi-time period voltage collaborative control model is constructed with the goal of minimizing node voltage deviation, voltage over-limit penalty, network loss, and reactive power fluctuation. The physical feasible domain constraints of the equipment include reactive power regulation constraints of the distributed photovoltaic or distributed wind power inverters connected to the nodes and reactive power output constraints of the static var compensators.

[0016] Furthermore, the Dec-POMDP constructed in step 1.3 is used to describe the distributed collaborative decision-making process of multiple agents under local observation. It includes an agent set, a regional interaction graph, a state space, an observation space, an action space, a state transition probability function, a reward function, a discount factor, and the time domain length of a single control task. By learning the optimal joint policy, each regional agent can reduce voltage deviation, limit violation, network loss, and action fluctuation while satisfying the physical feasible domain constraints of the device.

[0017] Furthermore, the PGMF network is embedded in the MADRL Critic network, while the Actor network adopts a simplified structure oriented towards online execution, generating regional reactive power control commands based solely on local observations and short-window historical information. The execution steps of the Actor network are as follows:

[0018] First, local physical features on the Actor side are constructed and used as input for short-window temporal modeling. Then, short-window state space representations and actions on the Actor side are generated to characterize the short-term change trend of local states. Based on the local physical features input by the short window, dynamic features related to the generation of the current action are obtained and normalized. Finally, the normalized actions are mapped and projected onto the physical feasible region of the corresponding area.

[0019] Furthermore, the adaptive sensitivity-guided graph attention network embeds the regional voltage-reactive sensitivity, which is recursively calibrated based on short-term operating measurements, into the attention weight calculation process as physical guidance information, so that spatial feature aggregation can simultaneously reflect the data-driven state correlation and the reactive-voltage coupling relationship between regions.

[0020] The steps of the regional voltage-reactive power sensitivity recursive calibration include: first, calculating the recursive gain vector, and then updating the regional voltage-reactive power sensitivity matrix according to the residual voltage response error. The residual voltage response error is the residual voltage response dominated by reactive power control action constructed by introducing the regional voltage-active power sensitivity approximation matrix.

[0021] Furthermore, the multi-scale selective Mamba state-space model extracts high-frequency dynamic features and low-frequency trend features through fast and slow dual channels, respectively.

[0022] Furthermore, sensitivity-guided multi-scale fusion gating utilizes regional electrical coupling information to guide the fusion of fast and slow dual-channel features.

[0023] Furthermore, the PGMF-MADRL model adopts a centralized training and distributed execution framework. Each region agent is configured with a local Actor network to generate region actions based on local historical information. All regions share a centralized Critic network, which successively includes an adaptive sensitivity-guided graph attention network, a multi-scale selective Mamba state space model, and a sensitivity-guided multi-scale fusion gating to perform joint spatiotemporal representation of joint historical inputs.

[0024] The centralized Critic network inputs the joint spatiotemporal representation obtained after sensitivity-guided multi-scale fusion gating and the joint actions output by the Actor networks in each region into the value evaluation layer to obtain the joint action value.

[0025] Based on joint action value estimation, the environment provides immediate rewards and the next state, and forms interactive samples that are stored in the experience replay pool. When the number of samples reaches the training requirements, a small batch of samples is randomly selected from the experience replay pool for parameter updates. After training converges, the parameters of the Actor network in each region are saved and sent to the corresponding controller. In the distributed execution phase, the centralized Critic network is removed, and each region generates control actions independently based solely on local information.

[0026] This application also proposes a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0027] The advantages of this application over the prior art are as follows:

[0028] 1) Considering the difficulty in fully describing the impact of regional joint reactive power operation by the sensitivity at the node or inverter level, a regional voltage-reactive power sensitivity model is constructed to characterize the regional voltage-reactive power coupling relationship. Adaptive calibration is performed using operational measurement information, thereby improving the ability to characterize the changes in electrical coupling between regions under complex dynamic operating scenarios and providing a physical guidance basis for collaborative decision-making of multi-regional intelligent agents.

[0029] 2) To address the problem that existing methods often use a unified modeling approach for historical states and are insufficient in characterizing the multi-scale dynamic characteristics under the mixed fluctuations of wind, solar and load, a multi-scale Mamba selective state-space model is constructed. By using a fast and slow dual-channel structure to characterize the operating characteristics under different scales of change, the model's ability to utilize long-term historical operating information and its adaptability to online voltage regulation are enhanced.

[0030] 3) To address the issue that different information fusion methods are relatively direct and the guiding role of regional electrical coupling information is insufficient, regional voltage-reactive power sensitivity information is introduced to participate in the multi-scale dynamic feature fusion process, thereby enhancing the ability to coordinate the use of physical information and operational dynamic information, and thus improving the voltage coordinated control capability under complex dynamic operating scenarios. Attached Figure Description

[0031] The following description, in conjunction with the accompanying drawings, further illustrates this application:

[0032] Figure 1 A framework diagram of the PGMF-MADRL model provided in the embodiments of this application;

[0033] Figure 2 This is a flowchart illustrating the method provided in an embodiment of this application. Detailed Implementation

[0034] like Figures 1 to 2As shown, this application provides a reactive power and voltage control method for distribution networks based on physics-guided multi-scale spatio-temporal fusion multi-agent deep reinforcement learning (PGMF-MADRL). First, it models the regional reactive power and voltage control, constructing a regional voltage-reactive power sensitivity model, thus modeling the regional reactive power and voltage control problem of the distribution network as a distributed partially observable Markov decision process. Second, it constructs a physics-guided multi-scale spatio-temporal fusion (PGMF) network architecture, including constructing an adaptive sensitivity-guided graph attention network. Then, it constructs a fast and slow dual-channel multi-scale Mamba selective state-space model (MS-Mamba), and achieves feature fusion through sensitivity-guided fusion gating. Finally, it trains the PGMF-MADRL deep reinforcement learning model.

[0035] The specific implementation steps of the method in this application are as follows:

[0036] Step 1: Model the reactive power and voltage control of the zones;

[0037] In complex dynamic operating scenarios dominated by wind, solar, and load fluctuations, the impact of reactive power actions on node voltage varies across different regions of the distribution network, and the electrical coupling relationships between regions change with operating conditions. To accurately describe zoned reactive power and voltage control, this application first constructs a regional-level voltage-reactive power sensitivity model; then, it provides the voltage control objective function and the physical feasible region constraints for controllable reactive power equipment; finally, it models the zoned reactive power and voltage control problem as a decentralized partially observable Markov decision process (Dec-POMDP), with specific implementation steps including:

[0038] Step 1.1: Construct a regional voltage-reactive power sensitivity model;

[0039] Existing voltage-reactive power sensitivity modeling often focuses on the node or individual device level, making it difficult to directly reflect the overall impact of coordinated reactive power actions on regional voltage in zoned control. Therefore, this application constructs a regional-level voltage-reactive power sensitivity model based on node-level sensitivity.

[0040] The relationship between branch power and node voltage is characterized by the DistFlow model of a radial distribution network. A radial distribution network considering single-phase equal-value distribution consists of a set of nodes. Hezhiluji The structure consists of nodes, where node 0 is the balanced node. For any branch... Under the condition that the node voltage is close to the rated value and the second-order loss term is small, the linearization of the DistFlow model yields:

[0041] (1);

[0042] In the formula, , They are nodes The voltage amplitude; , They are the branch roads that flow through. The active and reactive power at the head end; This represents the line impedance.

[0043] Let the voltage magnitude vector of all nodes in the network be... The net reactive power injection vector of the node is Then the node-level voltage-reactive power sensitivity matrix Defined as:

[0044] (2);

[0045] In the formula, The total number of nodes. It is the set of real numbers.

[0046] Since the actual control variable is the reactive power command of the controllable device, a mapping relationship between node reactive power injection and device reactive power command is established. Let the node-device mapping matrix be... Then the device-level voltage-reactive power sensitivity matrix It can be represented as:

[0047] (3);

[0048] In the formula, The number of controllable reactive power devices.

[0049] Furthermore, considering that this application employs regional intelligent agents for joint control, the device-level sensitivity is mapped to the regional control scale. Let's assume... In each region, the aggregation matrix from node voltage to region representative voltage is: The allocation matrix from regional joint actions to device-level reactive power commands is as follows: Then the regional voltage-reactive power sensitivity matrix It can be represented as:

[0050] (4).

[0051] The above regional voltage-reactive power sensitivity matrix The detailed derivation process is as follows:

[0052] In a radial distribution network, changes in power injected into a node are propagated along a single path to the slack node. Define the branch-node path matrix. If branch road Located from the balanced node to the node On the path, then Otherwise, it is 0. Under the linearization assumption that the node voltage is close to the rated value and the second-order loss term is small, the elements in the node-level voltage-reactive power sensitivity matrix can be obtained from equation (1). It can be approximated as:

[0053] (5);

[0054] In the formula, Path matrix Elements in; branch road The reactance.

[0055] Considering that the actual control variable is not the net reactive power injection of all network nodes, but the reactive power command of the controllable reactive power devices, let the set of controllable reactive power devices be denoted as . Its reactive motion vector is denoted as Define the node-device mapping matrix If the first Each controllable device access node ,but ,otherwise The net reactive power injection at a node can be written as:

[0056] (6);

[0057] In the formula, Inject vectors for load and uncontrollable reactive power.

[0058] The device-level voltage-reactive power sensitivity matrix can be obtained using the chain rule:

[0059] (7).

[0060] Assume the distribution network is divided into: Each region defines the node voltage. The aggregation matrix representing the voltage of the region is as follows The allocation matrix for the controllable device to the regional joint reactive power action is as follows: The region represents the voltage vector. Joint action vector with region It can be written as:

[0061] (8);

[0062] (9);

[0063] From the above formula, we can obtain:

[0064] (10);

[0065] Therefore, the regional voltage-reactive power sensitivity matrix is:

[0066] (4).

[0067] Step 1.2: Determine the voltage control target and the physical feasible region constraints of the equipment;

[0068] This application uses distributed photovoltaic (DPV), distributed wind generation (DWG) inverters, and static var compensators (SVCs) as controllable reactive power regulation devices. The DPV operates in maximum power point tracking / static synchronous compensator mode, without reducing active power output; the DWG adopts an active power priority operation mode. Both utilize remaining reactive power capacity for voltage regulation under rated apparent capacity constraints. The SVC provides continuous reactive power compensation within the capacity boundary. Based on the above operating assumptions, to balance voltage safety, operational economy, and control smoothness, this application constructs a multi-time period voltage collaborative control model with the goal of minimizing node voltage deviation, voltage over-limit penalties, network losses, and reactive power fluctuations.

[0069] (11);

[0070] Among them, the voltage deviation and over-limit penalty terms are defined as follows:

[0071] (12);

[0072] In the formula, To optimize the objective function; For expectation operators; To optimize the number of control steps in the time domain; The total number of nodes; For nodes At any moment The voltage amplitude; This is the rated voltage amplitude; These are the lower and upper limits of the node voltage safety, respectively. For a moment The cost of voltage control; This is the penalty coefficient for voltage exceeding the limit; This refers to the system's active power loss; This represents the change in reactive power at adjacent control moments; These are the weighting coefficients for voltage control cost, network loss, and motion smoothing term, respectively.

[0073] For access nodes DPV or DWG inverters, which at time The reactive power regulation is constrained by both the rated apparent capacity and the real-time active power output. The feasible region is expressed as:

[0074] (13);

[0075] In the formula, For distributed power nodes The reactive power output; The apparent power capacity of the inverter; For a moment The active power; These are the lower and upper limits of the reactive power that the inverter is allowed to adjust, respectively.

[0076] For SVC, its reactive power output is only limited by the device capacity:

[0077] (14);

[0078] In the formula, Indicates the first The actual reactive power output variable of each SVC; These are the lower and upper limits of reactive power that the SVC can adjust, respectively.

[0079] In subsequent reinforcement learning control, the objective function is directly solved online under the conditions of multi-region coupling and local observability. The problem is quite difficult, so the reactive voltage control problem of the partition is modeled as Dec-POMDP.

[0080] Step 1.3: Modeling of Zoned Reactive Voltage Control (Dec-POMDP);

[0081] In zoned reactive power and voltage control, there are significant coupling relationships between different zones, and it is difficult to obtain the complete state of the entire network. Dec-POMDP can describe the distributed collaborative decision-making process of multiple agents under local observation. Therefore, this application models it as Dec-POMDP, expressed as:

[0082] (15);

[0083] In the formula, A collection of intelligent agents; This is a regional interaction diagram; For state space; For observation space; For action space; Let be the state transition probability function; For the reward function; Discount factor; The time domain length of a single control task.

[0084] The parameters in Dec-POMDP are explained below:

[0085] 1) Agent Set and Region Interaction Graph: Each region in the distribution network is considered as an agent, and the agent set is denoted as... ,in The number of regions is defined by this partition. Each agent is responsible for the coordinated control of controllable reactive power regulation devices such as DPV, DWG, and SVC within its region. To characterize the electrical coupling relationships between regions, a region interaction graph is constructed. .in, It is a set of boundaries between regions, determined by physical adjacency relationships; The set of edge weights is constructed from the regional voltage-reactive power sensitivity matrix and is used to characterize the coupling strength of reactive power regulation between regions.

[0086] 2) State and Observation: The system at time t: The global state is denoted as This includes information such as the voltage of all network nodes, source load power, and the operating status of controllable reactive power devices. Considering that it is difficult for each region to obtain complete information about the entire network during the execution phase, the intelligent agent... At any moment Local observations are denoted as It is composed of local measurements in this region, joint actions performed in the previous time step, and shared boundary / representative information of adjacent regions:

[0087] (16);

[0088] In the formula, respectively intelligent agents At any moment Local voltage, active and reactive power measurement information; This refers to the combined reactive power actions that were performed in this area at the previous control time. For the image Middle intelligence agent The set of neighbors; Adjacent areas Shared boundary / represents voltage information.

[0089] 3) Action: Intelligent agent At any moment The action is recorded as This indicates the joint reactive power control command for this region:

[0090] (17);

[0091] In the formula, , and They are respectively regions Regional-level joint reactive power regulation of internal DPV, DWG and SVC. Each action output must satisfy the physical feasible domain constraints of the equipment given in equations (13) and (14) above.

[0092] 4) State transition: The state transition probability function is denoted as:

[0093] (18);

[0094] In the formula, This is a combined action. The state transition is jointly determined by changes in power flow in the distribution network, source load fluctuations, and combined reactive power actions.

[0095] 5) Reward and Optimization Objectives: To ensure that the reinforcement learning objective aligns with the voltage control objective, this application employs a normalized negative cost approach to construct single-step rewards. :

[0096] (19);

[0097] In the formula, and These are the network loss and action change normalization coefficients in the training set, respectively.

[0098] This application aims to learn optimal joint strategies. This enables agents in each region to reduce voltage deviation, limit exceedance, network loss, and action fluctuations while meeting the physical feasible domain constraints of the equipment.

[0099] (20);

[0100] In the formula, For joint strategy; For intelligent agents Local strategies.

[0101] In complex operating scenarios involving mixed fluctuations of wind, solar, and load, existing methods still fall short in adapting to the dynamic operating characteristics of distribution networks and regional coordinated voltage regulation relationships. Therefore, this application proposes an architecture based on physics-guided multi-scale spatio-temporal fusion (PGMF) and embeds it into the Critic network. This architecture mainly includes an adaptive sensitivity-guided graph attention network, a multi-scale Mamba selective state-space model, and sensitivity-guided multi-scale fusion gating.

[0102] The Actor network employs a simplified structure designed for online execution, generating regional reactive power control commands solely based on local observations and short-window historical information. The execution steps of the Actor network are as follows:

[0103] First, construct the local physical features on the Actor side:

[0104] During the distributed execution phase, the Critic does not participate in online control; each regional agent can only generate control commands based on local measurements, shared information from neighboring regions, and historical action information. To reduce online computational complexity and enhance the Actor's ability to perceive local voltage shifts, reactive power regulation margins, and neighborhood coupling states, this application constructs local physical features on the Actor side and uses them as input for short-window timing modeling.

[0105] To extract low-dimensional physical features suitable for online action generation, a selection vector and candidate feature vector are first obtained through gating mapping:

[0106] (twenty one);

[0107] (twenty two);

[0108] Subsequently, the gating vector is multiplied element-wise with the candidate features to obtain the local physical feature vector on the Actor side:

[0109] (twenty three);

[0110] In the formula, This is the gating selection vector; These are candidate physical feature vectors; The local physical features are fed into the Actor short-window state space representation module; The weight matrix is ​​trainable. For bias terms; This indicates that corresponding elements are multiplied.

[0111] Then, the short-window state space representation and actions of the Actor are generated:

[0112] Considering the need for rapid response during the online control phase, the Actor does not adopt the dual-channel MS-Mamba structure of the Critic, but instead retains only a short-window single-channel state-space representation module to characterize the short-term changing trends of local states. Let the historical window length on the Actor side be... Then the intelligent agent At any moment Local physical feature sequence for:

[0113] (twenty four);

[0114] Based on the local physical feature sequence of the short window input mentioned above, dynamic features related to the generation of the current action can be obtained:

[0115] (25);

[0116] Then, the Actor generates a normalized action output based on the short-window dynamic features:

[0117] (26);

[0118] To ensure that the actual actions meet the reactive power regulation range of the equipment, the normalized actions are mapped and projected onto the region. Physically feasible region:

[0119] (27);

[0120] In the formula, For the single-channel state-space representation operator on the Actor side; Generate an action network; For the region Upper limit of the operating amplitude of the internally controllable reactive power device; This indicates that the motion will be projected or clipped to the area. Device physical feasible domain .

[0121] Step 2: Construct the PGMF network framework, including:

[0122] Step 2.1: Construct an adaptive sensitivity-guided graph attention network;

[0123] During the centralized training phase, the Critic network needs to evaluate the global value of joint reactive power actions based on the operating status of each region. If only fixed adjacency relationships are used or regional features are simply pieced together, it is difficult to distinguish the differences in electrical coupling strength between different regions, and it is also difficult to adapt to changes in operating points caused by source-load fluctuations. Graph attention networks, through an attention mechanism, adaptively weight and aggregate the features of neighboring nodes, making them suitable for characterizing non-uniform electrical connections between regions in a distribution network. Therefore, this application introduces an adaptive sensitivity-guided graph attention network (ASG-GAT) into the Critic network, embedding regional voltage-reactive power sensitivity as physical guidance information into the attention weight calculation process. This allows spatial feature aggregation to simultaneously reflect data-driven state correlations and inter-regional reactive power-voltage coupling relationships.

[0124] The sensitivity information used in ASG-GAT is not a fixed static parameter, but rather a recursive calibration based on short-term operational measurements. Considering that the regional representative voltage changes in actual operation are not only caused by reactive power control actions, but also affected by active power fluctuations from photovoltaic, wind power, and loads, this application first performs approximate compensation for voltage disturbances caused by active power fluctuations before recursive calibration, in order to reduce the risk of misattributing source-load fluctuations to reactive power actions.

[0125] The steps for recursive calibration of regional voltage-reactive power sensitivity are as follows:

[0126] Set time The region-representing voltage vector, region-reactive power injection vector, and region-reactive power action vector are respectively... , and The increment of its adjacent control steps is:

[0127] (28).

[0128] To mitigate the interference of active power fluctuations on sensitivity calibration, a regional voltage-active power sensitivity approximation matrix is ​​introduced. And construct the residual voltage response dominated by reactive power control action. :

[0129] (29).

[0130] Residual voltage response under the first-order approximation Regional reactive power increment The following conditions must be met:

[0131] (30);

[0132] In the formula, For a moment The regional voltage-reactive power sensitivity matrix after recursive calibration.

[0133] To update the regional voltage-reactive power sensitivity matrix online, a recursive least squares approach is used for calibration. First, the recursive gain vector is calculated:

[0134] (31);

[0135] Then, the regional voltage-reactive power sensitivity matrix is ​​updated based on the residual voltage response error:

[0136] (32);

[0137] The recursive covariance matrix is ​​updated as follows:

[0138] (33)

[0139] In the formula, Forgetting factor; The recursive covariance matrix; This is the transpose of the recursive gain vector, used to update the regional voltage-reactive power sensitivity matrix.

[0140] Set up a region At any moment The input feature vector is The adaptively calibrated regional voltage-reactive power sensitivity elements are: , indicating region reactive action on the area The degree of influence of voltage response. For the region and its neighboring areas The regional features of both and the sensitivity guide term are input into the attention scoring function to obtain the attention coefficient:

[0141] (34);

[0142] To ensure the comparability of the influence of different neighboring regions on the target region, the attention coefficients obtained from equation (34) are normalized using the Softmax function, resulting in normalized aggregate weights:

[0143] (35);

[0144] After obtaining the normalized weights, the features of the neighboring regions are weighted and aggregated to obtain the region. At any moment Spatial feature vectors:

[0145] (36);

[0146] In the formula, For the region The set of neighboring regions; To share the linear transformation matrix; This is the attention weight vector; This indicates vector concatenation; This is a sensitivity nonlinear mapping function; It is a learnable physical guidance factor used to modulate the degree of influence of regional sensitivity on attention distribution; and These are nonlinear activation functions.

[0147] Step 2.2: Construct a multi-scale selective Mamba state-space model;

[0148] After obtaining the regional spatial characteristics, the Critic network also needs to characterize the temporal changes caused by source-load fluctuations and continuous voltage regulation. Distribution network voltage dynamics include both fast-changing components such as sudden changes in photovoltaic output, short-term fluctuations in wind power, and random load disturbances, as well as slow-changing trends caused by intraday variations in sunshine, diurnal variations in load, and changes in meteorological conditions. If only a single time window is used for historical state modeling, it is difficult to simultaneously consider both short-term disturbance responses and long-term historical trend utilization. State-space models have good computational efficiency and long-range information preservation capabilities in long-sequence dependency modeling, while Mamba introduces a selective state-space mechanism, allowing model parameters to dynamically change with the input, thus balancing long-sequence modeling capabilities and linear complexity inference efficiency. Therefore, this application constructs MS-Mamba in the Critic network, extracting high-frequency dynamic features and low-frequency trend features through fast and slow dual channels respectively.

[0149] Distribution network load evolution typically includes both short-term rapid disturbances and slow-changing trends, making it difficult for single-scale time-series models to simultaneously capture transient dynamic perception and extract long-range dependencies. Therefore, this application employs a selective state-space model to construct the MS-Mamba time-series modeling module. Let the continuous state-space model be:

[0150] (37);

[0151] In the formula, It is in a hidden state; The input sequence is... The output sequence; This is the state transition matrix; These are the input projection matrix and the output readout matrix, respectively.

[0152] To apply equation (37) to discrete control processes, a time scale parameter is introduced. The discretized form can be obtained as follows:

[0153] (38);

[0154] (39).

[0155] To enhance the model's selective memorization ability for different input content, the discrete time step and input / output projection parameters are dynamically changed with the current input:

[0156] (40);

[0157] Therefore, the discrete state update and output can be expressed as:

[0158] (41);

[0159] (42);

[0160] In the formula, and These are the discrete state transition term and the discrete input projection term, respectively. It is the identity matrix; Input the relevant discrete time step parameters; It is a smooth activation function; and These are the dynamic input projection item and the dynamic output readout item, respectively. These are trainable parameters.

[0161] Suppose the region obtained after ASG-GAT extraction At any moment The spatial feature vector is The construction length is The high-frequency input sequence and the length of The low-frequency original input sequences are as follows:

[0162] (43);

[0163] (44);

[0164] In the formula, For the region At any moment High-frequency input sequences; For the region At any moment The low-frequency raw input sequence.

[0165] Considering the relatively long original history window of the low-frequency channel, direct input would increase the computational burden and potentially introduce excessive high-frequency redundant information. Therefore, time-averaged pooling is performed on the original low-frequency input sequence to obtain the low-frequency input sequence. :

[0166] (45);

[0167] Subsequently, the high-frequency input sequence and low-frequency input sequences The data is fed into the corresponding Mamba mapping modules to obtain the regions. At any moment High-frequency dynamic characterization and low-frequency trend characterization :

[0168] (46);

[0169] (47);

[0170] Then, by combining the high-frequency dynamic characteristics and low-frequency trend characteristics of all regions respectively, we can obtain:

[0171] (48);

[0172] (49);

[0173] In the formula, For time-average pooling operators, The pooling length; and These are selective state-space mapping operators for the high-frequency channel and the low-frequency channel, respectively.

[0174] Step 2.3: Construct a sensitivity-guided multi-scale fusion gating system;

[0175] After extraction using MS-Mamba, the Critic network obtained high-frequency dynamic representations and low-frequency trend representations. The importance of these two representations for the joint action value assessment varies under different operating conditions. For example, when there are rapid fluctuations in source load or enhanced regional voltage coupling, the model needs to pay more attention to the high-frequency dynamic response; while when the operating condition is relatively stable, low-frequency trend information has a stronger reference value for value assessment. Directly using splicing or fixed-weight fusion makes it difficult to adaptively adjust the contribution of features at different time scales according to the current operating conditions. Therefore, this application constructs a sensitivity-guided multi-scale fusion gate (SMFG) to guide the fusion of fast and slow channel features using regional electrical coupling information.

[0176] set up The regional sensitivity score, extracted from the currently adaptively calibrated regional voltage-reactive power sensitivity matrix, is used to characterize the region. The electrical coupling strength at the current operating point. A region is defined to adaptively allocate fast and slow channel weights based on the current operating state. At any moment The fusion gate weights are:

[0177] (50);

[0178] Equation (50) yields Used to characterize the contribution of the high-frequency channel under current operating conditions. This corresponds to the contribution level of the low-frequency channel. Therefore, for the region... The high-frequency dynamic representation and low-frequency trend representation are adaptively fused to obtain a joint spatiotemporal representation of the region. :

[0179] (51);

[0180] Then, by combining the joint spatiotemporal representations of all regions, we can obtain the result at time [time]. Joint spatiotemporal characteristics:

[0181] (52);

[0182] In the formula, Use the Sigmoid activation function; Mapping to a multilayer perceptron; These are the neighborhood residual weighting coefficients constructed from the regional-level sensitivity. This is the residual compensation coefficient; It represents the joint spatiotemporal representation of all regions.

[0183] Step 3: Train and execute the PGMF-MADRL model;

[0184] The proposed PGMF-MADRL model employs a centralized training and distributed execution framework, and its model architecture is as follows: Figure 1 As shown. By Figure 1 It can be seen that each regional agent is configured with a local Actor to generate regional actions based on local historical information; in addition, all regions share a centralized Critic, which successively contains ASG-GAT, MS-Mamba, and SMFG, used to perform joint spatiotemporal representation of the joint historical input. From the above, the joint spatiotemporal representation obtained after SMFG fusion is: The combined action of the Actor output from each region is: The centralized Critic inputs both into the value assessment layer to obtain the combined action value:

[0185] (53);

[0186] In the formula, The action value function for a centralized Critic; These are the parameters for the Critic network. This is the value estimate of the joint action at time t.

[0187] The training and execution process of the proposed PGMF-MADRL model is as follows: Figure 2 As shown. During the training phase, the current state sample enters the Actor branch and the Critic branch respectively: the former generates joint actions that satisfy the device's feasible region constraints based on local historical inputs from agents in each region, while the latter sequentially completes spatial feature extraction, dual-timescale temporal modeling, and multi-scale fusion based on the joint historical inputs to form a joint spatiotemporal representation; subsequently, the joint actions and joint spatiotemporal representations are input together into the evaluation layer to output the global... Value. Based on this value estimate, environmental feedback provides immediate rewards. With the next state and form interactive samples The samples are stored in the experience replay pool. Once the number of samples reaches the training requirement, a small batch of samples is randomly selected from the experience replay pool for parameter updates. After training converges, the Actor parameters for each region are saved and distributed to the corresponding controllers. During the distributed execution phase, the centralized Critic is removed, and each region generates control actions independently based solely on local information.

[0188] This application also provides a computer device comprising: a memory and a processor, wherein the memory stores instructions executable on the processor. When the processor executes the instructions, it implements the methods described in the above embodiments. The number of memories and processors can be one or more. This computer device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The computer device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown in this application, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application as described and / or claimed herein.

[0189] The computer device may also include a communication interface for communicating with external devices and exchanging data. The devices are interconnected using different buses and can be mounted on a common motherboard or otherwise installed as needed. The processor processes instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The bus can be divided into address buses, data buses, control buses, etc.

[0190] Optionally, in a specific implementation, if the memory, processor, and communication interface are integrated on a single chip, then the memory, processor, and communication interface can communicate with each other through an internal interface.

[0191] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting advanced RISC machines (ARM) architecture.

[0192] This application provides a computer-readable storage medium (such as the memory described above) that stores computer instructions, which, when executed by a processor, implement the method provided in this application.

[0193] Optionally, the memory may include a stored program area and a stored data area, wherein the stored program area may store the operating system and application programs required for at least one function; the stored data area may store data created based on the use of the computer device for mapping. Furthermore, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the computer device for mapping via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A reactive power and voltage control method for distribution networks based on physical-guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning, characterized in that: This method constructs and trains a PGMF-MADRL model, and uses the trained PGMF-MADRL model for reactive power and voltage control in the distribution network. The method includes the following steps: Step 1: Model the zoned reactive power and voltage control, including: Step 1.1: Construct a regional voltage-reactive power sensitivity model; Step 1.2: Determine the voltage control target and the physical feasible region constraints of the equipment; Step 1.3: Modeling of Zoned Reactive Voltage Control (Dec-POMDP); Step 2: Construct the PGMF network, including: Step 2.1: Construct an adaptive sensitivity-guided graph attention network; Step 2.2: Construct a multi-scale selective Mamba state-space model; Step 2.3: Construct a sensitivity-guided multi-scale fusion gating system; Step 3: Train and execute the PGMF-MADRL model.

2. The reactive power and voltage control method for distribution networks based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning as described in claim 1, characterized in that: The regional voltage-reactive power sensitivity model in step 1.1 is built on the basis of the node-level sensitivity. First, the relationship between branch power and node voltage is characterized based on the radial distribution network branch power flow DistFlow model to obtain the node-level voltage-reactive power sensitivity matrix. Then, the mapping relationship between node reactive power injection and device reactive power command is established to obtain the device-level voltage-reactive power sensitivity matrix. Finally, the device-level voltage-reactive power sensitivity is mapped to the regional control scale to obtain the regional voltage-reactive power sensitivity matrix.

3. The reactive power and voltage control method for distribution networks based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning as described in claim 1, characterized in that: In step 1.2, a multi-time period voltage collaborative control model is constructed with the goal of minimizing node voltage deviation, voltage over-limit penalty, network loss and reactive power fluctuation. The physical feasible domain constraints of the equipment include reactive power regulation constraints of distributed photovoltaic or distributed wind power inverters at the access nodes and reactive power output constraints of static var compensators.

4. The reactive power and voltage control method for distribution networks based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning as described in claim 3, characterized in that: The Dec-POMDP constructed in step 1.3 is used to describe the distributed collaborative decision-making process of multiple agents under local observation. It includes an agent set, a regional interaction graph, a state space, an observation space, an action space, a state transition probability function, a reward function, a discount factor, and the time domain length of a single control task. By learning the optimal joint policy, each regional agent can reduce voltage deviation, limit violation, network loss, and action fluctuation while satisfying the physical feasible domain constraints of the device.

5. The reactive power and voltage control method for distribution networks based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning as described in claim 1, characterized in that: The PGMF network is embedded in the MADRL Critic network, while the Actor network adopts a simplified structure for online execution, generating regional reactive power control commands based solely on local observations and short-window historical information. The execution steps of the Actor network are as follows: First, local physical features on the Actor side are constructed and used as input for short-window temporal modeling. Then, short-window state space representations and actions on the Actor side are generated to characterize the short-term change trend of local states. Based on the local physical features input by the short window, dynamic features related to the generation of the current action are obtained and normalized. Finally, the normalized actions are mapped and projected onto the physical feasible region of the corresponding area.

6. The reactive power and voltage control method for distribution networks based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning according to claim 5, characterized in that: The adaptive sensitivity-guided graph attention network embeds the regional voltage-reactive sensitivity, which is recursively calibrated based on short-term operating measurements, into the attention weight calculation process as physical guidance information, so that the spatial feature aggregation can simultaneously reflect the data-driven state correlation and the reactive-voltage coupling relationship between regions. The steps of the regional voltage-reactive power sensitivity recursive calibration include: first, calculating the recursive gain vector, and then updating the regional voltage-reactive power sensitivity matrix according to the residual voltage response error. The residual voltage response error is the residual voltage response dominated by reactive power control action constructed by introducing the regional voltage-active power sensitivity approximation matrix.

7. The reactive power and voltage control method for distribution networks based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning according to claim 5, characterized in that: The multi-scale selective Mamba state-space model extracts high-frequency dynamic features and low-frequency trend features through fast and slow dual channels, respectively.

8. The reactive power and voltage control method for distribution networks based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning according to claim 7, characterized in that: Sensitivity-guided multi-scale fusion gating utilizes regional electrical coupling information to guide the fusion of fast and slow dual-channel features.

9. A method for reactive power and voltage control of a distribution network based on physically guided multi-scale spatiotemporal fusion multi-agent deep reinforcement learning according to any one of claims 1-8, characterized in that: The PGMF-MADRL model adopts a centralized training and distributed execution framework. Each region agent is configured with a local Actor network to generate region actions based on local historical information. All regions share a centralized Critic network, which successively includes an adaptive sensitivity-guided graph attention network, a multi-scale selective Mamba state space model, and a sensitivity-guided multi-scale fusion gating to perform joint spatiotemporal representation of joint historical input. The centralized Critic network inputs the joint spatiotemporal representation obtained after sensitivity-guided multi-scale fusion gating and the joint actions output by the Actor networks in each region into the value evaluation layer to obtain the joint action value. Based on joint action value estimation, the environment provides immediate rewards and the next state, and forms interactive samples that are stored in the experience replay pool. When the number of samples reaches the training requirements, a small batch of samples is randomly selected from the experience replay pool for parameter updates. After training converges, the parameters of the Actor network in each region are saved and sent to the corresponding controller. In the distributed execution phase, the centralized Critic network is removed, and each region generates control actions independently based solely on local information.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1-9.

Images