Low-voltage transformer area loss reduction method and system based on reinforcement learning and power flow screening

Abstract

A low-voltage transformer area collaborative loss reduction method and system based on reinforcement learning and power flow screening. It relates to the technical field of low-voltage distribution network operation control, and specifically relates to a low-voltage transformer area collaborative loss reduction method and system based on reinforcement learning and power flow screening. In view of the problems that the existing technology is still difficult to simultaneously consider the adaptability of the control strategy, the executability of the device action, and the safety and economy of the power distribution network operation under the complex and variable low-voltage transformer area operation scene, the first stage of the present application is a learning type suggestion stage: a DDPG intelligent agent trained by using the same state without control benchmark relative reward is used to output continuous suggestion actions of energy storage active and photovoltaic inverter reactive according to the transformer area operation state. The second stage is a physical verification decision stage: a candidate action set with device physical meaning is constructed around the suggestion action, the candidate action is calculated by power flow, hard constraint is screened out, and the actual physical index is scored, and the final executable action is selected.

Description

Technical Field

[0001] This invention relates to the field of low-voltage distribution network operation control technology, specifically to a method and system for collaborative loss reduction of low-voltage distribution areas based on reinforcement learning and power flow screening. Background Technology

[0002] With the widespread integration of distributed photovoltaic (PV) systems in rural and peri-urban low-voltage distribution areas, low-voltage distribution networks are gradually shifting from traditional unidirectional power reception to bidirectional fluctuating operation. Due to the temporal and spatial mismatch between peak PV output and peak loads from residential, commercial, and small industrial users, when PV output exceeds local absorption capacity, low-voltage feeders are prone to problems such as reverse power flow, node voltage rise, branch current redistribution, and line loss fluctuations. In severe cases, this can lead to PV power-limited operation, inverter shutdown, or increased heat generation in distribution equipment.

[0003] Low-voltage distribution areas are typically characterized by long line radii, numerous branches, dispersed nodes, and significant differences in grid connection conditions. Their operational status also continuously changes with photovoltaic penetration rate, grid connection location, load level, and weather conditions. Existing fixed-time control, preset rule control, or single-device adjustment methods are difficult to adapt to the changing operational scenarios, and are prone to problems such as unreasonable timing of energy storage charging and discharging, insufficient or excessive reactive power regulation, and the situation where local operating indicators improve while overall line loss or voltage status deteriorates.

[0004] While existing learning-based control methods can generate continuous control actions based on operating states, significant differences in baseline line loss levels and power flow characteristics across different operating scenarios can easily lead to inconsistent evaluations of control objectives, affecting the stability of strategy training and its cross-scenario adaptability. Furthermore, the control actions output by the learning model cannot guarantee that they will always satisfy power flow patterns and operational constraints such as voltage, energy storage state of charge, and equipment capacity. Direct execution of these actions may result in power flow calculation failures, node voltage exceeding limits, equipment capacity exceeding limits, or energy storage state of charge exceeding limits. Summary of the Invention

[0005] To address the challenge that existing technologies still struggle to simultaneously ensure the adaptability of control strategies, the feasibility of equipment actions, and the safety and economy of distribution network operation in complex and ever-changing low-voltage distribution area operating scenarios, this invention proposes a collaborative loss reduction method and system for low-voltage distribution areas based on reinforcement learning and power flow screening.

[0006] The method includes the following steps: S1. Construct a low-voltage distribution area model and obtain the operating status and scenario parameters of the low-voltage distribution area in the current training control cycle; S2. Construct a training state vector based on the parameters obtained in step S1, and input the training state vector into the DDPG agent to obtain the current training action; S3. Training the DDPG agent: For the same training state, apply uncontrolled actions and current training actions respectively in the low-voltage transformer area model, and perform training power flow calculations respectively. S4. Based on the results of the training power flow calculation, calculate the reward value of the pre-constructed reward function, and iteratively update the network parameters of the DDPG agent according to the reward value until the training termination condition is met, and obtain the trained DDPG agent. S5. Obtain the operating status and scenario parameters of the low-voltage distribution area in the current online control cycle, and construct the current online status vector; S6. Online generation of suggested actions: Input the current online state vector into the trained DDPG agent to obtain continuous suggested actions for the current online control cycle; S7. Based on the continuous suggested actions, construct a set of physical candidate actions including the energy storage active power candidate set and the inverter reactive power candidate set; S8. Under the same online state, apply each candidate action in the physical candidate action set to the low-voltage distribution area model, perform online power flow calculations, and then perform hard constraint screening based on the online power flow calculation results to obtain a set of feasible candidate actions. S9. Score the candidate actions in the feasible candidate action set by physical index, and select the candidate action with the best score as the optimal control action.

[0007] Furthermore, in step S1, constructing the low-voltage distribution area model includes: determining the electrical connection relationships between the upstream power grid, distribution transformers, low-voltage buses, low-voltage feeders and nodes based on the network topology of the low-voltage distribution area, and configuring the parameters of low-voltage lines, distribution transformers, loads, distributed photovoltaics, photovoltaic inverters and energy storage. The operating status includes: training control cycle number, active power of each load node, active power output of each photovoltaic access point, active power of the main transformer side, voltage of each node, current of each branch, and SOC of each energy storage. The scenario parameters include: photovoltaic penetration rate, photovoltaic access location, weather code, and load type.

[0008] Furthermore, train the state vector for:

[0009] in, For the first The state at each training moment of a control cycle For the first Total load for each control cycle For the first Total photovoltaic output per control cycle For the first Active power of the main transformer side in each control cycle; For the first The power fed back in one control cycle; For the first The lowest node voltage in each control cycle For the first The highest voltage at each node during each control cycle. For the first Line loss per control cycle For the first Energy storage SOC per control cycle, For photovoltaic penetration rate, For the first Weather codes for each control cycle.

[0010] Furthermore, the current training movements are as follows: ,in, For the index value of the training control cycle, For the first The stored active power of each training control cycle For the first Inverter reactive power during each training control cycle; no control action. .

[0011] Furthermore, the pre-constructed reward function is as follows:

[0012] in, For the first The reward value for each training control cycle. For the first After executing uncontrolled actions in each training control cycle, the baseline line loss is calculated based on the branch current and line resistance obtained from the power flow calculation. The current circuit loss after performing the current training action; For relative loss reduction profit weighting, This is the current line loss weight. , , and The first Voltage over-limit penalty, reverse power penalty, SOC over-limit penalty, and action change penalty for each training control cycle. To prevent positive numbers with a denominator of zero.

[0013] Furthermore, the current online state vector adopts the same state component arrangement order, state quantity calculation method, benchmark value and normalization parameter as the training state vector, and is constructed from the running state and corresponding scenario parameters obtained in real time during the current online control cycle; Continuous recommended actions during the current online control cycle Specifically: ,in, This is the index value for the online control cycle. Let be the recommended value for the active power of energy storage in the t-th online control cycle. Here is the recommended value for the inverter reactive power in the t-th online control cycle; The candidate set of active power for energy storage includes: zero active power, active power for energy storage determined according to preset fixed control rules, and... The preset proportional power in the energy storage charging direction and the preset proportional power in the energy storage discharging direction; the preset proportional power is the negative proportional value of the current maximum allowable charging power and the positive proportional value of the current maximum allowable discharging power, respectively. The candidate set of total reactive power for the inverter includes: the total reactive power of the inverter determined according to a preset fixed control rule. Zero reactive power and multiple positive or negative proportional values ​​determined according to the total available reactive power of the inverter; The candidate values ​​in the active power candidate set of the energy storage are combined with the candidate values ​​in the reactive power candidate set of the inverter, and duplicate candidate actions are merged to obtain a physical candidate action set. The maximum available reactive power of each inverter is determined based on its rated apparent power and current photovoltaic active power output. The maximum available reactive power of each inverter is then summed to obtain the range of total available reactive power of the inverter. The set of physical candidate actions and the total available reactive power range of the inverter together constitute the set of physical candidate actions.

[0014] Furthermore, the hard constraint screening includes: when the power flow calculation corresponding to the candidate action does not converge, or the active power of the energy storage exceeds the current allowable charging and discharging power range, or the total reactive power of the inverter exceeds the total available reactive power range of the inverter, or the energy storage SOC after the candidate action is executed exceeds the preset upper and lower limits, or the node voltage exceeds the preset hard constraint threshold, the candidate action is determined as an infeasible candidate action and is removed. The candidate actions that were not eliminated are identified as feasible candidate actions, forming a set of feasible candidate actions.

[0015] Furthermore, the formula for calculating the physical index score is as follows:

[0016] in, The score is for physical indicators. Candidate actions for which scores are to be calculated. Candidate actions The normalized line loss value, Candidate actions The node voltage exceeds the limit value, Candidate actions The normalized backfeed power value, Candidate actions SOC security margin penalty value, For the action of the previous control cycle, , , , and This represents the weight of the corresponding scoring item.

[0017] Furthermore, when the set of feasible candidate actions is empty, the amplitude of the energy storage active power candidate value and the inverter reactive power candidate value is reduced according to a preset scaling factor, and the power flow calculation and hard constraint screening are re-executed; if no feasible candidate actions are found after re-screening, the feasible actions of the previous online control cycle are limited to the current allowable power range of the equipment to form a safe backup action; when no feasible actions of the previous online control cycle are found, the energy storage active power and the inverter reactive power are both set to zero and an abnormal alarm is output.

[0018] A low-voltage distribution area collaborative loss reduction system based on reinforcement learning and power flow screening, the system being used to implement the above method, the system comprising: The training action generation module is used to generate a training state vector based on the training running state and scene parameters, and to obtain the current training action; The DDPG agent training module is used to update the DDPG agent based on the power flow calculation results of the uncontrolled action and the current training action, so as to obtain the trained DDPG agent. The online action suggestion generation module is used to generate an online state vector based on the online running status and scene parameters, and obtain continuous action suggestions. The candidate action construction module is used to construct a set of physical candidate actions based on the continuous suggested actions; The feasible action filtering module is used to perform power flow calculation and hard constraint filtering on the set of physical candidate actions to obtain a set of feasible candidate actions. The optimal control action determination module is used to score the physical indicators of feasible candidate actions and determine the optimal control action. The safety backup action generation module is used to narrow down the amplitude of candidate actions and re-filter them when no feasible candidate actions exist, or to generate safety backup actions based on the feasible actions of the previous online control cycle; when no feasible actions of the previous online control cycle exist, it outputs zero-power actions and abnormal alarms.

[0019] The beneficial effects of the method described in this invention are as follows: (1) In each training state, the present invention maintains consistent operating conditions such as load, photovoltaic output, and energy storage charge state. Power flow calculations are performed for both uncontrolled actions and the current training actions, and the normalized result of the line loss difference between the two is used to characterize the loss reduction contribution of the current training action. This reduces the impact of differences in natural line loss scale under different photovoltaic penetration rates, access locations, and weather scenarios on the reward value, enabling the DDPG agent to have relatively consistent learning objectives under different operating scenarios, thereby improving the stability of the training process and its cross-scenario adaptability.

[0020] (2) In this invention, the active power of energy storage and the reactive power of inverter output by the trained DDPG agent are used as continuous suggested actions, rather than directly as equipment execution instructions, thus separating the learning decision-making from the physical safety verification. The DDPG agent is used to provide coordinated adjustment directions that conform to the current operating state, and the physical verification process is used to determine whether the suggested actions and their extended actions meet the equipment capacity and safe operation requirements of the transformer area, thereby reducing the risks of power flow non-convergence, equipment exceeding limits, and energy storage state of charge exceeding limits caused by the direct execution of neural network output actions.

[0021] (3) This invention constructs a set of physical candidate actions based on uncontrolled actions, fixed rule actions, DDPG suggested actions, and proportional actions determined according to the allowable charging and discharging power of energy storage and the real-time available reactive power of the inverter. The candidate actions have clear meanings of energy storage charging and discharging and inverter reactive power regulation, and can take into account existing safety control strategies, learning model output results, and actual equipment regulation capabilities, avoiding insufficient physical executability of candidate actions caused by selecting candidate actions solely based on the mathematical distance between action vectors.

[0022] (4) This invention enables each candidate action to start from the same current operating state of the transformer area and the initial state of charge of the energy storage, and to perform independent power flow calculations. Infeasible candidate actions are eliminated based on power flow convergence, energy storage state of charge, and equipment capacity constraints. For the remaining feasible candidate actions, they are evaluated based on the line loss, node voltage, backfeed power, energy storage safety margin, and action change obtained from actual calculations. As a result, the final action is determined by the actual physical operation results corresponding to the candidate action, rather than solely by the prediction value of the Critic network. This improves the executability of the final control action and the reliability of loss reduction evaluation, while taking into account voltage, backfeed power, and energy storage operation safety.

[0023] (5) When no feasible candidate action exists in the physical candidate action set, the present invention first reduces the amplitude of the candidate action according to the preset scaling factor and re-screens it; if no feasible candidate action still exists after re-screening, the feasible action of the previous online control cycle is limited to the current allowable power range of the equipment as a safety backup action; when no feasible action exists in the previous online control cycle, the active power of the energy storage and the reactive power of the inverter are both set to zero and an abnormal alarm is output. After executing the final action, the line loss, node voltage, reverse power and energy storage state of charge are updated, and the next control cycle is entered, thereby forming a rolling closed-loop control process with safety backup capability, improving the continuity and fault safety of the control process under complex operating scenarios. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method described in this invention; Figure 2 This is a diagram of the collaborative control architecture for DDPG and power flow security screening as described in this invention; Figure 3 This is a diagram of the source-grid-load-storage collaborative control system architecture described in this invention; Figure 4 This is a schematic diagram of the topology and control resource layout of the high-penetration distributed photovoltaic low-voltage distribution area described in this invention. Detailed Implementation

[0025] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1 This embodiment provides a collaborative loss reduction method for low-voltage distribution areas based on reinforcement learning and power flow screening. The flowchart of the method is as follows: Figure 1 As shown, the method includes the following steps: S1. Construct a low-voltage distribution area model and obtain the operating status and scenario parameters of the low-voltage distribution area in the current training control cycle; The content described in step S1 will be explained using specific embodiments: When constructing a power flow calculation model for a low-voltage distribution area, the electrical connection relationships between the upstream power grid, distribution transformers, low-voltage buses, low-voltage feeders, and feeder nodes are first determined based on the network topology of the low-voltage distribution area.

[0027] Based on the length, resistance, reactance, and allowable current carrying capacity of each low-voltage line segment, set the parameters for the corresponding low-voltage line. The parameters of the distribution transformer are set according to its turns ratio, rated capacity, short-circuit voltage percentage, short-circuit resistance voltage percentage, no-load loss, and no-load current.

[0028] Configure load parameters, distributed photovoltaic parameters, and photovoltaic inverter parameters at each low-voltage feeder node, and configure one or more energy storage units at the feeder end or other pre-set nodes. Number the energy storage units within the low-voltage distribution area sequentially from unit 1 to unit 2. Taiwan's energy storage, among which, This indicates the total number of energy storage units and sets the electrical connection relationship between each energy storage unit and its corresponding feeder node.

[0029] Based on the load base capacity, the time sequence coefficient and power factor corresponding to the load type of each node, determine the node load power for each control cycle; The total installed capacity of distributed photovoltaic (PV) is determined based on the PV penetration rate. Distributed PV is configured to the corresponding feeder nodes according to the PV access location. The active power output of PV in each control cycle is determined based on the PV output curve corresponding to the weather scenario.

[0030] No. The parameters of Taiwan's energy storage include rated energy. Rated maximum charging power Rated maximum discharge power Charging efficiency Discharge efficiency Initial state of charge and upper and lower limits of state of charge and .

[0031] Based on the above electrical connections and parameters of low-voltage lines, distribution transformers, loads, distributed photovoltaics, photovoltaic inverters, and energy storage, a power flow calculation model for the low-voltage distribution area is formed for subsequent power flow calculations and control action evaluation.

[0032] The operating status obtained in this step includes: the current training control cycle number, the active power of each load node, the active power output of each photovoltaic access point, the active power of the low-voltage side of the main transformer, the voltage of each node, the current of each branch or the active power of the first and last ends of each branch, and the SOC of each energy storage. The scenario parameters include: photovoltaic penetration rate, photovoltaic access location, weather code, and load type. In online applications, the above operating states are obtained from the intelligent integrated terminal of the distribution area, inverter interface, energy storage management system, distribution transformer monitoring device, and smart meter; in simulations, they are obtained from the constructed low-voltage distribution area model and preset scenarios.

[0033] S2. Construct a training state vector based on the parameters obtained in step S1, and input the training state vector into the DDPG agent to obtain the current training action; The content described in step S2 will be explained using specific embodiments: The training state vector includes 11 state components: time, total load, total photovoltaic output, active power on the main transformer side, reverse power transmission, minimum node voltage, maximum node voltage, current line loss, energy storage SOC, photovoltaic penetration rate, and weather code. These components are arranged as follows:

[0034] Moment State It is obtained from the training control cycle number or timestamp; in a 24-hour scenario with a step size of 1 hour, the hour number from 0 to 23 can be used and divided by 23 to obtain the normalized time.

[0035] Total load ,in For the first The load node at the ... Active power per training control cycle This represents the total number of load nodes; the load type is used to generate the corresponding intraday load time sequence.

[0036] Total photovoltaic output ,in, Indicates the first Within the first control cycle, the first Each photovoltaic access point corresponds to the photovoltaic active power output (power) of the photovoltaic inverter during online operation. By the Each photovoltaic access point corresponds to a photovoltaic inverter interface or simulation scenario. This represents the total number of photovoltaic (PV) connection points.

[0037] Active power on the main transformer side It is measured directly by the distribution transformer monitoring device or obtained by power flow calculation; a positive value indicates that the transformer area receives power from the upper-level power grid, and a negative value indicates that power is fed back to the upper-level power grid.

[0038] The reverse power is .

[0039] No. The minimum and maximum node voltages for each training control cycle are respectively: and The node voltages are expressed in per-unit values. For the first Within the first control cycle, the first Voltage of each load node.

[0040] The current line loss is calculated based on the branch currents and line resistances obtained from power flow calculations: For the first Line loss per control cycle , The index value of the branch. The total number of branch roads, For the first Within the first control cycle, the first The current in the branch circuit, For the first The resistance of the branch circuit, For the first Within the first control cycle, the first The active power loss of each branch can be calculated; alternatively, the difference between the active power at the beginning and the active power at the end of each branch obtained from power flow calculation can be summed.

[0041] Indicates the first The aggregated state of charge (SOC) of multiple energy storage devices within a control cycle is calculated as follows: when only one energy storage device is installed, its real-time SOC is directly used; when multiple energy storage devices are installed and aggregated state is used, the calculation is based on a weighted average of rated energy. , This is the index value for the energy storage device. This represents the total number of energy storage devices. For the first The rated energy of the Taiwan energy storage device For the first Taiwan energy storage device in the The real-time state of charge (SOC) of each control cycle; in another embodiment, when each energy storage is controlled independently, each energy storage SOC can be used as a state component, and the dimension of the state vector can be adjusted accordingly.

[0042] Photovoltaic penetration rate , This is the sum of the rated active power capacity of all distributed photovoltaic systems within the low-voltage distribution area. This represents the maximum active load of the low-voltage distribution area within a preset statistical period.

[0043] No. Weather codes for each control cycle Based on meteorological measurement or forecast data, irradiance characteristics, or simulated scene labels, this embodiment encodes sunny days, cloudy days, and rainy days as 0, 1, and 2, respectively.

[0044] Network topology, line parameters, transformer parameters, load type, photovoltaic access location, and rated parameters of energy storage and inverters are used as environmental static parameters for power flow calculation, state variable aggregation, dimensionless conversion, and operational constraint verification. They are not required to be used as dynamic input components of DDPG.

[0045] Before inputting the training state vector into the Actor network of the DDPG agent, the state variables in the training state vector are dimensionless to reduce the impact of differences in the dimensions and numerical ranges of different state variables on network training and online inference.

[0046] This embodiment uses a unified reference power for the transformer substation. right , , , and Perform normalization; The active power baseline value corresponding to the rated capacity of the distribution transformer can be used. Node voltages are per-unit values, State of Charge (SOC) is a value within the range of 0 to 1, time is divided by the maximum control cycle number of the day, photovoltaic penetration rate is divided by the preset maximum penetration rate, and weather code is divided by the maximum code value. The same state variable arrangement order, baseline value, and normalization parameters are used in both the training and online phases.

[0047] The dimensionless training state vector is input into the Actor network of the DDPG agent. The Actor network generates continuous action outputs based on the current training state, including the energy storage active power control component and the inverter reactive power control component, which are used to characterize the energy storage charging and discharging adjustment direction and the inverter reactive power adjustment direction within the current training control cycle.

[0048] Based on the preset action range, each action component output by the Actor network is mapped to the allowable power range of the corresponding device to obtain the current training action: ,in, For the index value of the training control cycle, For the first The stored active power of each training control cycle For the first The inverter reactive power of each training control cycle.

[0049] A positive value for the active power of energy storage indicates energy storage discharge, while a negative value indicates energy storage charging. The positive or negative sign of the total reactive power of the inverter indicates whether the inverter injects reactive power into the low-voltage distribution area or absorbs reactive power.

[0050] S3. Training the DDPG agent: For the same training state, apply uncontrolled actions and current training actions respectively in the low-voltage transformer area model, and perform training power flow calculations respectively. The content described in step S3 will be explained using specific embodiments: For the In each training control cycle, a no-control branch and an action-controlled branch are set up under the same training state. The same training state means that the two branches use the same initial values ​​of load, photovoltaic active power output, and energy storage state of charge, and only the applied control actions are different, so as to eliminate the influence of differences in external operating conditions on the power flow calculation results.

[0051] When training power flow calculations are performed separately, the calculations are independent of each other and both start from the same training state. Therefore, the difference between the results of the two training power flow calculations is used to characterize the impact of the current training action on the operating state of the low-voltage transformer area.

[0052] In the uncontrolled branch, both the active power of the energy storage and the reactive power of the inverter are set to zero, resulting in uncontrolled operation. ; The uncontrolled action is applied to the low-voltage distribution area model, and power flow calculations are performed to obtain the baseline line loss under the uncontrolled state. .

[0053] In the action control branch, the current training action output by the DDPG agent in the current training state is displayed. By applying the power flow calculation to the low-voltage distribution area model, the current line loss corresponding to the current training action is obtained. The system also collects the node voltage, active power on the main transformer side, and energy storage state of charge (SOC) operation results. Based on these results, it determines the voltage overrun, reverse power, and SOC outrun conditions corresponding to the current training action. Furthermore, it calculates the reward value for the current training control cycle by considering the changes in the current training action relative to the previous training control cycle.

[0054] S4. Based on the results of training power flow calculation, calculate the reward value of the pre-constructed reward function, and iteratively update the network parameters of the DDPG agent according to the reward value until the training termination condition is met, thus obtaining the trained DDPG agent.

[0055] The content of step S4 will be described using a specific embodiment: The pre-built reward function is as follows:

[0056] in, For the first The reward value for each training control cycle. Power flow calculation first obtains the current of each branch, node voltage, and main transformer-side power; the baseline line loss and the current line loss are respectively calculated according to... as well as Alternatively, the active power losses of the equivalent branches can be directly read from the power flow program output and summed. and They represent the first Within the control cycle, after executing the uncontrolled action and the current training action, the result is calculated based on the power flow. Branch current, For relative loss reduction profit weighting, This is the current line loss weight. To prevent positive numbers with a denominator of zero, this embodiment can take... .

[0057] No. The voltage over-limit penalty for each training control cycle can be expressed as: , This represents the weighting coefficient for voltage over-limit penalty. and These represent the upper and lower limits of the node voltage, respectively. When the node voltage is at... When the voltage is within the specified range, the penalty for that node is zero; when the node voltage exceeds the upper limit or falls below the lower limit, the penalty is calculated based on the square of the amount exceeding the limit. The larger the value, the stronger the suppression of voltage limits during DDPG training; No. The feedback power penalty for each training control cycle can be expressed as: , This is the reverse power penalty factor.

[0058] No. The SOC out-of-bounds penalty for each training control cycle can be expressed as: , This represents the SOC (System Oscillator) out-of-bounds penalty coefficient. Indicates execution of the first The energy storage state of charge for the next training control cycle is calculated after the current training action in the previous training control cycle. and These represent the lower limit and upper limit of the energy storage state of charge, respectively. This indicates the out-of-bounds amount when the SOC is below the lower limit. This indicates the out-of-bounds amount when the SOC exceeds the upper limit; No. The penalty for action changes in a training control cycle can be represented as: ,in The action is normalized to the maximum power of the equipment, i.e., the first... Normalized actions for each training control cycle Indicates the first Normalized actions for each training control cycle , For the maximum allowable power of energy storage, For the first The current maximum available reactive power of the inverter in each training control cycle. This represents the penalty coefficient for changes in action.

[0059] All weights are non-negative and can be determined through grid search based on the loss reduction effect and constraint satisfaction in the verification scenario. A set of exemplary values ​​is as follows: , , , , and The values ​​mentioned above are merely examples and do not constitute a limitation.

[0060] When the current line loss is less than the baseline line loss, the relative loss reduction benefit term is positive, representing the loss reduction effect of the current training action compared to the uncontrolled action. When the current line loss is greater than the baseline line loss, the relative loss reduction benefit term is negative, used to suppress training actions that increase line loss. The baseline line loss is used to normalize the line loss difference, ensuring that the reward value reflects the degree of line loss improvement of the current training action compared to the uncontrolled state of the current operating scenario, reducing the impact of the absolute scale difference in line loss under different load levels, photovoltaic output, and energy storage charging states on the training objective.

[0061] The uncontrolled branch and the action control branch are independent of each other, both starting from the current training state. The uncontrolled branch is only used to obtain the baseline line loss corresponding to the current training action, and its power flow calculation results are not used to update the energy storage state of charge, nor are they used as the initial state for the next training control cycle; the operation results corresponding to the action control branch are used to calculate the reward value and determine the next training state.

[0062] The Actor, Critic, and corresponding target network parameters of the DDPG agent are updated based on the calculated reward value, and the next training control cycle begins. Each training round contains 24 control cycles; in this embodiment, a cumulative training step count of 3000 steps is used as the training termination condition, corresponding to approximately 125 typical daily training rounds. An early termination condition can also be set: the average cumulative reward relative change rate over 20 consecutive training rounds is less than... Training is stopped when the condition is met. The above training process is repeated until any training termination condition is met, resulting in a fully trained DDPG agent.

[0063] like Figure 2As shown, the trained DDPG agent outputs continuous suggested actions based on the current online state during the online control phase. These continuous suggested actions are not directly executed but instead constitute a physical candidate action set together with no-control actions, fixed-rule actions, and candidate actions determined based on the actual power range of the energy storage and inverter. Power flow calculations are performed on each candidate action in the physical candidate action set, and hard constraint screening and physical index scoring are performed based on the power flow calculation results to ultimately determine the control action for the current online control cycle.

[0064] S5. Obtain the operating status and scenario parameters of the low-voltage distribution area in the current online control cycle, and construct the current online status vector; The content of step S5 will be described using specific embodiments: The current online state vector uses the same 11 state components, arrangement order, state quantity calculation method, reference value, and normalization parameters as the training state vector. The load node power, photovoltaic output, main transformer side active power, node voltage, branch current, and energy storage SOC for the current online control cycle are obtained from online measurements. Photovoltaic penetration rate and weather codes are obtained from equipment ledgers, scenario configurations, or meteorological interfaces. Photovoltaic access location and load type are used as static environmental parameters, obtained from equipment ledgers or scenario configurations, for model configuration, state quantity calculation, and operational constraint verification; they are not considered as dynamic state components in this embodiment.

[0065] Construct the current online state vector from the above online data: , The dimensionless transformation is performed in the manner described in step S2, and the result is used as the output of step S5 and the input of step S6.

[0066] S6. Generate suggested actions online: Input the current online state vector into the trained DDPG agent to obtain continuous suggested actions for the current online control cycle.

[0067] The content of step S6 will be described using a specific embodiment: The current online status vector The trained DDPG agent is input into the Actor network, which generates continuous suggested actions for the current online control cycle based on the current online state. Specifically: ,in, This is the index value for the online control cycle; For the first The recommended value of the active power of energy storage for each online control cycle is used to characterize the direction and amount of energy storage charging or discharging regulation. For the first The recommended value of the total reactive power of the inverter for each online control cycle is used to characterize the direction and amount of reactive power regulation of the inverter.

[0068] The continuously suggested actions serve only as a basis for constructing candidate actions for the current online control cycle and are not directly used as control commands issued to the energy storage and inverter. Subsequently, a set of physical candidate actions is constructed based on the continuously suggested actions, and the optimal control action for the current online control cycle is determined through power flow calculation, hard constraint screening, and physical index scoring.

[0069] S7. Based on the continuous suggested actions, construct a set of physical candidate actions including the energy storage active power candidate set and the inverter reactive power candidate set; The content of step S7 will be described using a specific embodiment: Within the allowable charging and discharging power range of the energy storage and the total available reactive power range of the inverter in the current online control cycle, candidate sets of active power for energy storage and candidate sets of total reactive power for the inverter are constructed respectively.

[0070] First, determine the currently available reactive power of the inverter. The rated apparent power of the inverter The active power output of photovoltaic power during the current online control cycle can be read from the inverter nameplate, equipment ledger, or inverter controller parameters. The data is acquired in real time via the inverter interface and is given by the photovoltaic output scenario during simulation. The current maximum available reactive power of each inverter is .

[0071] The total available reactive power of the inverters in the current online control cycle is obtained by summing the current maximum available reactive power of each inverter. .

[0072] When the controller outputs the total reactive power command of the inverter, the total reactive power command is distributed to each inverter according to the ratio of the current maximum available reactive power of each inverter to the total available reactive power of the inverter; when the total available reactive power of the inverter is zero, the reactive power of each inverter is set to zero.

[0073] Based on the first step determined when constructing the energy storage model in step S1 Taiwan's rated energy storage Rated maximum charging power Rated maximum discharge power Charging efficiency Discharge efficiency State of charge limit and upper limit of state of charge Combined with the current online control cycle obtained in step S5 Taiwan Energy Storage Status of Charge and the preset control cycle duration Calculate the first one respectively Taiwan Energy Storage's maximum allowable charging power and maximum allowable discharging power during the current online control cycle:

[0074]

[0075] When multiple energy storage units are given a unified total power command, the allowable charging and discharging powers of each unit are summed to obtain the current allowable charging and discharging power of the distribution area. In this embodiment, 50% of the current allowable charging and discharging power of the distribution area is used as the preset proportional power for the charging and discharging directions, respectively. and .

[0076] In this embodiment, a control cycle of 1 hour is used, and the preset control cycle length is [not specified]. The time interval is set to 1 hour. The method described in this invention acquires the operating status of the low-voltage distribution area every hour, generates and evaluates candidate actions for the current control cycle, and determines the control commands for energy storage active power and inverter reactive power.

[0077] The preset fixed control rule in this embodiment is as follows: when the total photovoltaic output is greater than the total load and the SOC is lower than the upper limit, the energy storage charges at a power not exceeding 50% of the current maximum allowable charging power; when the total load is greater than the total photovoltaic output and the SOC is higher than the lower limit, the energy storage discharges at a power not exceeding 50% of the current maximum allowable discharge power; at other times, the active power of the energy storage is zero. The corresponding fixed rule energy storage active power is denoted as... Reactive power of fixed-rule inverters In this embodiment, zero is taken. According to... , The current allowed charging and discharging power of energy storage is used to construct a candidate set of active power for energy storage.

[0078] Specifically, when the charging conditions are met, the active power of the fixed-rule energy storage is the negative value of the smaller of the difference between the total photovoltaic output and the total load and 50% of the current allowable charging power of the transformer area; when the discharging conditions are met, the active power of the fixed-rule energy storage is the positive value of the smaller of the difference between the total load and the total photovoltaic output and 50% of the current allowable discharging power of the transformer area; when the above charging or discharging conditions are not met, the active power of the fixed-rule energy storage is zero.

[0079] The candidate set of active power for energy storage includes zero active power, fixed-rule active power for energy storage, suggested values ​​for active power for energy storage, preset proportional power in the direction of energy storage charging, and preset proportional power in the direction of energy storage discharging.

[0080] In one specific implementation, the energy storage active power candidate set Represented as: ; in, The active power of energy storage is determined according to preset fixed control rules. Recommended values ​​for the active power output of the DDPG agent for energy storage; negative power indicates energy storage charging, and positive power indicates energy storage discharging.

[0081] Based on the total available reactive power of the inverter Reactive power of fixed-rule inverters and DDPG recommended values Construct a candidate set of reactive power for the inverter.

[0082] The inverter reactive power candidate set includes fixed-rule inverter reactive power, suggested total inverter reactive power, zero reactive power, and multiple positive or negative proportional powers determined according to the total available reactive power of the inverter.

[0083] In one specific implementation, the inverter reactive power candidate set Represented as: ; in, In this embodiment, the total reactive power of the inverter, determined according to a preset fixed control rule, is set to zero. Recommended total reactive power output of the inverter for the DDPG agent.

[0084] Candidate set of active power for energy storage The candidate values ​​and the candidate set of reactive power of the inverter. The candidate values ​​in the data are combined using Cartesian products to form multiple candidate actions; each candidate action includes a candidate value of the energy storage active power. A candidate value for inverter reactive power Candidate actions with identical content are merged to obtain the physical candidate action set for the current online control cycle.

[0085] Among them, zero active power in the candidate set of energy storage active power and zero reactive power in the candidate set of inverter reactive power combine to form uncontrolled action; fixed rule energy storage active power and fixed rule inverter reactive power combine to form fixed rule action.

[0086] The aforementioned candidate ratio values ​​and the number of candidate ratio values ​​can be set according to the available power range of the energy storage and inverter, as well as the power flow calculation of subsequent candidate actions.

[0087] S8. Under the same online state, apply each candidate action in the physical candidate action set to the low-voltage distribution area model, perform online power flow calculations, and then perform hard constraint screening based on the online power flow calculation results to obtain a set of feasible candidate actions. The content of step S8 will be described using a specific embodiment: For each candidate action in the physical candidate action set, the operating state of the low-voltage distribution area in the current online control cycle is copied to obtain a temporary state copy corresponding to each candidate action. Each temporary state copy uses the same load, photovoltaic active power output, network parameters, and initial values ​​of energy storage state of charge, so that each candidate action is evaluated from the same online state.

[0088] The candidate values ​​of active power from the energy storage and reactive power from the inverter are written into the corresponding temporary state replicas. Based on the candidate values ​​of active power from the energy storage, the duration of the online control cycle, and the energy storage charging and discharging efficiency, the state of charge of the energy storage after the execution of the candidate action is calculated. At the same time, an online power flow calculation is performed in the low-voltage distribution area model after the candidate action is written to obtain the power flow convergence state and node voltage corresponding to the candidate action.

[0089] If positive values ​​of energy storage active power represent discharging and negative values ​​represent charging, then the first... The State of Charge (SOC) after the execution of the candidate action for Taiwan's energy storage system can be calculated according to the following segmented relationship:

[0090] in, Indicates the first The online control cycle is assigned to the first The active power of the energy storage unit is allocated by the total active power instruction of the energy storage area according to the proportion of the current allowed charging power or allowed discharging power of each energy storage unit.

[0091] The above segmented relationship is used to calculate the number of segments after one online control cycle of a candidate action, based on the stored active power in the candidate action. The state of charge (SBC) of the energy storage is used to determine whether the candidate action would cause the SBC to exceed a preset range. This calculation result is only used for candidate action evaluation and hard constraint screening; the actual SBC of the energy storage is not updated until the candidate action is finally determined.

[0092] The evaluation processes for each candidate action are independent of each other; the power flow calculation results and energy storage state of charge changes of the previous candidate action are not propagated to the subsequent candidate action. The evaluation of each candidate action is only performed in the corresponding temporary state replica and does not change the actual operating state of the low-voltage distribution area or the actual state of charge of the energy storage.

[0093] Based on the power flow calculation results, candidate action values, and the state of charge (SOC) of the energy storage after the execution of the candidate action, hard constraints are applied to the candidate actions. A candidate action is deemed infeasible and eliminated if any of the following conditions exist: power flow calculation fails to converge; the active power of the energy storage exceeds the current allowable charging and discharging power range; the total reactive power of the inverter exceeds the total available reactive power range of the inverter; the SOC of the energy storage after the execution of the candidate action exceeds the preset SOC range (upper and lower limits); or the node voltage exceeds the preset hard constraint threshold.

[0094] When the node voltage corresponding to a candidate action exceeds the preset normal voltage range but does not exceed the preset hard constraint threshold, the candidate action is not directly eliminated. Instead, the candidate action is retained, and a corresponding voltage limit violation penalty is set in the subsequent physical index scoring based on the degree of node voltage exceeding the limit.

[0095] Candidate actions that were not eliminated are identified as feasible candidate actions, and each feasible candidate action is combined into a set of feasible candidate actions.

[0096] S9. Score the candidate actions in the feasible candidate action set by physical index, and select the candidate action with the best score as the optimal control action.

[0097] The content of step S9 will be described using a specific embodiment: For each feasible candidate action in the set of feasible candidate actions, a comprehensive score is calculated based on the online power flow calculation results in step S8 and the energy storage state of charge after the candidate action is executed. One implementation of the comprehensive score is as follows:

[0098] in, Candidate actions The overall score is calculated, with smaller values ​​indicating better candidate actions. Normalized line loss is defined as... ,in For line losses without control actions in the same online state, This refers to the line loss when performing candidate actions under the same online state.

[0099] Node voltage exceeding the limit is defined as , This represents the total number of nodes participating in the voltage over-limit evaluation. For the node index value participating in the voltage over-limit evaluation, Indicates the application of candidate actions Then, the first result obtained through power flow calculation The voltage of each node; Normalized backfeed power is defined as , Indicates the application of candidate actions Then, based on the power flow calculation, the active power transmitted from the low-voltage distribution area to the upper-level power grid in the reverse direction.

[0100] SOC security margin penalty is defined as , Indicates the first Apply candidate actions in each online control cycle. Then, based on the current energy storage state of charge, the active power of the energy storage in the candidate actions, the control cycle duration, and the energy storage charge and discharge efficiency, the first... The state of charge of energy storage during each online control cycle. As a safety margin penalty coefficient, this embodiment can take... The motion variation items use normalized motion. .

[0101] All scoring weights are non-negative and are determined through a grid search by comparing loss reduction effect, voltage over-limit, reverse power, and SOC safety in a preset verification scenario. A set of exemplary values ​​is provided. , , , and Among them, the line loss item is used to prioritize reducing line losses, the voltage item and the SOC item are used to improve safety, the backfeed item is used to suppress backfeeding to the upper-level power grid, and the action change item is used to reduce abrupt changes in control commands; the above values ​​are only examples and do not constitute limitations.

[0102] The comprehensive score of each feasible candidate action is calculated, and the feasible candidate action with the lowest comprehensive score is determined as the optimal control action for the current online control cycle. The feasibility of the candidate actions and the final selection result are determined based on the actual power flow calculation results and energy storage state of charge corresponding to the candidate actions. The value estimation of the Critic network is not used as the basis for judging the safety of online candidate actions or the final action selection.

[0103] When the set of feasible candidate actions is not empty, the optimal control action determined above shall be taken as the final action of the current online control cycle.

[0104] When the set of feasible candidate actions is empty, the amplitude of the candidate values ​​for active power of energy storage and reactive power of inverter is first reduced according to a preset scaling factor, and the candidate action evaluation and screening are re-executed. The preset scaling factor is a number greater than 0 and less than 1, and in this embodiment, it is taken as 0.5, that is, the original candidate values ​​for active power of energy storage and reactive power of inverter are reduced to 50% of their original amplitudes. If no feasible candidate actions are found after re-screening, the feasible actions of the previous online control cycle are limited to the allowable power range of the current equipment as a safety backup action; when no feasible actions of the previous online control cycle are found, the active power of energy storage and the reactive power of inverter are both set to zero, and the controller outputs an abnormal alarm.

[0105] After determining the final action of the current online control cycle, the actual state of charge of the energy storage is updated according to the final action, and the line loss, node voltage, active power on the main transformer side and reverse power of the current online control cycle are recorded as the basic data for constructing the online state vector in the next online control cycle.

[0106] Example 2 This embodiment further defines Embodiment 1. This embodiment provides a low-voltage distribution area collaborative loss reduction system based on reinforcement learning and power flow screening, used to implement the method described in Embodiment 1. The system includes: The training action construction module is used to construct a low-voltage distribution area model, obtain the low-voltage distribution area operation status and scenario parameters in the current training control cycle, construct a training state vector, and input the training state vector into the DDPG agent to obtain the current training action. The DDPG agent training module is used to apply uncontrolled actions and current training actions to the low-voltage distribution area model for the same training state and perform training power flow calculations respectively. It is also used to calculate the reward value of a pre-built reward function based on the results of the training power flow calculation, and iteratively update the network parameters of the DDPG agent according to the reward value until the training termination condition is met, thus obtaining a trained DDPG agent. The online action suggestion generation module is used to obtain the low-voltage transformer area operating status and scenario parameters of the current online control cycle and construct the current online state vector. It is also used to input the current online state vector into the trained DDPG agent to obtain the continuous action suggestions for the current online control cycle. The physical candidate action generation module is used to construct a set of physical candidate actions, including a set of active power candidates for energy storage and a set of reactive power candidates for inverters, based on the continuous suggested actions. The feasible candidate action generation module is used to apply each candidate action in the physical candidate action set to the low-voltage area model under the same online state and perform online power flow calculations respectively. Then, hard constraint screening is performed based on the online power flow calculation results to obtain the feasible candidate action set. The optimal control action generation module is used to score the physical indicators of the candidate actions in the feasible candidate action set and select the candidate action with the best score as the optimal control action. The safety backup action generation module is used to reduce the amplitude of the energy storage active power candidate value and the inverter reactive power candidate value according to a preset scaling factor and re-execute the candidate action evaluation and screening when the set of feasible candidate actions is empty; if no feasible candidate actions are found after re-screening, the feasible actions of the previous online control cycle are limited to the current equipment's allowable power range as safety backup actions; if no feasible actions are found in the previous online control cycle, the energy storage active power and the inverter reactive power are both set to zero and an abnormal alarm is output. The indicator output module is used to record the line loss, node voltage, active power on the main transformer side, reverse power, and energy storage state of charge in the current online control cycle, and feeds the operation results back to the status acquisition module to construct the online state vector for the next online control cycle.

[0107] Example 3 This embodiment does not use the method of mapping the continuous proposed actions output by the Actor network to discrete actions with the smallest action vector distance through the k-nearest neighbor method, and determines the final action processing method based on the value estimation of the Critic network.

[0108] In this embodiment, the continuously suggested actions serve only as the basis for constructing the physical candidate action set. The physical candidate action set consists of uncontrolled actions, fixed rule actions, DDPG suggested actions, and proportional actions determined based on the allowable charging and discharging power of the energy storage and the currently available reactive power of the inverter. Each candidate action has a corresponding physical meaning of the active power of the energy storage and the reactive power of the inverter.

[0109] For each candidate action in the physical candidate action set, power flow calculation is performed under the same online state, and hard constraint screening is performed based on the power flow convergence state, energy storage state of charge, and equipment power range. Then, physical index scores are performed based on the line loss, node voltage, reverse power, and energy storage state of charge corresponding to the feasible candidate actions to determine the final control action.

[0110] Therefore, the distance between candidate actions in the action vector space is not used as the basis for judging the feasibility of a candidate action or determining the final control action. Even if the action vectors of two candidate actions are similar, if one of the candidate actions causes the power flow calculation to fail to converge, the energy storage state of charge to exceed the limit, or the line loss to be high, it will still be eliminated or given a higher comprehensive score based on its actual power flow calculation results, rather than being determined as the final control action simply because the Critic network has a high value estimate for that candidate action.

[0111] Example 4 like Figure 3 and Figure 4 As shown, this embodiment constructs a high-penetration distributed photovoltaic low-voltage distribution area with coordinated source-grid-load-storage control. Among them, Figure 3 This is used to explain the collaborative loss reduction system for low-voltage distribution areas and the collaborative relationships between its various functional modules. Figure 4 This is used to illustrate the physical topology of the low-voltage distribution area and the arrangement of photovoltaic, load, energy storage and controller.

[0112] This embodiment constructs a 10 kV / 0.4 kV radial low-voltage distribution area. The upstream 10 kV power grid is connected to the 0.4 kV low-voltage busbar via a 10 kV / 0.4 kV distribution transformer. Six low-voltage feeders extend from the low-voltage busbar, each with 12 nodes, forming a total of 72 load or distributed photovoltaic access nodes. An energy storage device is installed at the end of each low-voltage feeder, for a total of six energy storage devices. The loads include residential loads, commercial loads, small industrial or workshop loads, and public loads.

[0113] like Figure 3 As shown, the status acquisition module acquires operational data from low-voltage distribution areas, distributed photovoltaic systems, energy storage systems, and low-voltage loads; the DDPG control module outputs suggested actions based on the acquired operational data; the candidate action construction module constructs a set of physical candidate actions based on the suggested actions; the power flow safety screening module performs power flow calculations, hard constraint screening, and physical index scoring on each candidate action; the action execution module applies the finally determined active power of the energy storage system and the total reactive power of the inverter to the energy storage system and the photovoltaic inverter, respectively; the index output module records line losses, node voltages, reverse power, and energy storage state of charge, and feeds the operational results back to the status acquisition module to form a closed-loop control.

[0114] To create simulation scenarios under different operating conditions, this embodiment sets the photovoltaic penetration rate to 20%, 40%, 60%, 80%, 100%, and 120%, respectively; the distributed photovoltaic access locations are set as front, middle, rear, and mixed, respectively; and the weather scenarios are set as sunny, cloudy, and rainy, respectively. Combining the 6 photovoltaic penetration rates, 4 access locations, and 3 weather scenarios results in a total of 72 simulation scenarios.

[0115] The simulation duration for each simulation scenario is 24 hours, and the control step size is 1 hour, meaning that each simulation scenario includes 24 control cycles.

[0116] In this embodiment, the total rated energy of the six energy storage devices is 0.6 MWh, the total maximum charging power and the total maximum discharging power are both 0.2 MW, the initial state of charge (SOC) of the energy storage is 50%, the lower limit of SOC is 10%, and the upper limit of SOC is 90%. The rated apparent capacity of each photovoltaic inverter is set at 1.1 times the corresponding rated active capacity of the photovoltaic system.

[0117] Among the average results across 72 scenarios, the average daily equivalent line losses for the uncontrolled strategy, fixed-rule energy storage strategy, hourly power flow feedback optimization, and DDPG-safety screening strategy were 1703.7 kWh, 1705.1 kWh, 1391.0 kWh, and 1367.2 kWh, respectively. DDPG-safety screening achieved an average loss reduction of 21.28% compared to the uncontrolled strategy. In the 100% PV penetration, mixed-access, and sunny scenarios, the corresponding daily equivalent line losses were 963.6 kWh, 901.6 kWh, 670.7 kWh, and 633.1 kWh, respectively. DDPG-safety screening achieved a loss reduction of 34.30% compared to the uncontrolled strategy.

[0118] Figure 4 This diagram illustrates the radial topology of a low-voltage distribution area and the relative arrangement of loads, distributed photovoltaic (PV) systems, energy storage devices, and controllers. The number of feeders and the number of energy storage devices at the feeder ends are consistent with this embodiment. The number of nodes shown for each feeder and the PV and load connection locations are for illustrative purposes only; the actual number of nodes and connection locations in the simulation should be based on the above settings. The above-described distribution area structure, number of scenarios, and equipment parameters are merely one embodiment of the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0119] Example 5 This embodiment introduces a method for constructing a low-voltage distribution area model: This embodiment constructs a low-voltage distribution area model including an upstream 10 kV power grid, 10 / 0.4 kV distribution transformers, a 0.4 kV low-voltage busbar, and 6 low-voltage feeders. The model has a total of 74 nodes, including one 10 kV balancing node, one 0.4 kV low-voltage busbar node, and 72 low-voltage feeder nodes; each low-voltage feeder has 12 feeder nodes and 12 low-voltage line segments, for a total of 72 low-voltage line segments.

[0120] The rated capacity of the distribution transformer is set at 1.6 MVA, the rated voltage on the high-voltage side is set at 10 kV, the rated voltage on the low-voltage side is set at 0.4 kV, the short-circuit impedance is set at 4%, the short-circuit resistance voltage is set at 1%, the no-load loss is set at 0.6 kW, and the no-load current is set at 0.2%. The resistance per unit length of each phase of each low-voltage line section is set at 0.642 Ω / km, the reactance per unit length of each phase is set at 0.083 Ω / km, the length of a single line section is set within the range of 0.025235–0.054157 km, and the allowable current carrying capacity is set at 450 A.

[0121] Constant power loads were configured at 72 low-voltage feeder nodes, including commercial, public, residential, and small industrial loads. The total baseline active power of the loads was 1.036788 MW, and the total baseline reactive power was 0.456530 Mvar. Based on the 24-hour load time series coefficients corresponding to each load type, the active and reactive power loads of the nodes in each control cycle were determined.

[0122] Distributed photovoltaic (PV) systems were configured in the middle of each feeder, with a total of 24 PV access points and a total rated active power capacity of 0.829431 MW. The rated apparent capacity of each PV inverter was set at 1.1 times the corresponding rated active power capacity of the PV system, resulting in a total rated apparent capacity of 0.912374 MVA for the inverters. Based on the 24-hour PV output curve corresponding to a sunny weather scenario, the active power output of the PV system in each control cycle was determined.

[0123] An energy storage unit is installed at the end of each of the six low-voltage feeders, for a total of six energy storage units. Each energy storage unit has a rated energy of 0.1 MWh, a rated maximum charging power and a rated maximum discharging power of 0.033333 MW, an initial state of charge of 50%, a minimum state of charge of 10%, and a maximum state of charge of 90%. The total rated energy of the six energy storage units is 0.6 MWh, and the total rated maximum charging power and the total rated maximum discharging power are both 0.2 MW.

[0124] This embodiment uses a typical 24-hour day as the operating cycle, sets the control cycle length to 1 hour, and includes 24 online control cycles. The allowable range of node voltage is set to 0.95–1.05 pu, and the capacity limit of the distribution transformer is set to 1.6 MVA. The low-voltage distribution area model is constructed using a three-phase balanced positive sequence equivalent method, without separately considering three-phase imbalance and neutral current.

Claims

1. A low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening, characterized in that, The method includes the following steps: S1. Construct a low-voltage distribution area model and obtain the operating status and scenario parameters of the low-voltage distribution area in the current training control cycle; S2. Construct a training state vector based on the parameters obtained in step S1, and input the training state vector into the DDPG agent to obtain the current training action; S3. Training the DDPG agent: For the same training state, apply uncontrolled actions and current training actions respectively in the low-voltage transformer area model, and perform training power flow calculations respectively. S4. Based on the results of the training power flow calculation, calculate the reward value of the pre-constructed reward function, and iteratively update the network parameters of the DDPG agent according to the reward value until the training termination condition is met, and obtain the trained DDPG agent. S5. Obtain the operating status and scenario parameters of the low-voltage distribution area in the current online control cycle, and construct the current online status vector; S6. Online generation of suggested actions: Input the current online state vector into the trained DDPG agent to obtain continuous suggested actions for the current online control cycle; S7. Based on the continuous suggested actions, construct a set of physical candidate actions including the energy storage active power candidate set and the inverter reactive power candidate set; S8. Under the same online state, apply each candidate action in the physical candidate action set to the low-voltage distribution area model, perform online power flow calculations, and then perform hard constraint screening based on the online power flow calculation results to obtain a set of feasible candidate actions. S9. Score the candidate actions in the feasible candidate action set by physical index, and select the candidate action with the best score as the optimal control action.

2. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 1, characterized in that, In step S1, constructing the low-voltage distribution area model includes: determining the electrical connection relationships between the upstream power grid, distribution transformers, low-voltage buses, low-voltage feeders and nodes based on the network topology of the low-voltage distribution area, and configuring the parameters of low-voltage lines, distribution transformers, loads, distributed photovoltaics, photovoltaic inverters and energy storage. The operating status includes: training control cycle number, active power of each load node, active power output of each photovoltaic access point, active power of the main transformer side, voltage of each node, current of each branch, and SOC of each energy storage. The scenario parameters include: photovoltaic penetration rate, photovoltaic access location, weather code, and load type.

3. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 2, characterized in that, Training state vector for: in, For the first The state at each training moment of a control cycle For the first Total load for each control cycle For the first Total photovoltaic output per control cycle For the first Active power of the main transformer side in each control cycle; For the first The power fed back in one control cycle; For the first The lowest node voltage in each control cycle For the first The highest voltage at each node during each control cycle. For the first Line loss per control cycle For the first Energy storage SOC per control cycle, For photovoltaic penetration rate, For the first Weather codes for each control cycle.

4. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 3, characterized in that, The current training movements are as follows: ,in, For the index value of the training control cycle, For the first The stored active power of each training control cycle For the first Inverter reactive power during each training control cycle; no control action. .

5. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 4, characterized in that, The pre-built reward function is as follows: in, For the first The reward value for each training control cycle. For the first After executing uncontrolled actions in each training control cycle, the baseline line loss is calculated based on the branch current and line resistance obtained from the power flow calculation. The current circuit loss after performing the current training action; For relative loss reduction profit weighting, This is the current line loss weight. , , and The first Voltage over-limit penalty, reverse power penalty, SOC over-limit penalty, and action change penalty for each training control cycle. To prevent positive numbers with a denominator of zero.

6. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 5, characterized in that, The current online state vector adopts the same state component arrangement order, state quantity calculation method, baseline value and normalization parameter as the training state vector, and is constructed by the running state and corresponding scenario parameters obtained in real time during the current online control cycle; Continuous recommended actions during the current online control cycle Specifically: ,in, This is the index value for the online control cycle. Let be the recommended value for the active power of energy storage in the t-th online control cycle. Here is the recommended value for the inverter reactive power in the t-th online control cycle; The candidate set of active power for energy storage includes: zero active power, active power for energy storage determined according to preset fixed control rules, and... The preset proportional power in the energy storage charging direction and the preset proportional power in the energy storage discharging direction; the preset proportional power is the negative proportional value of the current maximum allowable charging power and the positive proportional value of the current maximum allowable discharging power, respectively. The candidate set of total reactive power for the inverter includes: the total reactive power of the inverter determined according to a preset fixed control rule. Zero reactive power and multiple positive or negative proportional values ​​determined according to the total available reactive power of the inverter; The candidate values ​​in the active power candidate set of the energy storage are combined with the candidate values ​​in the reactive power candidate set of the inverter, and duplicate candidate actions are merged to obtain a physical candidate action set. The maximum available reactive power of each inverter is determined based on its rated apparent power and current photovoltaic active power output. The maximum available reactive power of each inverter is then summed to obtain the range of total available reactive power of the inverter. The set of physical candidate actions and the total available reactive power range of the inverter together constitute the set of physical candidate actions.

7. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 6, characterized in that, The hard constraint screening includes: when the power flow calculation corresponding to the candidate action does not converge, or the active power of the energy storage exceeds the current allowable charging and discharging power range, or the total reactive power of the inverter exceeds the total available reactive power range of the inverter, or the energy storage SOC after the candidate action is executed exceeds the preset upper and lower limits, or the node voltage exceeds the preset hard constraint threshold, the candidate action is determined as an infeasible candidate action and is removed. The candidate actions that were not eliminated are identified as feasible candidate actions, forming a set of feasible candidate actions.

8. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 7, characterized in that, The formula for calculating the physical index score is: in, The score is for physical indicators. Candidate actions for which scores are to be calculated. Candidate actions The normalized line loss value, Candidate actions The node voltage exceeds the limit value, Candidate actions The normalized backfeed power value, Candidate actions SOC security margin penalty value, For the action of the previous control cycle, , , , and This represents the weight of the corresponding scoring item.

9. The low-voltage distribution area collaborative loss reduction method based on reinforcement learning and power flow screening according to claim 8, characterized in that, When the set of feasible candidate actions is empty, the magnitude of the energy storage active power candidate value and the inverter reactive power candidate value is reduced according to the preset scaling factor, and the power flow calculation and hard constraint screening are re-executed. If no feasible candidate action is found after re-screening, the feasible action of the previous online control cycle will be limited to the current allowable power range of the equipment to form a safety backup action; if no feasible action is found in the previous online control cycle, the active power of the energy storage and the reactive power of the inverter will be set to zero and an abnormal alarm will be output.

10. A low-voltage distribution area collaborative loss reduction system based on reinforcement learning and power flow screening, characterized in that, The system is used to implement the method according to any one of claims 1 to 9, the system comprising: The training action generation module is used to generate a training state vector based on the training running state and scene parameters, and to obtain the current training action; The DDPG agent training module is used to update the DDPG agent based on the power flow calculation results of the uncontrolled action and the current training action, so as to obtain the trained DDPG agent. The online action suggestion generation module is used to generate an online state vector based on the online running status and scene parameters, and obtain continuous action suggestions. The candidate action construction module is used to construct a set of physical candidate actions based on the continuous suggested actions; The feasible action filtering module is used to perform power flow calculation and hard constraint filtering on the set of physical candidate actions to obtain a set of feasible candidate actions. The optimal control action determination module is used to score the physical indicators of feasible candidate actions and determine the optimal control action. The safety backup action generation module is used to narrow down the amplitude of candidate actions and re-filter them when no feasible candidate actions exist, or to generate safety backup actions based on the feasible actions of the previous online control cycle; when no feasible actions of the previous online control cycle exist, it outputs zero-power actions and abnormal alarms.

Images