Intelligent learning control method, system, equipment and medium for middle and low voltage matching table in sensitivity screening

Abstract

The application discloses a kind of sensitivity screening medium-low voltage distribution table intelligent learning control method, system, equipment and medium, method includes: obtaining distribution network parameter information and distributed photovoltaic and load historical operation data in table area;Each table area voltage sensitivity is calculated;The collaborative control process of distribution network and table area is modeled into Markov decision process, each table area is the intelligent agent of reinforcement learning, the voltage sensitivity of table area is included in the state space observed by multi-agent, and the distribution network is training environment;According to the parameter information of distribution network and distributed photovoltaic and load historical operation data in table area, each table area intelligent agent is trained, and the exploration noise amplitude of each table area intelligent agent in training stage is dynamically adjusted according to voltage sensitivity, to obtain trained table area intelligent agent, for controlling the active power and reactive power of each table area and distribution network interaction, to control table area voltage.The application can make more efficient, stable and self-adaptive decision under complex power grid environment.

Description

Technical Field

[0001] This invention relates to the field of distributed energy storage control, specifically to a sensitivity-screening intelligent learning control method, system, device, and medium for medium and low voltage distribution stations. Background Technology

[0002] As the energy system accelerates its transformation towards a cleaner and lower-carbon model, a large number of distributed renewable energy sources are being integrated into the distribution network, driving profound changes in the network's form and operation. Future active distribution networks will exhibit new characteristics: the network structure will shift from the traditional unidirectional power supply mode to a multi-source interactive active network; the operation and control mode will shift from centralized management to hierarchical and regional coordinated optimization, ultimately forming a smart distribution network architecture that integrates medium and low voltage and AC / DC power. However, the output of distributed power sources, represented by photovoltaics, is intermittent and uncertain, easily leading to problems such as voltage fluctuations and uneven power flow distribution in the distribution network, posing challenges to the efficient absorption of new energy. Against this backdrop, distribution transformer substations, as important carriers of flexible resources, can achieve precise power regulation by aggregating internal controllable loads and energy storage resources, thereby supporting the safe operation of the distribution network and optimizing voltage distribution. With the large-scale integration of substations, the distribution network's regulation capacity is improved, but it is also necessary to conduct in-depth research on the collaborative optimization strategies between substations and the distribution network to achieve more efficient and flexible energy dispatch.

[0003] Currently, optimization and control methods for distribution substations in medium and low voltage distribution networks can be mainly divided into two technical routes: physical model-based and data-driven. Traditional model-driven methods rely primarily on mathematical programming theory and intelligent optimization algorithms to solve distribution network optimization problems by establishing precise mathematical models. However, due to the common problems of insufficient communication infrastructure coverage and frequent network topology changes in medium and low voltage distribution networks, it is difficult for the system to obtain complete real-time operational data, posing challenges to these model-dependent optimization methods in practical applications. In recent years, with the rapid development of power grid technology, data-driven methods based on deep reinforcement learning have provided innovative solutions for real-time optimization and control of distribution networks. Multi-agent systems, as a distributed decision-making architecture, can achieve autonomous coordination and optimization between the distribution network and substations, improving the flexibility, reliability, and economy of system operation. This method treats various regulatory resources as agents, continuously interacting and learning with the distribution network environment, and autonomously extracting optimization patterns from massive operational data using deep neural networks, ultimately achieving rapid and adaptive control decisions. This novel method can effectively cope with complex situations such as data gaps and topology changes in distribution networks, demonstrating significant technical advantages. Summary of the Invention

[0004] This invention addresses the problems of incomplete data parameters and slow calculation speed in existing voltage control methods for medium and low voltage distribution stations. It provides a sensitivity-screening intelligent learning control method, system, device, and medium for medium and low voltage distribution stations. By using a sensitivity-screening deep reinforcement learning method, the distribution station is controlled as an intelligent agent. After offline training based on historical data, it provides online voltage control strategies to the distribution network in real time.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A sensitivity-screened intelligent learning control method for medium and low voltage distribution stations includes the following steps: S1. Obtain parameter information of the distribution network and historical operation data of distributed photovoltaic and loads within the transformer area; S2. Calculate the voltage sensitivity of each distribution area based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load within the distribution area. S3. The collaborative control process between the distribution network and the transformer substations is modeled as a Markov decision process. Each transformer substation is used as a reinforcement learning agent. The voltage sensitivity of the transformer substations is incorporated into the state space of the multi-agent observation, and the distribution network is used as the training environment. S4. Based on the power distribution network parameter information in S1 and the historical operation data of distributed photovoltaic and load in the distribution area, conduct offline training for each distribution area agent. Dynamically adjust the exploration noise amplitude of each distribution area agent during the training phase according to the voltage sensitivity, and finally obtain the trained distribution area agent. S5. Use the trained transformer area agent to control the active and reactive power of each transformer area interacting with the distribution network, so as to control the transformer area voltage.

[0007] To optimize the above technical solution, the specific measures also include: Furthermore, in S2, the calculation of the voltage sensitivity of each transformer area specifically involves: Based on the active and reactive power regulation characteristics of medium and low voltage distribution networks, the active power-voltage sensitivity and reactive power-voltage sensitivity of each node are calculated. The weighted sum of each term in the sensitivity matrix is ​​then obtained to obtain the comprehensive voltage sensitivity of each node. The calculation formula is:

[0008] In the formula, n is the total number of nodes in the distribution network, and n-1 represents the number of nodes remaining after removing the balancing node; and These are the preset active power voltage sensitivity weighting coefficients and reactive power voltage sensitivity weighting coefficients, respectively, and they satisfy... . It is the voltage sensitivity of the i-th node to the active power injected by the j-th node; It is the voltage sensitivity of the i-th node to the reactive power injected by the j-th node; Let be the voltage partial derivative at the i-th node. Let be the partial derivative of the active power at the j-th node. Let be the partial derivative of the reactive power at the j-th node.

[0009] The comprehensive voltage sensitivity corresponding to the grid-connected node of the k-th transformer area is taken as the voltage sensitivity of the environment where the intelligent agent of that transformer area is located. , representing the comprehensive voltage sensitivity of the k-th transformer area in the m-th training round, is used as input to the state space of the corresponding transformer area agent.

[0010] Furthermore, in S3, the specific steps of modeling the coordinated control process of the distribution network and transformer substations as a Markov decision process are as follows: Construct the state space, action space, state transition function, and reward function; The state space represents the environmental state of the power distribution network in which each transformer substation agent is located, including the net active power, net reactive power, substation outlet voltage, and voltage sensitivity of each substation under this state. The state space is mathematically represented as:

[0011] In the formula, It is the state space of the k-th agent in the m-th training round. It is the net active power of the k-th transformer area in the m-th training round. It is the net reactive power of the k-th transformer area in the m-th training round. It is the output voltage of the k-th transformer area in the m-th training round. It is the voltage sensitivity of the k-th transformer area in the m-th training round. This represents the total number of intelligent agents in the distribution area; The action space is the collection of actions of each distribution transformer agent, that is, the active and reactive power interacting between each distribution transformer and the distribution network. The action space is represented as:

[0012] In the formula, and The first k The active and reactive power of the interaction between the intelligent agent in each distribution area and the distribution network; For the first k The action space of an agent in a certain area during the m-th training round; The state transition function represents the probability that an agent in a given area will transition from one state to the next after observing a new environment and performing a new action. It is expressed by the formula:

[0013] In the formula, For the intelligent agent in the region in strategy From the previous state Transition to the next state The probability, For the agent to perform actions in the previous state a The strategy function; Execute an action in the previous state of the agent. a The probability of transitioning to the next state; A It is the action space; The reward function is used to measure the value of each agent's transition from the previous state to the new state. The reward function is represented by a negative objective function and a penalty term, as shown in the following expression:

[0014]

[0015]

[0016]

[0017] In the formula, Let be the reward value for the m-th training round. This is the weighting factor for the voltage deviation in the distribution network; This is the weighting coefficient for distribution network losses; The weighting coefficients for the combined voltage sensitivity and control action; This is the bonus value for voltage deviation. This is the bonus value for distribution network losses. The penalty is a weighted combination of voltage sensitivity and control action; M represents the total number of training rounds. For the first m training round nodes i The voltage value on; This is the reference voltage value; n is the total number of nodes in the distribution network. Let be the resistance value of the branch between node i and node j; and Branch roads ij Active and reactive power transmitted upstream. It is the voltage sensitivity of the k-th transformer area in the m-th training round. For the first k The action space of a regional agent in the m-th training round.

[0018] Furthermore, in S4, the offline training of the intelligent agents in each distribution area specifically involves: S4.1 Input the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load in the transformer area, initialize the Actor network and Critic network of each regional agent, and initialize the shared experience playback pool. D ; S4.2 Setting the number of training rounds M Set the initial maximum noise standard deviation. and initial minimum noise standard deviation Set the learning rate for the Actor network. Learning rate of the Critic network and discount factor Initial sensitivity attenuation coefficient ; S4.3, Set the current training round number Reset the power distribution network simulation environment to its initial state; S4.4 Receive the voltage sensitivity of each station area calculated by S2; S4.5. Dynamically adjust the exploration noise amplitude of each agent in the training phase based on voltage sensitivity, using the following formula: Normalize the voltage sensitivity:

[0019] In the formula, For normalized voltage sensitivity, This is the minimum voltage sensitivity among all nodes. This is the maximum voltage sensitivity among all nodes; It is the voltage sensitivity of the kth transformer area in the mth training round; The noise amplitude was explored by dynamically adjusting the normalized voltage sensitivity, maximum noise standard deviation, minimum noise standard deviation, and sensitivity attenuation coefficient.

[0020] In the formula, The dynamically adjusted exploration noise amplitude, It is the maximum noise standard deviation in the m-th training round. It is the minimum noise standard deviation in the m-th training round; It is the sensitivity decay coefficient in the m-th training round; S4.6 The agent in the district produces joint actions based on the current policy network and the dynamically adjusted exploration noise amplitude. The formula is as follows:

[0021]

[0022] In the formula, It is the action of the k-th agent in the m-th training round. This function represents the function that restricts the actions of the agent in the control area to within upper and lower limits. Indicates the current state space The policy network below; and These are the upper and lower limits of the agent's actions, respectively; The dynamically adjusted exploration noise amplitude, This represents the dynamically adjusted exploration noise amplitude under a Gaussian distribution. S4.7 After the intelligent agent in the control area performs a joint action, the environment provides a global reward value. and the next state ; S4.8, Transfer the generated samples Stored in the shared experience replay pool D ; S4.9 Learning Rate Based on Critic Network Update the Critic network parameters using the loss function, based on the learning rate of the Actor network. Update the Actor network parameters with the objective function; S4.10 Determine whether the reward function has converged. If it has, exit the loop. Otherwise, increment the training round number by one, dynamically update the noise parameters based on the training round number, and return to step S4.5.

[0023] Furthermore, S4.9 specifically includes: Define the sensitivity adaptive weighting coefficients of the intelligent agent in the distribution area. The calculation formula is:

[0024] In the formula, For the sensitivity adaptive weighting coefficients of the k-th station agent in the m-th training round; This is the normalized voltage sensitivity calculated in S4.5; This is a preset sensitivity amplification factor used to adjust the degree to which the sensitivity affects the gradient.

[0025] By incorporating the aforementioned sensitivity adaptive weighting coefficients, the loss function of the Critic network is weighted, resulting in the improved expression for the Critic network loss function:

[0026]

[0027] In the formula, is the loss function of the Critic network; The desired target value; It is the reward value for the m-th training round; Discount factor; The value estimated by the target Critic network for the next state; For the target Actor policy network; The value estimated by the Critic network in the current state.

[0028] The Critic network parameter update formula is:

[0029] In the formula, These are the parameters of the Critic network. It is the learning rate of the Critic network. This is the gradient operator.

[0030] The gradient update formula for the Actor network policy is:

[0031] In the formula, The gradient of the Actor network policy; This represents the gradient of the Critic network's evaluation of the action, serving as a guide for direction. This represents the influence of Actor network parameters on action output.

[0032] The formula for updating the Actor network parameters is:

[0033] In the formula, For Actor network parameters, It is the learning rate of the Actor network.

[0034] Furthermore, in S4.10, the step of dynamically updating the noise parameters according to the number of training rounds specifically involves: The maximum noise standard deviation, minimum noise standard deviation, and sensitivity attenuation coefficient are updated using the following formulas:

[0035]

[0036]

[0037] In the formula, The maximum noise standard deviation in the m-th training round. The initial maximum noise standard deviation, Let m be the minimum noise standard deviation of the m-th training round. This represents the initial minimum noise standard deviation; Noise attenuation rate (rounds) Let be the sensitivity decay coefficient for the m-th training round. This is the initial sensitivity attenuation coefficient. This represents the growth rate of the attenuation coefficient.

[0038] This invention also proposes a sensitivity-screening intelligent learning control system for medium and low voltage distribution stations, comprising: The data acquisition module is used to acquire parameter information of the distribution network and historical operating data of distributed photovoltaic and loads within the transformer area; The transformer substation agent model building module is used to calculate the voltage sensitivity of each transformer substation based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load within the transformer substation; the collaborative control process of the distribution network and transformer substation is modeled as a Markov decision process, with each transformer substation as a reinforcement learning agent, and the voltage sensitivity of the transformer substation is incorporated into the state space of multi-agent observation, with the distribution network as the training environment; The training module is used to conduct offline training on each distribution area agent based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load in the distribution area. It dynamically adjusts the exploration noise amplitude of each distribution area agent during the training phase according to the voltage sensitivity, and finally obtains the trained distribution area agent. The power allocation module is used to use the trained transformer area agents to control the active and reactive power interaction between each transformer area and the distribution network, so as to control the transformer area voltage.

[0039] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the intelligent learning control method for medium and low voltage distribution platforms with sensitivity screening as described above.

[0040] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to execute the sensitivity screening intelligent learning control method for medium and low voltage distribution as described above.

[0041] The beneficial effects of this invention are: This invention addresses the overvoltage problems commonly found in low- and medium-voltage distribution networks, such as insufficient communication infrastructure coverage and frequent network topology changes, as well as the slow or unstable convergence during reinforcement learning training. By utilizing voltage sensitivity information, the agent can make more accurate and adaptive responses when the grid voltage fluctuates drastically, thereby enhancing the robustness and stability of the grid. It accurately identifies voltage-sensitive regions, avoiding excessive adjustments in these areas and effectively preventing instability caused by over-adjustment. The observation space introduced by voltage sensitivity allows the agent to explore control actions that do not significantly affect voltage changes during training, effectively narrowing the search space, accelerating training convergence, and improving training stability and efficiency. Ultimately, it enhances the performance and robustness of low- and medium-voltage distribution systems, enabling the agent to make more efficient, stable, and adaptive decisions in complex grid environments. Attached Figure Description

[0042] Figure 1 This is a flowchart of a sensitivity-screening intelligent learning control method for medium and low voltage distribution systems.

[0043] Figure 2 This describes the 33-node power distribution line topology and its feeder connection to the transformer substation in this embodiment of the invention. Figure 3 This is the voltage distribution of each node in each distribution area 24 hours after the intelligent agent is connected to the distribution network before training in this embodiment of the invention; Figure 4 These are the voltage sensitivity values ​​of each node at a typical moment in an embodiment of the present invention; Figure 5 This is the voltage distribution of each node over 24 hours after using a traditional intelligent agent solution algorithm in this embodiment of the invention. Figure 6 This is the voltage distribution of each node over 24 hours after using the method of the present invention in this embodiment of the invention; Figure 7 This is a comparison of the traditional method and the method of the present invention in the multi-agent training convergence process in the embodiments of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0045] Example 1 This invention proposes a sensitivity-screening intelligent learning control method for medium and low voltage distribution stations. The process of this method is as follows: Figure 1As shown, it includes the following steps: S1. Obtain the parameter information of the distribution network and the historical operation data of distributed photovoltaic and loads within the transformer area; the parameter information of the distribution network includes local load power, line parameters and network topology.

[0046] See Figure 2 This invention is applied to a 33-node medium- and low-voltage distribution network, connected to transformer substations at nodes 13, 14, 18, 21, 22, 25, 29, and 32. Each substation contains several photovoltaic systems, energy storage systems, and household loads. The per-unit voltage of the nodes is 1.00 pu, and the safe operating range is 0.95 pu to 1.05 pu. (See reference) Figure 3 When a large number of transformer substations are connected to the distribution network, the voltage at each node of the line exceeds the limit within 24 hours, and the voltage deviation is relatively large. Therefore, a 24-hour period is selected for analysis.

[0047] S2. Calculate the voltage sensitivity of each distribution substation based on the distribution network parameters and historical operating data of distributed photovoltaic systems and loads within the substation area. In distribution networks, especially in medium and low voltage distribution substations, load changes are frequent and difficult to predict, making voltage sensitivity a crucial parameter in designing intelligent control strategies. Because the voltage regulation performance of different nodes in the distribution network varies, terminal nodes and grid-connected substation nodes are more prone to voltage fluctuations. These locations require more attention to implement more precise control strategies and avoid voltage over-limits or excessive fluctuations.

[0048] S2 specifically refers to: Based on the active and reactive power regulation characteristics of medium and low voltage distribution networks, the active power-voltage sensitivity and reactive power-voltage sensitivity of each node are calculated. The weighted sum of each term in the sensitivity matrix is ​​then obtained to obtain the comprehensive voltage sensitivity of each node. The calculation formula is:

[0049] In the formula, n is the total number of nodes in the distribution network, and n-1 represents the number of nodes remaining after removing the balancing node; and These are the preset active power voltage sensitivity weighting coefficients and reactive power voltage sensitivity weighting coefficients, respectively, and they satisfy... . It is the voltage sensitivity of the i-th node to the active power injected by the j-th node; It is the voltage sensitivity of the i-th node to the reactive power injected by the j-th node; Let be the voltage partial derivative at the i-th node. Let be the partial derivative of the active power at the j-th node. Let be the partial derivative of the reactive power at the j-th node.

[0050] The comprehensive voltage sensitivity corresponding to the grid-connected node of the k-th transformer area is taken as the voltage sensitivity of the environment where the intelligent agent of that transformer area is located. , representing the comprehensive voltage sensitivity of the k-th transformer area in the m-th training round, is used as input to the state space of the corresponding transformer area agent.

[0051] See Figure 4 , which represents the voltage sensitivity value of each node at a typical moment. At this time, the voltage sensitivity of the transformer areas at the grid connection points of nodes 13, 15, and 32 is relatively high, and these transformer area agents are the main control objects.

[0052] S3. The collaborative control process of the distribution network and transformer substations is modeled as a Markov decision process. Each transformer substation is used as a reinforcement learning agent. The voltage sensitivity of the transformer substation is incorporated into the state space of the multi-agent observation, and the distribution network is used as the training environment. Each agent makes a decision in the next moment based on the state space generated by the action executed in the previous moment.

[0053] Modeling the coordinated control process between the distribution network and transformer substations as a Markov decision process is specifically as follows: Construct the state space, action space, state transition function, and reward function; The state space represents the environmental state of the power distribution network in which each transformer substation agent is located, including the net active power, net reactive power, substation outlet voltage, and voltage sensitivity of each substation under this state. The state space is mathematically represented as:

[0054] In the formula, It is the state space of the k-th agent in the m-th training round. It is the net active power of the k-th transformer area in the m-th training round. It is the net reactive power of the k-th transformer area in the m-th training round. It is the output voltage of the k-th transformer area in the m-th training round. It is the voltage sensitivity of the k-th transformer area in the m-th training round. This represents the total number of intelligent agents in the distribution area. By incorporating voltage sensitivity into the state space, the actions of the intelligent agents will precisely adjust the distribution areas with large voltage changes, thereby improving the control effect.

[0055] The action space is the collection of actions of each distribution transformer agent, that is, the active and reactive power interacting between each distribution transformer and the distribution network. The action space is represented as:

[0056] In the formula, and The firstk The active and reactive power of the interaction between the intelligent agent in each distribution area and the distribution network; This indicates the active power output of the transformer substation. Indicates the absorption of active power. Similarly. For the first k The action space of an agent in a certain area during the m-th training round; The state transition function represents the probability that an agent in a given area will transition from one state to the next after observing a new environment and performing a new action. It is expressed by the formula:

[0057] In the formula, For the intelligent agent in the region in strategy From the previous state Transition to the next state The probability, For the agent to perform actions in the previous state a The strategy function; Execute an action in the previous state of the agent. a The probability of transitioning to the next state; A It is the action space; The reward function measures the value of each agent's transition from the previous state to the new state. The reward function is represented by a negative objective function and a penalty term; the smaller the voltage deviation and network loss, the larger the reward. In addition to considering changes in voltage deviation and network loss, voltage sensitivity is also combined with control actions. This allows the agent to focus on areas with high voltage sensitivity, avoiding excessive control actions at nodes with high voltage sensitivity, thereby improving the accuracy of control decisions. The expression is as follows:

[0058]

[0059]

[0060]

[0061] In the formula, Let be the reward value for the m-th training round. This is the weighting factor for the voltage deviation in the distribution network; This is the weighting coefficient for distribution network losses; The weighting coefficients for the combined voltage sensitivity and control action; This is the bonus value for voltage deviation. This is the bonus value for distribution network losses. The penalty is a weighted combination of voltage sensitivity and control action; M represents the total number of training rounds. For the first m training round nodes i The voltage value on; This is the reference voltage value; n is the total number of nodes in the distribution network. Let be the resistance value of the branch between node i and node j; and Branch roads ij Active and reactive power transmitted upstream. It is the voltage sensitivity of the k-th transformer area in the m-th training round. For the first k The action space of a regional agent in the m-th training round.

[0062] S4. Based on the distribution network parameter information in S1 and the historical operation data of distributed photovoltaic and loads within the distribution area, offline training is performed on each distribution area agent. The exploration noise amplitude of each distribution area agent during the training phase is dynamically adjusted according to voltage sensitivity, ultimately resulting in a well-trained distribution area agent. The multi-agent training adopts an Actor-Critic (AC) architecture. By analyzing observation information including voltage sensitivity, the adjustment priority under the current system state is identified, and targeted power control commands are generated. This is equivalent to extracting key features of voltage state and sensitivity, thereby improving training efficiency and stability.

[0063] The offline training of the agents in each distribution area is specifically as follows: S4.1 Input the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load in the transformer area, initialize the Actor network and Critic network of each regional agent, and initialize the shared experience playback pool. D ; S4.2 Setting the number of training rounds M Set the initial maximum noise standard deviation. and initial minimum noise standard deviation Set the learning rate for the Actor network. Learning rate of the Critic network and discount factor and soft update rate Initial sensitivity attenuation coefficient ; The parameters of the agent are set as shown in Table 1.

[0064] Table 1

[0065] S4.3, Set the current training round number Reset the power distribution network simulation environment to its initial state; S4.4 Receive the voltage sensitivity of each station area calculated by S2; S4.5. Dynamically adjust the exploration noise amplitude of each agent in the training phase based on voltage sensitivity, using the following formula: Normalize the voltage sensitivity:

[0066] In the formula, For normalized voltage sensitivity, This is the minimum voltage sensitivity among all nodes. This is the maximum voltage sensitivity among all nodes; It is the voltage sensitivity of the kth transformer area in the mth training round; The noise amplitude was explored by dynamically adjusting the normalized voltage sensitivity, maximum noise standard deviation, minimum noise standard deviation, and sensitivity attenuation coefficient.

[0067] In the formula, The explored noise amplitude is dynamically adjusted. It is the maximum noise standard deviation in the m-th training round. It is the minimum noise standard deviation in the m-th training round; It is the sensitivity decay coefficient in the m-th training round; voltage sensitivity is inversely proportional to noise. The higher the voltage sensitivity, the less noise the station explores, reducing the station movement amplitude and avoiding excessive adjustments that cause large fluctuations in the reward function.

[0068] S4.6 The agent in the district produces joint actions based on the current policy network and the dynamically adjusted exploration noise amplitude. The formula is as follows:

[0069]

[0070] In the formula, It is the action of the k-th agent in the m-th training round. This function represents the function that restricts the actions of the agent in the control area to within upper and lower limits. Indicates the current state space The policy network below; and These are the upper and lower limits of the agent's actions, respectively; The explored noise amplitude is dynamically adjusted. This represents the dynamically adjusted exploration noise amplitude under a Gaussian distribution. S4.7 After the intelligent agent in the control area performs a joint action, the environment provides a global reward value. and the next state ; S4.8, Transfer the generated samples Stored in the shared experience replay pool D The strategy is continuously adjusted based on the reward value generated each time, so that the expected return is higher and higher. S4.9 Learning Rate Based on Critic Network Update the Critic network parameters using the loss function, based on the learning rate of the Actor network. Update the Actor network parameters with the objective function; utilize the soft update rate. Soft update the target network.

[0071] In S4.9, the specific steps for updating the Critic network parameters and Actor network parameters are as follows: Define the sensitivity adaptive weighting coefficients of the intelligent agent in the distribution area. The calculation formula is:

[0072] In the formula, For the sensitivity adaptive weighting coefficients of the k-th station agent in the m-th training round; This is the normalized voltage sensitivity calculated in S4.5; This is a preset sensitivity amplification factor used to adjust the degree to which the sensitivity affects the gradient.

[0073] By incorporating the aforementioned sensitivity adaptive weighting coefficients, the loss function of the Critic network is weighted, resulting in the improved expression for the Critic network loss function:

[0074]

[0075] In the formula, is the loss function of the Critic network; The desired target value; It is the reward value for the m-th training round; Discount factor; The value estimated by the target Critic network for the next state; For the target Actor policy network; The value estimated by the Critic network in the current state.

[0076] The Critic network parameter update formula is:

[0077] In the formula, These are the parameters of the Critic network. It is the learning rate of the Critic network. This is the gradient operator.

[0078] The gradient update formula for the Actor network policy is:

[0079] In the formula, The gradient of the Actor network policy; This represents the gradient of the Critic network's evaluation of the action, serving as a guide for direction. This represents the influence of Actor network parameters on action output.

[0080] The formula for updating the Actor network parameters is:

[0081] In the formula, For Actor network parameters, It is the learning rate of the Actor network.

[0082] During the update process, the gradient will be directly affected by the voltage sensitivity. The higher the voltage sensitivity of the transformer area, the greater the impact of the corresponding Q value on the policy gradient.

[0083] S4.10 Determine whether the reward function has converged. If it has, exit the loop. Otherwise, increment the training round number by one, dynamically update the noise parameters based on the training round number, and return to step S4.5.

[0084] The noise parameters are dynamically updated based on the number of training rounds as follows: The maximum noise standard deviation, minimum noise standard deviation, and sensitivity attenuation coefficient are updated using the following formulas:

[0085]

[0086]

[0087] In the formula, The maximum noise standard deviation in the m-th training round. The initial maximum noise standard deviation, Let m be the minimum noise standard deviation of the m-th training round. This represents the initial minimum noise standard deviation; Noise attenuation rate (rounds) Let be the sensitivity decay coefficient for the m-th training round. This is the initial sensitivity attenuation coefficient. This represents the growth rate of the attenuation coefficient.

[0088] S5. Use the trained transformer area agent to control the active and reactive power of each transformer area interacting with the distribution network, so as to control the transformer area voltage.

[0089] The trained agent in the transformer substation was used to test a voltage exceedance scenario. (See also...) Figure 3 When a large number of transformer substations are connected to the distribution network, the 24-hour voltage at various nodes of the line exceeds the limit, and the voltage deviation is relatively large. (See also...) Figure 5 After being solved using traditional intelligent agent algorithms, the node voltages were effectively controlled, and the voltages of each node returned to their normal operating range. (See also...) Figure 6 By incorporating voltage sensitivity into the observation range before controlling the agent, it can be found that voltage sensitivity is significantly different from that of the control agent. Figure 5 The further decrease indicates that the intelligent agent with voltage sensitivity as the controlled object has a better regulating effect on the voltage of the distribution network.

[0090] See Figure 7 Compared with traditional methods, the method of this invention is significantly more effective in terms of convergence, accelerating training convergence, improving training efficiency, and achieving more obvious optimization results.

[0091] Example 2 This invention proposes a sensitivity-screening intelligent learning control system for medium and low voltage distribution stations, corresponding to the method in Embodiment 1, comprising: The data acquisition module is used to acquire parameter information of the distribution network and historical operating data of distributed photovoltaic and loads within the transformer area; The transformer substation agent model building module is used to calculate the voltage sensitivity of each transformer substation based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load within the transformer substation; the collaborative control process of the distribution network and transformer substation is modeled as a Markov decision process, with each transformer substation as a reinforcement learning agent, and the voltage sensitivity of the transformer substation is incorporated into the state space of multi-agent observation, with the distribution network as the training environment; The training module is used to conduct offline training on each distribution area agent based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load in the distribution area. It dynamically adjusts the exploration noise amplitude of each distribution area agent during the training phase according to the voltage sensitivity, and finally obtains the trained distribution area agent. The power allocation module is used to use the trained transformer area agents to control the active and reactive power interaction between each transformer area and the distribution network, so as to control the transformer area voltage.

[0092] The implementation methods of each module and its function in the system are completely consistent with the steps of the method in Implementation Example 1, so they will not be repeated here.

[0093] Example 3 This invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a sensitivity-screening intelligent learning control method for medium and low voltage distribution stations, as described in Embodiment 1.

[0094] Example 4 This invention proposes a computer-readable storage medium storing a computer program that enables a computer to execute a sensitivity-screening intelligent learning control method for medium and low voltage distribution stations, as described in Embodiment 1.

[0095] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, and portable compact disc read-only memory (CD). ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A sensitivity-screening intelligent learning control method for medium and low voltage distribution stations, characterized in that, Includes the following steps: S1. Obtain parameter information of the distribution network and historical operation data of distributed photovoltaic and loads within the transformer area; S2. Calculate the voltage sensitivity of each distribution area based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load within the distribution area. S3. The collaborative control process between the distribution network and the transformer substations is modeled as a Markov decision process. Each transformer substation is used as a reinforcement learning agent. The voltage sensitivity of the transformer substations is incorporated into the state space of the multi-agent observation, and the distribution network is used as the training environment. S4. Based on the power distribution network parameter information in S1 and the historical operation data of distributed photovoltaic and load in the distribution area, conduct offline training for each distribution area agent. Dynamically adjust the exploration noise amplitude of each distribution area agent during the training phase according to the voltage sensitivity, and finally obtain the trained distribution area agent. S5. Use the trained transformer area agent to control the active and reactive power of each transformer area interacting with the distribution network, so as to control the transformer area voltage.

2. The intelligent learning control method for medium and low voltage distribution stations with sensitivity screening as described in claim 1, characterized in that, In S2, the calculation of the voltage sensitivity of each transformer area specifically involves: Based on the active and reactive power regulation characteristics of medium and low voltage distribution networks, the active-voltage sensitivity and reactive-voltage sensitivity of each node are calculated. The weighted sum of each term in the sensitivity matrix is ​​then obtained to obtain the comprehensive voltage sensitivity of each node. The calculation formula is: In the formula, n is the total number of nodes in the distribution network, and n-1 represents the number of nodes remaining after removing the balancing node; and These are the preset active power voltage sensitivity weighting coefficients and reactive power voltage sensitivity weighting coefficients, respectively, and they satisfy... ; It is the voltage sensitivity of the i-th node to the active power injected by the j-th node; It is the voltage sensitivity of the i-th node to the reactive power injected by the j-th node; Let be the voltage partial derivative at the i-th node. Let be the partial derivative of the active power at the j-th node. Let be the partial derivative of the reactive power at the j-th node; The comprehensive voltage sensitivity corresponding to the grid-connected node of the k-th transformer area is taken as the voltage sensitivity of the environment where the intelligent agent of that transformer area is located. , representing the comprehensive voltage sensitivity of the k-th transformer area in the m-th training round, is used as input to the state space of the corresponding transformer area agent.

3. The intelligent learning control method for medium and low voltage distribution stations with sensitivity screening as described in claim 1, characterized in that, In S3, the specific steps of modeling the coordinated control process of the distribution network and transformer substations as a Markov decision process are as follows: Construct the state space, action space, state transition function, and reward function; The state space represents the environmental state of the power distribution network in which each transformer substation agent is located, including the net active power, net reactive power, substation outlet voltage, and voltage sensitivity of each substation under this state. The state space is mathematically represented as: In the formula, This is the state space of the k-th agent in the m-th training round. It is the net active power of the k-th transformer area in the m-th training round. It is the net reactive power of the k-th transformer area in the m-th training round. It is the output voltage of the k-th transformer area in the m-th training round. It is the voltage sensitivity of the k-th transformer area in the m-th training round. This represents the total number of intelligent agents in the distribution area; The action space is the collection of actions of each distribution transformer agent, that is, the active and reactive power interacting between each distribution transformer and the distribution network. The action space is represented as: In the formula, and The first k The active and reactive power of the interaction between the intelligent agent in each distribution area and the distribution network; For the first k The action space of an agent in a certain area during the m-th training round; The state transition function represents the probability that an agent in a given area will transition from one state to the next after observing a new environment and performing a new action. It is expressed by the formula: In the formula, For the intelligent agent in the region in strategy From the previous state Transition to the next state The probability, For the agent to perform actions in the previous state a The strategy function; Execute an action in the previous state of the agent. a The probability of transitioning to the next state; A It is the action space; The reward function is used to measure the value of each agent's transition from the previous state to the new state. The reward function is represented by a negative objective function and a penalty term, as shown in the following expression: In the formula, Let be the reward value for the m-th training round. This is the weighting factor for the voltage deviation in the distribution network; This is the weighting coefficient for distribution network losses; The weighting coefficients for the combined voltage sensitivity and control action; This is the bonus value for voltage deviation. This is the bonus value for distribution network losses. The penalty is a weighted combination of voltage sensitivity and control action; M represents the total number of training rounds. For the first m training round nodes i The voltage value on; This is the reference voltage value; n is the total number of nodes in the distribution network. Let be the resistance value of the branch between node i and node j; and Branch roads ij Active and reactive power transmitted upstream. It is the voltage sensitivity of the k-th transformer area in the m-th training round. For the first k The action space of a regional agent in the m-th training round.

4. The intelligent learning control method for medium and low voltage distribution stations with sensitivity screening as described in claim 1, characterized in that, In S4, the offline training of the agents in each substation area specifically refers to: S4.1 Input the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load in the transformer area, initialize the Actor network and Critic network of each regional agent, and initialize the shared experience playback pool. D ; S4.2 Setting the number of training rounds M Set the initial maximum noise standard deviation. and initial minimum noise standard deviation Set the learning rate for the Actor network. Learning rate of the Critic network and discount factor Initial sensitivity attenuation coefficient ; S4.3, Set the current training round number Reset the power distribution network simulation environment to its initial state; S4.4 Receive the voltage sensitivity of each station area calculated by S2; S4.

5. Dynamically adjust the exploration noise amplitude of each agent in the training phase based on voltage sensitivity, using the following formula: Normalize the voltage sensitivity: In the formula, For normalized voltage sensitivity, This is the minimum voltage sensitivity among all nodes. This is the maximum voltage sensitivity among all nodes; It is the voltage sensitivity of the kth transformer area in the mth training round; The noise amplitude was explored by dynamically adjusting the normalized voltage sensitivity, maximum noise standard deviation, minimum noise standard deviation, and sensitivity attenuation coefficient. In the formula, The dynamically adjusted exploration noise amplitude, It is the maximum noise standard deviation in the m-th training round. It is the minimum noise standard deviation in the m-th training round; It is the sensitivity decay coefficient in the m-th training round; S4.6 The agent in the district produces joint actions based on the current policy network and the dynamically adjusted exploration noise amplitude. The formula is as follows: In the formula, It is the action of the k-th agent in the m-th training round. This function represents the function that restricts the actions of the agent in the control area to within upper and lower limits. In the current state space The policy network below; and These are the upper and lower limits of the agent's actions, respectively; The dynamically adjusted exploration noise amplitude, This represents the dynamically adjusted exploration noise amplitude under a Gaussian distribution. S4.7 After the intelligent agent in the control area performs a joint action, the environment provides a global reward value. and the next state ; S4.8, Transfer the generated samples Stored in the shared experience replay pool D ; S4.9 Learning Rate Based on Critic Network Update the Critic network parameters using the loss function, based on the learning rate of the Actor network. Update the Actor network parameters with the objective function; S4.10 Determine whether the reward function has converged. If it has, exit the loop. Otherwise, increment the training round number by one, dynamically update the noise parameters based on the training round number, and return to step S4.

5.

5. The intelligent learning control method for medium and low voltage distribution stations with sensitivity screening as described in claim 4, characterized in that, In S4.9, the updating of the Critic network parameters and Actor network parameters introduces adaptive weight coefficients based on voltage sensitivity, specifically as follows: Define the sensitivity adaptive weighting coefficients of the intelligent agent in the distribution area. The calculation formula is: In the formula, For the sensitivity adaptive weighting coefficients of the k-th station agent in the m-th training round; This is the normalized voltage sensitivity calculated in S4.5; This is a preset sensitivity amplification factor used to adjust the degree to which the sensitivity affects the gradient; By incorporating the aforementioned sensitivity adaptive weighting coefficients, the loss function of the Critic network is weighted, resulting in the improved expression for the Critic network loss function: In the formula, Let be the loss function of the Critic network; This represents the expected return value. The desired target value; It is the reward value for the m-th training round; Discount factor; The value estimated by the target Critic network for the next state; For the target Actor policy network; The value estimated by the Critic network in the current state; The Critic network parameter update formula is: In the formula, These are the parameters of the Critic network. It is the learning rate of the Critic network. This is the gradient operator. The gradient update formula for the Actor network policy is: In the formula, The gradient of the Actor network policy; This represents the expected return value. This represents the gradient of the Critic network's evaluation of the action, serving as a guide for direction. This represents the influence of Actor network parameters on action output; The formula for updating the Actor network parameters is: In the formula, For Actor network parameters, It is the learning rate of the Actor network.

6. The intelligent learning control method for medium and low voltage distribution stations with sensitivity screening as described in claim 4, characterized in that, In S4.10, the dynamic updating of noise parameters based on the number of training rounds specifically refers to: The maximum noise standard deviation, minimum noise standard deviation, and sensitivity attenuation coefficient are updated using the following formulas: In the formula, The maximum noise standard deviation in the m-th training round. The initial maximum noise standard deviation, Let m be the minimum noise standard deviation of the m-th training round. This represents the initial minimum noise standard deviation; Noise decay rate (rounds) Let be the sensitivity decay coefficient for the m-th training round. This is the initial sensitivity attenuation coefficient. This represents the growth rate of the attenuation coefficient.

7. A sensitivity-screening intelligent learning control system for medium and low voltage distribution platforms, characterized in that, include: The data acquisition module is used to acquire parameter information of the distribution network and historical operating data of distributed photovoltaic and loads within the transformer area; The transformer substation agent model building module is used to calculate the voltage sensitivity of each transformer substation based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load within the transformer substation; the collaborative control process of the distribution network and transformer substation is modeled as a Markov decision process, with each transformer substation as a reinforcement learning agent, and the voltage sensitivity of the transformer substation is incorporated into the state space of multi-agent observation, with the distribution network as the training environment; The training module is used to conduct offline training on each distribution area agent based on the parameter information of the distribution network and the historical operation data of distributed photovoltaic and load in the distribution area. It dynamically adjusts the exploration noise amplitude of each distribution area agent during the training phase according to the voltage sensitivity, and finally obtains the trained distribution area agent. The power allocation module is used to use the trained transformer area agents to control the active and reactive power interaction between each transformer area and the distribution network, so as to control the transformer area voltage.

8. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the sensitivity screening intelligent learning control method for medium and low voltage distribution as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to execute the intelligent learning control method for medium and low voltage distribution stations with sensitivity screening as described in any one of claims 1-6.

Images