Intelligent power distribution network adjustable equipment optimization method and system based on deep reinforcement learning
Through the intelligent distribution network adjustable equipment optimization method based on deep reinforcement learning, the problem of slow response speed and insufficient adjustment capability of adjustable equipment in the distribution network is solved, and the flexibility, reliability and economics of the distribution network are improved, reducing the line loss and voltage limit of the entire network, and improving energy utilization efficiency.
Patent Information
- Application Number
- CN202411798369.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-09
- Publication Date
- 2025-04-04
AI Technical Summary
In the prior art, the adjustable equipment in the distribution network has slow response speed and insufficient adjustment capabilities, making it difficult to effectively solve problems such as voltage fluctuations, current control and reactive power compensation.
The intelligent distribution network adjustable equipment optimization method based on deep reinforcement learning is adopted. By collecting distribution network operating status data, a regulation model of adjustable equipment is established, constraint conditions and objective functions are determined, state space, action space, state transition probability and reward functions are defined, and deep reinforcement learning algorithms are used for optimization.
It significantly improves the flexibility, reliability and economy of the distribution network, reduces the cable loss of the entire network, avoids voltage overload, and improves energy utilization efficiency.
Smart Images
Figure CN119765288A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of optimization of adjustable equipment in a smart distribution network, and specifically relates to a method and system for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning. Background Art
[0002] In the new power system, the distribution network plays a vital role as the last link of power transmission and distribution. In addition to large-scale centralized renewable energy generation as a centralized power source through the transmission network, the distribution network also accommodates a large number of distributed power sources with smaller capacity. The access of these distributed power sources has brought unprecedented challenges to the operation and management of the distribution network, such as voltage fluctuations, power flow control, reactive power compensation and other issues.
[0003] In the face of these challenges, there is an urgent need to use advanced sensors, communication technology, control algorithms and artificial intelligence technology to achieve intelligent monitoring, control and optimization of adjustable equipment in the distribution network, so as to improve the flexibility, reliability and economy of the distribution network. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for optimizing adjustable equipment in an intelligent distribution network based on deep reinforcement learning, so as to solve the technical problems of slow response speed and insufficient regulation capability in the prior art.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] The optimization method of adjustable equipment in smart distribution network based on deep reinforcement learning includes:
[0007] Step 1: Collect the operation status data of the distribution network and obtain feature data through feature extraction;
[0008] Step 2: Establish a control model for adjustable equipment in the distribution network;
[0009] Step 3: Determine the constraint conditions based on the premise of safe and stable operation of the distribution network, and determine the objective function by comprehensively considering the line loss and voltage over-limit situation of the entire distribution network;
[0010] Step 4: Define the state space, action space, state transition probability and reward function, and apply deep reinforcement learning algorithm to optimize the adjustable equipment of the distribution network.
[0011] Furthermore, a control model for adjustable equipment in the distribution network is established. The specific method is as follows:
[0012] An adjustable device control model consisting of common power electronic components, filters, capacitors, and inductors is established. The basic circuit structure of the model consists of n TSC branches containing capacitors and m TCR branches containing inductors. Based on the power that the TSC branch and the TCR branch of the control model can generate, the formula is used to calculate the power of the adjustable device. It indicates the power that the control model can output, where Mp indicates the output power of the control model, w indicates the angular frequency, C indicates the capacitance of the control model, L indicates the reactance of the control model, U indicates the voltage of the control model, and α indicates the delayed trigger angle.
[0013] Furthermore, based on the premise of safe and stable operation of the distribution network, the constraint conditions are determined. The specific method is as follows:
[0014] The constraints include constraints on power generation equipment and constraints on voltage at distribution network nodes;
[0015] Using the inequality Ua(i) min ≤Ua(i)≤Ua(i) max , i∈Ni determines the constraints on the voltage of the power generation equipment, using the inequality Ia(i) min ≤Ia(i)≤Ia(i) max , i∈Ni determines the constraints on the current of the power generation equipment, using the inequality Pa(i) min ≤Pa(i)≤Pa(i) max , i∈Ni determines the constraints on the output power of the power generation equipment, where i represents the i-th power generation equipment, Ni represents the set of all power generation equipment in the distribution network, Ua(i) represents the voltage of the i-th power generation equipment, and Ua(i) max Indicates the voltage upper limit of the i-th generating equipment, Ua(i) min represents the lower voltage limit of the ith power generation equipment, Ia(i) represents the voltage of the ith power generation equipment, Ia(i) max represents the upper limit of the current of the i-th power generation equipment, Ia(i) min represents the lower limit of the current of the ith power generation equipment, Pa(i) represents the output power of the ith power generation equipment, Pa(i) max represents the upper limit of the output power of the i-th power generation equipment, Pa(i) min represents the lower limit of the output power of the i-th power generation equipment;
[0016] Using the inequality minV j ≤V j ≤maxV j , j∈L determines the constraints on the voltage of the distribution network nodes, where j represents the jth node of the distribution network, V j represents the voltage of the jth node, maxV jIndicates the voltage upper limit of the jth node in the distribution network, minV j represents the voltage lower limit of the jth node in the distribution network, and L represents the set of all nodes in the distribution network.
[0017] Furthermore, the objective function is determined by comprehensively considering the line loss and voltage over-limit situation of the entire distribution network. The specific method is as follows:
[0018] Using the formula The objective function is constructed to minimize the line loss of the entire network and avoid voltage exceeding the limit, where j represents the jth node of the distribution network, t represents the tth time period, L represents the set of all nodes in the distribution network, and Lo j (t) represents the line loss of the jth node in the distribution network in the tth time period, ΔV j (t) represents the voltage over-limit value of the jth node in the distribution network in the tth time period, maxV j Indicates the voltage upper limit of the jth node in the distribution network, minV j represents the lower voltage limit of the j-th node in the distribution network, k1 represents the line loss weight coefficient of the entire network, k2 represents the voltage over-limit weight coefficient, λ represents the voltage over-limit coefficient, and e is a constant.
[0019] Furthermore, the state space, action space, state transition probability and reward function are defined as follows:
[0020] The state space is determined based on the feature data contained in the state sample set. These feature data are used as the state input of the deep reinforcement learning model to define the control action of the adjustable device. The control action of the adjustable device is used as the output of the deep reinforcement learning model. The state transition probability is determined based on the change in the objective function caused by the control action of the adjustable device changing the control model parameters. The reward function is set according to the objective function. When the action taken by the model leads to a reduction in the line loss of the entire network or a reduction in the voltage over-limit degree, a positive reward is given; otherwise, a negative reward is given.
[0021] The present invention also provides an intelligent distribution network adjustable equipment optimization system based on deep reinforcement learning, which specifically includes:
[0022] The distribution network data feature extraction module is used to collect the electric energy data of the distribution network in the working state, obtain the feature data of the distribution network in the corresponding time through feature extraction, combine the extracted feature data with the corresponding distribution network operation state, divide the time period into T, and form a state sample set for each time period in chronological order.
[0023] The control model establishment module is used to establish an adjustable device control model composed of common power electronic components, filters, capacitors, and inductors.
[0024] The objective function and constraint condition determination module is used to construct an objective function with the goal of minimizing the line loss of the entire network and avoiding voltage exceeding the limit, and to determine the constraints including constraints on power generation equipment and voltage constraints on distribution network nodes.
[0025] The deep reinforcement learning module is used to define the state space, action space, state transition probability and reward function, select the deep deterministic policy gradient algorithm to establish a deep reinforcement learning model, and output the optimal control action based on the real-time state input. The reward function and model parameters are adjusted according to the actual operation effect.
[0026] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0027] 1. The present invention collects characteristic data of the distribution network, establishes an adjustable equipment control model, constructs objective functions and constraints, and applies deep reinforcement learning algorithms to optimize the adjustable equipment of the distribution network, which can significantly improve the flexibility, reliability and economy of the distribution network. By optimizing the operating status of the adjustable equipment, it helps to reduce the line loss of the entire network, avoid voltage over-limit, and effectively improve the energy utilization efficiency of the distribution network. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0029] Figure 1 A step diagram of a method for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning is shown. DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0031] Embodiment 1, as Figure 1 The method and system for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning specifically include the following steps:
[0032] Step 1: Collect the operating status data of the distribution network and obtain feature data through feature extraction.
[0033] The electric energy data of each network node in the distribution network network structure under working status is collected, and the restriction conditions and electric energy data of the distribution network under working status are integrated. The restriction conditions include: load demand and line capacity limitation. The electric energy data is feature extracted according to the time series to obtain the feature data within the corresponding time. The feature data include: voltage amplitude and voltage phase angle of each node in the distribution network, power of each branch of the distribution network, and line loss of the entire distribution network. The extracted feature data is combined with the corresponding distribution network operation status, and the time period is divided into T to form a state sample set for each time period in chronological order.
[0034] Step 2: Establish a control model for adjustable equipment in the distribution network.
[0035] An adjustable device control model consisting of common power electronic components, filters, capacitors, and inductors is established. The basic circuit structure of the model consists of n TSC branches containing capacitors and m TCR branches containing inductors. The model is used to adjust the voltage and power factor in the distribution network. The power flow is regulated by controlling the voltage of the capacitor and the current of the inductor. Based on the power that the TSC branch and the TCR branch of the control model can emit, the power that the control model can output is determined. The specific formula is as follows:
[0036]
[0037] Wherein Mp represents the output power of the control model, w represents the angular frequency, C represents the capacitance of the control model, L represents the reactance of the control model, U represents the voltage of the control model, and α represents the delayed trigger angle.
[0038] Step 3: Determine the constraint conditions based on the premise of safe and stable operation of the distribution network, and determine the objective function by comprehensively considering the line loss and voltage exceeding the limit of the entire distribution network.
[0039] The characteristics of the line loss of the whole network changing with the load are analyzed, and the voltage exceeding the limit caused by the load fluctuation and line impedance change are considered. The objective function is constructed to minimize the line loss of the whole network and avoid voltage exceeding the limit. The specific formula is as follows:
[0040]
[0041] Where j represents the jth node of the distribution network, t represents the tth time period, L represents the set of all nodes in the distribution network, Lo j (t) represents the line loss of the jth node in the distribution network in the tth time period, ΔV j (t) represents the voltage over-limit value of the jth node in the distribution network in the tth time period, maxV j Indicates the voltage upper limit of the jth node in the distribution network, minV jrepresents the voltage lower limit of the j-th node in the distribution network, k1 represents the line loss weight coefficient of the entire network, k2 represents the voltage over-limit weight coefficient, λ represents the voltage over-limit coefficient, e is a constant, and in this embodiment, e=1 is set.
[0042] To ensure the safe and stable operation of the distribution network and power generation equipment, constraints are set on the distribution network to limit the distribution network and power generation equipment. The constraints include constraints on power generation equipment and constraints on the voltage of distribution network nodes.
[0043] Furthermore, the constraints of the power generation equipment include:
[0044] Ua(i) min ≤Ua(i)≤Ua(i) max , i∈Ni;
[0045] Where i represents the i-th power generation equipment, Ua(i) represents the voltage of the i-th power generation equipment, and Ua(i) max Indicates the voltage upper limit of the i-th generating equipment, Ua(i) min represents the voltage lower limit of the ith generating equipment, and Ni represents the set of all generating equipment in the distribution network.
[0046] Ia(i) min ≤Ia(i)≤Ia(i) max , i∈Ni;
[0047] Where i represents the i-th power generation equipment, Ia(i) represents the voltage of the i-th power generation equipment, and Ia(i) max represents the upper limit of the current of the i-th power generation equipment, Ia(i) min represents the lower limit of the current of the ith power generation equipment, and Ni represents the set of all power generation equipment in the distribution network.
[0048] Pa(i) min ≤Pa(i)≤Pa(i) max , i∈Ni;
[0049] Where i represents the i-th power generation equipment, Pa(i) represents the output power of the i-th power generation equipment, and Pa(i) max represents the upper limit of the output power of the i-th power generation equipment, Pa(i) min represents the lower limit of the output power of the ith power generation equipment, and Ni represents the set of all power generation equipment in the distribution network.
[0050] Furthermore, the constraints on the distribution network node voltage include:
[0051] minV j ≤V j ≤maxV j , j∈L;
[0052] Where j represents the jth node of the distribution network, V j represents the voltage of the jth node, maxV j Indicates the voltage upper limit of the jth node in the distribution network, minV j represents the voltage lower limit of the jth node in the distribution network, and L represents the set of all nodes in the distribution network.
[0053] Step 4: Define the state space, action space, state transition probability and reward function, and apply deep reinforcement learning algorithm to optimize the adjustable equipment of the distribution network.
[0054] The state space is determined based on the feature data contained in the state sample set. These feature data are used as the state input of the deep reinforcement learning model to define the control actions of the adjustable equipment, such as capacitor input or removal, inductor adjustment. These actions are used as the output of the deep reinforcement learning model to adjust the operating state of the distribution network. These actions are recorded as the action space. The state transition probability is determined based on the change of the objective function caused by changing the control model parameters based on the action data. The state transition probability represents the probability of a state transferring to another state after an action is taken in the Markov transition model, which is determined by the environment. The reward function is set according to the objective function. When the action taken by the model leads to a reduction in the line loss of the entire network or a reduction in the voltage over-limit degree, a positive reward is given; otherwise, a negative reward is given.
[0055] A deep deterministic policy gradient algorithm is selected to establish a deep reinforcement learning model. The model is trained based on the characteristic data of the historical distribution network. The performance of the model is optimized by adjusting the model parameters and hyperparameters such as the learning rate. The trained deep reinforcement learning model is applied to the actual distribution network. According to the real-time state input, the model outputs the optimal control action to adjust the operating state of the adjustable equipment. According to the actual operating results, the reward function and model parameters are adjusted to further improve the performance of the model, ultimately realizing an intelligent optimization method for the adjustable equipment in the distribution network.
[0056] Embodiment 2, a smart distribution network adjustable equipment optimization system based on deep reinforcement learning, specifically includes the following contents:
[0057] The distribution network data feature extraction module is used to collect the electric energy data of each network node in the distribution network network structure under working status, integrate the restriction conditions and electric energy data of the distribution network under working status, extract the features of the electric energy data according to the time series, obtain the feature data within the corresponding time, combine the extracted feature data with the corresponding distribution network operation status, divide the time period into T, and form a state sample set for each time period in chronological order.
[0058] The control model establishment module is used to establish an adjustable device control model composed of common power electronic components, filters, capacitors, and inductance elements. The model is used to adjust the voltage and power factor in the distribution network. The power flow is regulated by controlling the voltage of the capacitor and the current of the inductance element. Based on the power that the TSC branch and the TCR branch of the control model can emit, the power that the control model can output is determined. The specific formula is as follows:
[0059]
[0060] Wherein Mp represents the output power of the control model, w represents the angular frequency, C represents the capacitance of the control model, L represents the reactance of the control model, U represents the voltage of the control model, and α represents the delayed trigger angle.
[0061] The objective function and constraint condition determination module is used to analyze the characteristics of the line loss of the entire network changing with the load, as well as the voltage exceeding the limit caused by load fluctuations and line impedance changes, and to construct an objective function with the goal of minimizing the line loss of the entire network and avoiding voltage exceeding the limit. The specific formula is as follows:
[0062]
[0063] Where j represents the jth node of the distribution network, t represents the tth time period, L represents the set of all nodes in the distribution network, Lo j (t) represents the line loss of the jth node in the distribution network in the tth time period, ΔV j (t) represents the voltage over-limit value of the jth node in the distribution network in the tth time period, maxV j Indicates the voltage upper limit of the jth node in the distribution network, minV j represents the voltage lower limit of the j-th node in the distribution network, k1 represents the line loss weight coefficient of the entire network, k2 represents the voltage over-limit weight coefficient, λ represents the voltage over-limit coefficient, e is a constant, and in this embodiment, e=1 is set.
[0064] In order to ensure the safe and stable operation of the distribution network and power generation equipment, constraints are set on the distribution network to restrict the distribution network and power generation equipment. The constraints include constraints on the power generation equipment and constraints on the voltage of the distribution network nodes.
[0065] The deep reinforcement learning module is used to define the state space, action space, state transition probability and reward function, select the deep deterministic policy gradient algorithm to establish a deep reinforcement learning model, train the model based on the characteristic data of the historical distribution network, optimize the model performance by adjusting the model parameters and hyperparameters such as the learning rate, and apply the trained deep reinforcement learning model to the actual distribution network. According to the real-time state input, the model outputs the optimal control action to adjust the operating state of the adjustable equipment. According to the actual operating results, the reward function and model parameters are adjusted to further improve the performance of the model.
[0066] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
[0067] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.
Claims
1. A method for optimizing adjustable equipment in smart distribution networks based on deep reinforcement learning, characterized in that: include: Step 1: Collect the operation status data of the distribution network and obtain feature data through feature extraction; Step 2: Establish a control model for adjustable equipment in the distribution network; Step 3: Determine the constraint conditions based on the premise of safe and stable operation of the distribution network, and determine the objective function by comprehensively considering the line loss and voltage over-limit situation of the entire distribution network; Step 4: Define the state space, action space, state transition probability and reward function, and apply deep reinforcement learning algorithm to optimize the adjustable equipment of the distribution network.
2. The method for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning according to claim 1, characterized in that: Establish a control model for adjustable equipment in the distribution network. The specific method is as follows: An adjustable device control model consisting of common power electronic components, filters, capacitors, and inductors is established. The basic circuit structure of the model consists of n TSC branches containing capacitors and m TCR branches containing inductors. Based on the power that the TSC branch and the TCR branch of the control model can generate, the formula is used to calculate the power of the adjustable device. It indicates the power that the control model can output, where Mp indicates the output power of the control model, w indicates the angular frequency, C indicates the capacitance of the control model, L indicates the reactance of the control model, U indicates the voltage of the control model, and α indicates the delayed trigger angle.
3. The method for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning according to claim 1, characterized in that: The constraints are determined based on the premise of safe and stable operation of the distribution network. The specific method is as follows: The constraints include constraints on power generation equipment and constraints on voltage at distribution network nodes; Using the inequality Ua(i) min ≤Ua(i)≤Ua(i) max , i∈Ni determines the constraints on the voltage of the power generation equipment, using the inequality Ia(i) min ≤Ia(i)≤Ia(i) max , i∈Ni determines the constraints on the current of the power generation equipment, using the inequality Pa(i) min ≤Pa(i)≤Pa(i) max , i∈Ni determines the constraints on the output power of the power generation equipment, where i represents the i-th power generation equipment, Ni represents the set of all power generation equipment in the distribution network, Ua(i) represents the voltage of the i-th power generation equipment, and Ua(i) max Indicates the voltage upper limit of the i-th generating equipment, Ua(i) min represents the lower voltage limit of the ith power generation equipment, Ia(i) represents the voltage of the ith power generation equipment, Ia(i) max represents the upper limit of the current of the i-th power generation equipment, Ia(i) min represents the lower limit of the current of the ith power generation equipment, Pa(i) represents the output power of the ith power generation equipment, Pa(i) max represents the upper limit of the output power of the i-th power generation equipment, Pa(i) min represents the lower limit of the output power of the i-th power generation equipment; Using the inequality minV j ≤V j ≤maxV j , j∈L determines the constraints on the voltage of the distribution network nodes, where j represents the jth node of the distribution network, V j represents the voltage of the jth node, maxV j Indicates the voltage upper limit of the jth node in the distribution network, minV j represents the voltage lower limit of the jth node in the distribution network, and L represents the set of all nodes in the distribution network.
4. The method for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning according to claim 1, characterized in that: The objective function is determined by considering the line loss and voltage over-limit of the entire distribution network. The specific method is as follows: Using the formula The objective function is constructed to minimize the line loss of the entire network and avoid voltage exceeding the limit, where j represents the jth node of the distribution network, t represents the tth time period, L represents the set of all nodes in the distribution network, and Lo j (t) represents the line loss of the jth node in the distribution network in the tth time period, ΔV j (t) represents the voltage over-limit value of the jth node in the distribution network in the tth time period, maxV j Indicates the voltage upper limit of the jth node in the distribution network, minV j represents the lower voltage limit of the j-th node in the distribution network, k1 represents the line loss weight coefficient of the entire network, k2 represents the voltage over-limit weight coefficient, λ represents the voltage over-limit coefficient, and e is a constant.
5. The method for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning according to claim 1, characterized in that: Define the state space, action space, state transition probability and reward function. The specific method is: The state space is determined based on the feature data contained in the state sample set. These feature data are used as the state input of the deep reinforcement learning model. The control actions of the adjustable devices are defined. The control actions of the adjustable devices are used as the output of the deep reinforcement learning model. The state transition probability is determined based on the change in the objective function caused by the control actions of the adjustable devices changing the parameters of the control model. The reward function is set according to the objective function. When the actions taken by the model lead to a reduction in the line loss of the entire network or a reduction in the voltage over-limit degree, a positive reward is given. Otherwise, negative rewards are given.
6. A system for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning, applied to a method for optimizing adjustable equipment in a smart distribution network based on deep reinforcement learning as described in any one of claims 1 to 7, characterized in that: include: The distribution network data feature extraction module is used to collect the electric energy data of the distribution network in the working state, obtain the feature data of the distribution network in the corresponding time through feature extraction, combine the extracted feature data with the corresponding distribution network operation state, divide the time period into T, and form a state sample set for each time period in chronological order. The control model establishment module is used to establish an adjustable device control model composed of common power electronic components, filters, capacitors, and inductors. The objective function and constraint condition determination module is used to construct an objective function with the goal of minimizing the line loss of the entire network and avoiding voltage exceeding the limit, and to determine the constraints including constraints on power generation equipment and voltage constraints on distribution network nodes. The deep reinforcement learning module is used to define the state space, action space, state transition probability and reward function, select the deep deterministic policy gradient algorithm to establish a deep reinforcement learning model, and output the optimal control action based on the real-time state input. The reward function and model parameters are adjusted according to the actual operation effect.