Island micro-grid integrated low-frequency load shedding method fusing various load correlation factors

Abstract

The invention discloses an island micro-grid integrated low-frequency load shedding method fusing multiple types of load correlation factors. An island micro-grid integrated load shedding model with the purpose of simultaneously minimizing the frequency fluctuation amplitude, load shedding cost and system three-phase unbalance degree of an island micro-grid after fault disturbance is constructed; the island micro-grid integrated load shedding model is described as a Markov decision process MDP problem, and an intelligent agent is used for learning an optimal load shedding decision; on the basis of a double-delay depth deterministic policy gradient (TD3) framework, a double-delay depth deterministic policy gradient (DBR-SD3) method integrating softmax and double playback buffers is provided to solve the MDP problem of a Markov decision process; and the fully learned DBR-SD3 can adaptively generate an optimal integrated load shedding decision according to the operating environment of the micro-grid. According to the method, the rapid drop of the system frequency can be prevented, and the problem of three-phase imbalance in system operation is corrected while the power supply reliability of the important load is ensured with relatively low load shedding cost.

Description

Technical Field

[0001] The present invention relates to the field of low-frequency load shedding for isolated island microgrids, and in particular to an integrated low-frequency load shedding method for isolated island microgrids integrating multiple types of load correlation factors. Background Art

[0002] Low-frequency load shedding can prevent the system frequency from dropping rapidly in the event of a fault or power disturbance. It is one of the important means of emergency control of the power system. Especially for isolated microgrids, it is crucial to prevent its frequency from becoming unstable and ensure the safety of the system frequency through low-frequency load shedding strategies. The traditional model-based method for determining the load shedding scheme has the problems of poor scalability and long model solution time, and cannot meet the requirements of instantaneous response of emergency control of the power grid. At present, machine learning-based methods have shown great potential in making control decisions. However, traditional machine learning models rely on high-quality databases, and data processing is relatively complex. Therefore, how to reasonably design an effective low-frequency load shedding method to quickly restore the system frequency is crucial for the safe and stable operation of isolated microgrids.

[0003] In the prior art, the document [1]: "Adaptive under-voltage load shedding scheme using model predictive control" (T.Amraee,AM Ranjbar,and R.Feuillet, "Adaptive under-voltage load shedding scheme using model predictive control," Electr.Power Syst.Res.,vol.81,no.7,pp.1507-1513,Jul.2011.) proposes an adaptive under-voltage load shedding scheme based on model predictive control (MPC). The scheme introduces the concept of voltage and reactive power variable reference, which can alleviate voltage instability in the event of an unexpected fault.

[0004] Reference [2]: "Dynamic multi-stage under frequency load shedding considering uncertainty of generation loss" (DG Mohammad and A. Turaj, "Dynamic multi-stage under frequency load shedding considering uncertainty of generation loss," IET Gener. Transm. Distrib., vol. 11, no. 13, pp. 3202-3209, Sept. 2017.) proposed a dynamic multi-stage under frequency load shedding strategy considering the uncertainty of generation loss. The strategy describes the UFLS problem as a mixed integer linear programming optimization problem to minimize the load shedding. However, the above method relies on the system model, has high requirements on the accuracy of the model, and has a long decision response time.

[0005] There are some papers that apply machine learning to the field of microgrid control. [3]: "Coordinated load shedding control scheme for recovering frequency in islanded microgrids" (C. Wang, S. Mei, and Q. Dong et al., "Coordinated load shedding control scheme for recovering frequency in islanded microgrids," IEEE Access, vol. 8, pp. 215388-215398, 2020.) proposes a coordinated load shedding control strategy for isolated microgrids based on the Q learning framework. This strategy uses the Q value table to record the value of each data during the model training process, which greatly reduces the workload of data processing. However, Q learning will face the dimensionality curse in high-dimensional space, which greatly limits the application of the strategy.

[0006] At present, there are some papers that solve this problem by integrating deep learning and reinforcement learning. The paper [4]: ​​"Distributional deep reinforcement learning-based emergency frequency control" (J. Xie and W. Sun, "Distributional deep reinforcement learning-based emergency frequency control," IEEE Trans. Power Syst., vol. 37, no. 4, pp. 2720-2730, Jul. 2022.) proposes a frequency control method based on multi-agent deep deterministic policy gradient (DDPG). This method can handle continuous states and actions, and adaptively derive the optimal coordinated control strategy of multiple frequency modulation controllers. However, DDPG itself is sensitive to parameter settings, and there is a problem that the learned strategy fails due to overestimation of Q value.

[0007] At the same time, in addition to a large number of three-phase loads, there are also some single-phase source-load-storage in the actual microgrid system. These single-phase components will cause the system to have three-phase imbalance, affecting the power supply quality of the microgrid. Reference [5]: "Control strategy of unintentional islanding transition with high adaptability for three / single-phase hybrid multimicrogrids" (C.Wang, S.Chu, and H.Yu, et al., "Control strategy of unintentional islanding transition with high adaptability for three / single-phase hybrid multimicrogrids," Int.J.Electr.Power Energy Syst., vol.136, 107724, Mar.2022.) Based on the DQN algorithm, a low-frequency load reduction strategy is formulated. At the same time, the strategy takes into account the three-phase imbalance of the system and proposes to avoid the three-phase imbalance problem of the system during the passive transfer of the microgrid by sorting the virtual three-phase combination of heterogeneous source-load-storage. However, the above research first obtains the contribution value of the load, and then constructs a load reduction action set based on the contribution value and executes the load reduction action. This method of evaluating first and then reducing the load will increase the response time of the system load reduction action, thereby affecting the system frequency recovery effect. Summary of the invention

[0008] In order to solve the above technical problems, the present invention provides an integrated low-frequency load reduction method for an isolated microgrid that integrates multiple types of load-related factors. This method can prevent the rapid drop in system frequency, ensure the power supply reliability of important loads at a lower load reduction cost, and correct the three-phase imbalance problem in system operation.

[0009] The technical solution adopted by the present invention is:

[0010] An island microgrid integrated low-frequency load reduction method integrating multiple types of load correlation factors comprises the following steps:

[0011] Step 1: Construct an integrated load reduction model for the isolated microgrid with the goal of minimizing the frequency fluctuation amplitude, load reduction cost, and system three-phase imbalance of the isolated microgrid after the fault disturbance;

[0012] Step 2: Describe the island microgrid integrated load reduction model as a Markov decision process MDP problem, and use the intelligent agent to learn the optimal load reduction decision;

[0013] Step 3: Based on the double-delayed deep deterministic policy gradient (TD3) framework, a double-delayed deep deterministic policy gradient (DBR-SD3) method integrating softmax and double replay buffer is proposed to solve the Markov decision process MDP problem. This method uses the softmax function to avoid the Q-value underestimation bias of TD3, and significantly improves the learning efficiency and convergence speed of the agent through the double replay buffer mechanism.

[0014] Step 4: After sufficient learning, DBR-SD3 can adaptively generate the optimal integrated load reduction decision according to the microgrid operating environment.

[0015] In step 1, the construction of the isolated microgrid integrated load reduction model is as follows:

[0016] 1) Load frequency characteristics:

[0017] Considering the strong coupling between system frequency and active power, the present invention ignores the influence of load reactive power and voltage change when constructing the load frequency characteristic model. The load frequency characteristic model is:

[0018] P L =ε 0 P L0 +ε 1 P L0 (f / f 0 )+ε 2 P L0 (f / f 0 ) 2 +

[0019] …+ε n P L0 (f / f 0 ) n

[0020] Among them, P L Indicates the active power absorbed by the load at frequency f; P L0 Indicates the rated active power of the load; ε 0 Indicates that the load proportional to the power of the system frequency is P L0 The proportion of 1 Indicates that the load proportional to the power of the system frequency is P L0 The proportion of 2 Indicates that the load proportional to the square of the system frequency is P L0 The proportion of n Indicates that the load proportional to the system frequency nth power is at P L0 The proportion of 0 is the rated frequency of the system; n means there are n types of loads in total.

[0021] Differentiate the above equation and convert it into per-unit form:

[0022] K L =dP L * / df * =ε 1 +2ε 2 f * +…+nε n f *(n-1)

[0023] Among them, K L Indicates the frequency regulation effect coefficient of the load; P L * Indicates the per-unit value of the active power absorbed by the load at frequency f; f * Indicates the per-unit value of frequency f; f *(n-1) represents f *(n-1) per unit value.

[0024] When the system frequency drops and triggers the low-frequency load reduction action, the load with a small frequency regulation effect coefficient should be removed first, so as to help the system active power restore balance as soon as possible while reducing the load.

[0025] 2) Load shedding cost:

[0026] Normally, loads are divided into three categories: primary load, secondary load, and tertiary load. However, this simple classification method ignores the differences in demand for different types of loads by electricity users at different time scales. In this regard, the present invention proposes a load cost coefficient to measure the load shedding priority at different times. The cost coefficient F of load i at time t is i,t for:

[0027] F i,t =C i,t ωh i

[0028] Among them, C i,t represents the power demand of load i at time t, which is determined by the type of load. In the present invention, the load is divided into three categories: industrial, commercial, and residential, and the demand of each type of load is different; ω represents the load level weight, which is divided into three levels: I, II, and III according to the social and economic losses caused by the power failure of the load, corresponding to 100, 10, and 1 respectively; h i Indicates the load reduction loss coefficient.

[0029] 3) Load three-phase unbalanced characteristics:

[0030] Since the hybrid microgrid contains a large number of single-phase devices, there will be a phenomenon of power three-phase imbalance on the interconnection line during the operation of the microgrid. Therefore, the present invention takes the three-phase imbalance characteristics of the load into the load reduction related factors, and the three-phase imbalance calculation formula of the isolated island microgrid is as follows:

[0031]

[0032] Among them, Unbalance is the three-phase imbalance of the isolated microgrid; I 1 is the root mean square root of the positive sequence component of the three-phase current; I 2 is the square root of the negative sequence component of the three-phase current; P A , P B , P C is the three-phase active power; Q A , Q B , Q C is the three-phase reactive power; S L is the positive sequence apparent power; S L2 is the negative sequence apparent power; P L2 is the negative sequence active power; Q L2 is the negative sequence reactive power.

[0033] Based on the above selected load shedding related factors, the present invention takes the frequency fluctuation amplitude, load shedding cost and three-phase imbalance of the system after load shedding as the minimum as the objective function of low-frequency load shedding:

[0034]

[0035] Among them, f ∧ and f ∨ They represent the peak and valley values ​​in the frequency recovery process of the isolated microgrid respectively; m represents the number of loads in the isolated microgrid system; Represents the amount of load i removed at time t.

[0036] In order to ensure that the low-frequency load reduction meets the operation requirements of the isolated island microgrid system, the present invention sets the following constraints:

[0037] 1) Power flow constraints:

[0038]

[0039] Among them, P i,t , Q i,t Respectively represent the active power and reactive power of the node at time t; U i,t , U j,t Respectively represent the voltage amplitude of node i and node j at time t; θ ij,t represents the voltage phase difference between node i and node j at time t; G ij , B ijdenote the conductance and susceptance between nodes i and j respectively;

[0040] 2) Load reduction constraints:

[0041] P i cut,min ≤P i cut ≤P i cut,max

[0042] Among them, P i cut,min , P i cut,max They represent the minimum and maximum values ​​of the load shedding of node i respectively; P i cut Indicates the load shedding value of node i.

[0043] 3) Frequency constraints:

[0044] f min ≤f≤f max

[0045] Among them, f min 、f max They respectively represent the minimum and maximum frequencies allowed for normal system operation.

[0046] 4) Three-phase imbalance constraint:

[0047] Unbalance t ≤15%.

[0048] In step 2, the action decision of integrated load shedding in the isolated microgrid integrated load shedding model proposed in the present invention is only related to the current state of the microgrid, and has nothing to do with the action and state corresponding to the previous time, satisfying the Markov property. Therefore, the present invention describes the load shedding model as an MDP;

[0049] MDP is mainly composed of five tuples (S, A, R, P, γ), where S represents the state space perceived by the agent; A represents the action space taken by the agent; R represents the reward function; P represents the state transition probability; and γ represents the discount factor. The MDP formula established by the island microgrid integrated load reduction model is as follows:

[0050] (1) State space s:

[0051] In order to reflect the operating characteristics of the isolated microgrid as much as possible, the output of the distributed generation unit P t DG , load real-time power P t L , microgrid real-time frequency f, frequency change rate ROCOF tis the environmental information element observed by the agent. For any time t, the state s t It is expressed as:

[0052] s t = {P t DG ,P t L ,ROCOF t}

[0053] (2) Action space a:

[0054] The present invention considers the reduction power P of the node i load i cut is an action, then the action space a of the isolated island microgrid system at time t is t It can be expressed as:

[0055]

[0056] (3) Reward r:

[0057] The goal of the integrated low-frequency load reduction proposed in the present invention is to ensure that the frequency of the isolated microgrid after the fault returns to the normal operating level with the minimum frequency fluctuation amplitude and load reduction cost, while the three-phase imbalance of the system is as small as possible. At the same time, considering the set constraints, the reward value of the intelligent agent at time t is expressed as:

[0058]

[0059] Among them, α 1 , α 2 , α 3 They represent the coefficients of frequency fluctuation amplitude, load reduction cost and system three-phase imbalance in the reward function respectively; t represents the penalty term. When any constraint is not satisfied, the penalty value is -1000, otherwise it is 0. m represents that there are m types of loads in the isolated microgrid.

[0060] In step 3, a DBR-SD3 method is proposed to solve the MDP problem, which specifically includes:

[0061] 3.1: By introducing softmax into the TD3 algorithm to reduce the calculation error of the Q-value function, it not only avoids the overestimation problem of the DDPG algorithm, but also effectively alleviates the underestimation bias of the TD3 algorithm, and achieves accurate estimation of the value function. Represents the state of the b-th historical experience sample at time t+1; represents the action of the bth historical experience sample at time t+1; It represents the Q value obtained after passing the softmax operator. The specific formula is:

[0062]

[0063] in, Indicates the smaller Q value estimated by the two Critic networks; represents the probability density function that satisfies the Gaussian distribution; β represents the operating parameter of softmax; E[] represents the expectation.

[0064] After the softmax operator is introduced, the target Q value The calculation formula is:

[0065]

[0066] Among them, r t b is the reward function.

[0067] 3.2: A DBR mechanism is designed based on the TD3 framework. This mechanism stores empirical data according to their importance and extracts part of the empirical data from the two buffers as training samples according to different probabilities during training, which improves training efficiency and convergence speed.

[0068] The TD3 framework refers to the priority experience replay mechanism in TD3. TD3 stores the experience data obtained through exploration in the buffer by setting a replay buffer, and randomly extracts small batches of data from the buffer as samples using a random sampling method to train the agent. TD3 assigns a priority weight value to each experience sample according to its importance to model training. A larger priority weight value leads to a higher probability of being sampled. The priority p of the sample data is measured by the temporal difference error. For example, the priority of the kth sample is:

[0069]

[0070] p k represents the priority of the kth sample, represents the target Q value, Current Q value, represents the state of the kth sample at time t, Represents the action at the kth sample time t.

[0071] The probability p of the kth sample being sampled k 'for:

[0072]

[0073] Specifically, under the DBR mechanism, experience data is calculated based on its immediate reward rt In buffer D 1 , D 2 Considering the delay of reward feedback in reinforcement learning, a temporary experience playback buffer D with a capacity of M is set up 0 Used to store the adjacent M pieces of experience data. 0 When the storage capacity of the empirical data reaches the upper limit, calculate D 0 Average instant reward of all experience data in And according to the order of deposit time, D 0 The data in is judged, if its reward value is r t Greater than or equal to Then store it in buffer D 1 , otherwise stored in cache D 2 Then the new experience data is stored in D 0 Until the entire learning process is completed. 0 The remaining experience data is based on the reward average Stored in D 1 , D 2 In, clear D 0 .

[0074] According to the empirical classification storage principle, D 1 The sample data stored in D 2 High. In the training process, we hope to get a higher probability from D 1 Extract more valuable experience from buffer D 1 The priority experience replay method is used. At the same time, in order to ensure the diversity of sample data, a small batch D should also be extracted. 2 The sample data with small immediate rewards and low importance constitutes small batch sampling. The specific sampling method is shown in the following formula:

[0075]

[0076] Among them, M 1 、M 2 Respectively represent the buffer D 1 , D 2 The number of sample data extracted from the buffer D 1 The sampling rate of the sample drawn in ; m represents the number of samples in the small batch.

[0077] Prioritized experience playback is to extract experience data with higher value in the buffer first. The priority p of sample data is measured by the time difference error. For example, the priority of the kth sample is:

[0078]

[0079] Where: p k represents the priority of the kth sample, represents the target Q value, Current Q value, represents the state of the kth sample at time t, Represents the action at the kth sample time t.

[0080] In step 4, the integrated load shedding model is trained offline based on the DBR-SD3 method, and the trained integrated load shedding model is used to generate the optimal integrated load shedding decision;

[0081] 1) Offline training:

[0082] Step 1: Initialize the Critic network and Actor network parameters w, θ and the target network parameters w', θ';

[0083] Critic Network Q w The iterative optimization of parameter w is achieved by minimizing the loss function L(w), which is expressed as:

[0084]

[0085] Where: m represents the number of small batch samples drawn from the buffer; b represents the bth historical experience sample; Represents the target Q value.

[0086] TD3 uses two Critic networks on the DDPG framework to estimate the Q function and selects a smaller Q value as the estimated value, effectively avoiding the problem of overestimation of the Q value. The calculation formula is:

[0087]

[0088] in, By the target policy network π θ′ Calculated,

[0089]

[0090] Where: δ represents the introduced random noise based on normal distribution; -c and c represent the upper and lower bounds of the noise respectively.

[0091] Critic Network Q w The parameter w is updated according to the gradient theory, and the specific formula is:

[0092]

[0093] Where: μ w Represents the Critic network learning rate.

[0094] Actor Network π θ The gradient descent method is used to optimize the parameter θ, that is:

[0095]

[0096] According to the deterministic policy gradient, update π θ Parameter θ:

[0097]

[0098] Where: μ θ Represents the Actor network learning rate.

[0099] In order to reduce the deviation of the policy network update caused by the instability of Q value, TD3 adjusts the network update frequency so that the Critic network is updated at a higher frequency and the Actor network is updated at a lower frequency. Specifically, the Actor network parameters are updated after the Critic network completes d updates to reduce the cumulative error. The Critic network and Actor network parameter updates are performed in the form of soft updates.

[0100] w n ′=τw n +(1-τ)w n '

[0101] θ′=τθ+(1-τ)θ′

[0102] Where: τ represents the soft update coefficient.

[0103] Step 2: Initialize the exploration noise and obtain the initial environmental state s of the isolated island microgrid 0 ;

[0104] Step 3: According to the state s t Select actions and add noise;

[0105] Step 4: Execute action a t , get reward r t and the next state s t+1 ;

[0106] Step 5: Store the experience data into the temporary buffer D 0 , and store the experience data in the buffer D according to the classification storage rules 1 and D 2 middle;

[0107] Step 6: Calculate the value from D 1 The time difference error of the samples extracted from the update sample priority p k ;

[0108] Step 7: Calculate the next action a based on the Critic network t+1 ;

[0109] Step 8: Use softmax to calculate the target Q value y for each sample t ;

[0110] Step 9: Update the Critic network parameters w, Actor network parameters θ, and target network parameters w′, θ′;

[0111] Step 10: Loop step,

[0112] Step 11: Repeat Step 3 to Step 9 until all training sessions are completed;

[0113] Step 12: Repeat Step 2-Step 9 until all rounds of training are completed.

[0114] 2) Online application:

[0115] Step 1: The current state value of the island microgrid system plus the random error value is used as the comprehensive state input to the Actor network;

[0116] Step 2: Generate the optimal integrated load reduction decision;

[0117] The optimal integrated load reduction decision means that when the frequency of the isolated microgrid decreases, the system frequency is restored to a normal level after the failure of the isolated microgrid at the minimum load reduction cost, taking into account the load frequency regulation effect and load importance, and minimizing the three-phase imbalance problem of the system caused by the failure or low-frequency load reduction.

[0118] step3: Complete the load reduction operation.

[0119] The present invention provides an island microgrid integrated low-frequency load reduction method integrating multiple types of load correlation factors, and the technical effects are as follows:

[0120] 1) The strategy proposed in this invention fully considers the impact of multiple types of load-related factors on the system load shedding process, and combines the two separate processes of load assessment and low-frequency load shedding into one, proposing a new integrated load shedding mode, which overcomes the shortcomings of long response time caused by insufficient consideration of load factors and the independence of load assessment and load shedding decisions in current load shedding decisions.

[0121] 2) The strategy proposed in this invention directly determines the optimal low-frequency load reduction strategy through intelligent learning. The application scenario of the proposed strategy is not limited by the initial three-phase imbalance of the system, and can quickly restore the system frequency while correcting the unbalanced operation of the system.

[0122] 3) This paper proposes a new deep reinforcement learning method DBR-SD3 for generating the optimal load shedding strategy. Different from the traditional deep reinforcement learning method, the proposed DBR-SD3 achieves accurate estimation of Q value during parameter training by integrating softmax in TD3. In order to further improve the quality of the optimal strategy, a dual replay buffer mechanism is adopted to improve the strategy learning speed and convergence stability.

[0123] 4) The optimal low-frequency load reduction method generated by the present invention can prevent the rapid drop of system frequency through integrated load reduction decision-making, and ensure the reliability of power supply to important loads at a lower load reduction cost while correcting the three-phase imbalance problem in system operation. In addition, the method has stronger robustness and adaptability when facing a complex microgrid system environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0124] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0125] Figure 1 Model diagram of the improved IEEE-37 node island microgrid system built for the present invention.

[0126] Figure 2 This is a graph showing the average reward changes during the training process for the DBR-SD3 method proposed in this invention and other DRL methods.

[0127] Figure 3 It is a frequency recovery waveform diagram of the isolated island microgrid system under the integrated load reduction method proposed in the present invention and other load reduction methods.

[0128] Figure 4 This is a flow chart of the low-frequency load reduction method of the present invention. DETAILED DESCRIPTION

[0129] The present invention will be further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0130] An integrated low-frequency load shedding method for isolated microgrids that integrates multiple types of load correlation factors is proposed. Firstly, an integrated load shedding model for isolated microgrids is constructed with the goal of minimizing the frequency fluctuation amplitude, load shedding cost, and system three-phase imbalance of the isolated microgrid after fault disturbance. Secondly, the integrated load shedding model is described as a Markov decision process (MDP), and the optimal load shedding decision is learned by an intelligent agent. Then, a double-delayed deep deterministic policy gradient (DBR-SD3) method integrating softmax and double replay buffer is developed based on the double-delayed deep deterministic policy gradient (TD3) framework to solve the MDP problem. This method uses the softmax function to avoid the Q-value underestimation bias of TD3, and significantly improves the learning efficiency and convergence speed of the intelligent agent through the double replay buffer mechanism. Finally, the fully learned DBR-SD3 can adaptively generate the optimal low-frequency load shedding strategy according to the microgrid operating environment.

[0131] Figure 1 The improved IEEE-37 node island microgrid system model diagram built for the present invention. The model includes 9 DGs, 9 energy storage batteries (BES) and 19 loads. Among them, LD1-10 are three-phase loads, LD11-19 are single-phase loads, and the loads are divided into levels I, II, and III according to their importance, and are also divided into industrial loads, commercial loads, and residential loads according to the type of electricity consumption.

[0132] Figure 2 This is a graph showing the average reward changes of the DBR-SD3 method and other DRL methods during the training process. Figure 2 It can be seen that the average reward of the DBR-SD3 method proposed in the present invention is close to the optimal after about 400 training rounds, and it reaches convergence 200 / 900 / 1000 rounds earlier than TD3 / DDPG / DQN respectively. This shows that the speed of DBR-SD3 learning the optimal load reduction control strategy is about 1.5 / 3.25 / 3.5 times that of TD3 / DDPG / DQN respectively. In addition, the average reward obtained by DBR-SD3 when it converges is higher than that of TD3 / DDPG / DQN algorithms. This is mainly because the strategy proposed in the present invention greatly improves the learning speed and convergence performance of the intelligent agent by introducing softmax and dual replay buffer mechanisms into the TD3 algorithm, so that the intelligent agent can adaptively learn a more stable control strategy.

[0133] Figure 3 This is a frequency recovery waveform diagram of the isolated island microgrid system after the load shedding action of the integrated load shedding method proposed in the present invention and other load shedding methods.

[0134] Method 1 is the integrated load reduction method proposed by the present invention;

[0135] Method 2 is an adaptive load shedding method based on MPC;

[0136] Method 3 is an emergency load reduction strategy based on distributed DRL;

[0137] Method 4 is a low-frequency load reduction strategy based on DQN.

[0138] The fluctuation amplitude of the system frequency during the recovery period under methods 2, 3, and 4 are 0.75Hz, 0.52Hz, and 0.6Hz, respectively. In comparison, the fluctuation range of the system frequency during the recovery period under strategy 1 is 49.66Hz-50.1Hz, and the frequency fluctuation amplitude is only 0.44Hz, which is 41.33%, 15.38%, and 26.67% lower than methods 2, 3, and 4, respectively. This is because method 1 takes into account the load frequency regulation effect coefficient in the integrated load reduction process, and preferentially removes loads with small frequency regulation effect coefficients, effectively suppressing the decrease in system frequency, so that the system frequency can be restored to a normal state more stably. At the same time, the frequency recovery time of the four methods is 0.21s, 0.291s, 0.225s, and 0.239s, respectively. The frequency recovery time of the method proposed in the present invention is the shortest. This is because the proposed method introduces softmax into TD3 and increases its playback buffer to three to buffer and classify and store experience samples. It is easier to obtain samples with high reference value during the training process, the intelligent agent makes decisions faster, and the system frequency recovers faster. In addition, the present invention combines the two stages of comprehensive load evaluation and load reduction decision-making into one, and directly generates an integrated load reduction decision through the trained intelligent agent, which reduces the response time of the load reduction decision and thus shortens the frequency recovery time.

[0139] Table 1 Load reduction cost and three-phase imbalance of isolated island microgrid system

[0140]

[0141] Table 1 shows the load reduction cost and three-phase unbalance degree under the integrated load reduction method proposed by the present invention and other load reduction methods. Figure 3The load shedding cost data in the table show that the load shedding cost of method 4 is the highest, 4.07% and 5.81% higher than methods 2 and 3, respectively. This is because method 4 takes into account the impact of the three-phase imbalance of the system, and is subject to certain load shedding distribution constraints during the load shedding process, and must remove a virtual combination of approximately balanced loads on the three phases. Furthermore, although method 1 also takes into account the impact of the three-phase imbalance of the system, its load shedding process also takes into account the demand differences of different types of loads, and removes a batch of loads with the lowest cost coefficient in the fault scenario, making the load shedding cost of method 1 still the lowest among the four strategies, which is 17.3%, 15.92%, and 20.54% lower than methods 2, 3, and 4, respectively. From the three-phase imbalance data in the table, it can be seen that the three-phase imbalance of methods 2 and 3 after the system is in stable operation is 19.15% and 26.68%, respectively, which is higher than methods 1 and 4. This is because these two methods do not consider the three-phase imbalance during the load shedding process. Method 4 uses load combination to construct single-phase load into virtual three-phase load, and then preferentially removes virtual three-phase load with smaller imbalance in the load shedding process to achieve the purpose of reducing the three-phase imbalance of the load system. This load shedding method has a relatively ideal effect on the three-phase balance of the system before the fault. When the system is initially in an unbalanced state, the imbalance of method 4 reaches 12.34% after removing a group of balanced virtual load combinations. In contrast, the proposed strategy 1 reasonably distributes the load shedding amount on each sub-phase by considering the integrated load shedding method of the three-phase imbalance of the load, corrects the three-phase imbalance of the system to 0, and completely avoids the impact of three-phase imbalance on system loss and load power quality.

Claims

1. An integrated low-frequency load reduction method for isolated microgrids integrating multiple load-related factors, characterized in that The following steps are involved: Step 1: Construct an integrated load reduction model for the isolated microgrid with the goal of minimizing the frequency fluctuation amplitude, load reduction cost, and system three-phase imbalance of the isolated microgrid after the fault disturbance; Step 2: Describe the island microgrid integrated load reduction model as a Markov decision process MDP problem; Step 3: Based on the double-delayed deep deterministic policy gradient (TD3) framework, a double-delayed deep deterministic policy gradient (DBR-SD3) method integrating softmax and double replay buffer is proposed to solve the Markov decision process MDP problem; Step 4: Adaptively generate the optimal integrated load reduction decision according to the microgrid operating environment.

2. According to claim 1, a method for low-frequency load shedding of an isolated microgrid integrating multiple load-related factors, characterized in that: In step 1, the construction of the isolated microgrid integrated load reduction model is as follows: 1) The frequency characteristic model of the load is: P L =ε0P L0 +ε1P L0 (f / f0)+ε2P L0 (f / f0) 2 +…+e n P L0 (f / f0) n Among them, P L Indicates the active power absorbed by the load at frequency f; P L0 Represents the rated active power of the load; ε0 represents the load proportional to the zeroth power of the system frequency at P L0 ε1 represents the load proportional to the power of the system frequency in P L0 ε2 represents the load proportional to the square of the system frequency in P L0 The proportion of n Indicates that the load proportional to the system frequency nth power is at P L0 f0 is the rated frequency of the system; n means there are n types of loads; Differentiate the above equation and convert it into per-unit form: K L sdP L * / df * 1+2ε2f * +...+nε n f *(n-1) Among them, K L Indicates the frequency regulation effect coefficient of the load; P L * Indicates the per-unit value of the active power absorbed by the load at frequency f; f * Indicates the per-unit value of frequency f; f *(n-1) represents f *(n-1) The per unit value of 2) The load cost coefficient is used to measure the load shedding priority at different times. The cost coefficient F of load i at time t is i,t for: F i,t =C i,t ωh i Among them, C i,t represents the power demand of load i at time t, which is determined by the type of load; loads are divided into three categories: industrial, commercial, and residential, and the demand of each type of load is different; ω represents the load level weight, which is divided into three levels: I, II, and III according to the social and economic losses caused by load power failure; h i Indicates the load shedding loss coefficient; 3) Taking the three-phase unbalanced characteristics of the load into account in the load shedding related factors, the three-phase unbalance degree calculation formula of the isolated island microgrid is as follows: Among them, Unbalance is the three-phase imbalance of the isolated microgrid; I1 is the mean square root of the positive sequence component of the three-phase current; I2 is the mean square root of the negative sequence component of the three-phase current; P A , P B , P C is the three-phase active power; Q A , Q B , Q C is the three-phase reactive power; S L is the positive sequence apparent power; S L2 is the negative sequence apparent power; P L2 is the negative sequence active power; Q L2 is the negative sequence reactive power.

3. According to claim 2, a method for low-frequency load shedding of isolated microgrids integrating multiple load-related factors, characterized in that: Based on the selected load shedding related factors, the frequency fluctuation amplitude, load shedding cost and three-phase imbalance of the system after load shedding are minimized at the same time as the objective function of low-frequency load shedding: Among them, f ∧ and f ∨ They represent the peak and valley values ​​in the frequency recovery process of the isolated microgrid respectively; m represents the number of loads in the isolated microgrid system; Represents the amount of load i removed at time t.

4. According to claim 3, a method for low-frequency load shedding of isolated microgrid integrated with multiple load-related factors is characterized by: In order to ensure that the low-frequency load reduction meets the operation requirements of the island microgrid system, the following constraints are set: 1) Power flow constraints: Among them, P i,t , Q i,t Respectively represent the active power and reactive power of the node at time t; U i,t , U j,t Respectively represent the voltage amplitude of node i and node j at time t; θ ij,t represents the voltage phase difference between node i and node j at time t; G ij , B ij denote the conductance and susceptance between nodes i and j respectively; 2) Load reduction constraints: in, They represent the minimum and maximum values ​​of the load shedding of node i respectively; represents the load shedding value of node i; 3) Frequency constraints: f min ≤f≤f max Among them, f min 、f max They represent the minimum and maximum frequencies allowed for normal system operation respectively; 4) Three-phase imbalance constraint: Unbalance t ≤15%。 5. According to claim 1, an isolated microgrid integrated low-frequency load reduction method integrating multiple types of load correlation factors is characterized by: In step 2, the action decision of integrated load shedding in the isolated microgrid integrated load shedding model is only related to the current state of the microgrid, and has nothing to do with the action and state corresponding to the previous time, satisfying the Markov property; therefore, the load shedding model is described as an MDP; The MDP is composed of a five-tuple (S, A, R, P, γ), where S represents the state space perceived by the agent; A represents the action space taken by the agent; R represents the reward function; P represents the state transition probability; γ represents the discount factor; the MDP formula established by the isolated microgrid integrated load reduction model is as follows: (1) State space s: In order to reflect the operating characteristics of the isolated microgrid, the output of distributed power units is selected Load real-time power Microgrid real-time frequency f, frequency change rate ROCOF t is the environmental information element observed by the agent; for any time t, the state s t It is expressed as: (2) Action space a: Consider the power reduction of node i load is an action, then the action space a of the isolated island microgrid system at time t is t It is expressed as: (3) Reward r: The goal of integrated low-frequency load shedding is to ensure that the frequency of the isolated microgrid after the fault returns to the normal operating level with the minimum frequency fluctuation amplitude and load shedding cost, while the three-phase imbalance of the system is as small as possible. Taking into account the set constraints, the reward value of the agent at time t is expressed as: Among them, α1, α2, and α3 represent the coefficient factors of frequency fluctuation amplitude, load reduction cost, and system three-phase imbalance in the reward function respectively; χ t represents the penalty term. When any constraint is not satisfied, the penalty value is -1000, otherwise it is 0. m represents that there are m types of loads in the isolated microgrid.

6. According to claim 1, a method for low-frequency load shedding of an isolated microgrid integrating multiple load-related factors, characterized in that: In step 3, a DBR-SD3 method is proposed to solve the MDP problem, which specifically includes: 3.1: Introduce softmax into the TD3 algorithm to reduce the calculation error of the Q value function, and obtain it through importance sampling Represents the state of the b-th historical experience sample at time t+1; represents the action of the bth historical experience sample at time t+1; It represents the Q value obtained after passing the softmax operator. The specific formula is: in, Indicates the smaller Q value estimated by the two Critic networks; represents the probability density function that satisfies the Gaussian distribution; β represents the operating parameter of softmax; After the softmax operator is introduced, the target Q value The calculation formula is: in, is the reward function; 3.2: A DBR mechanism is designed based on the TD3 framework. The mechanism stores the empirical data according to their importance and extracts part of the empirical data from the two buffers as training samples according to different probabilities during training. The TD3 framework refers to the priority experience replay mechanism in TD3. TD3 stores the experience data obtained through exploration in the buffer by setting a replay buffer, and randomly extracts small batches of data from the buffer as samples by random sampling to train the agent. TD3 assigns a priority weight value to each experience sample according to its importance to model training. The larger the priority weight value, the higher the probability of being sampled. The priority p of the sample data is measured by the temporal difference error. For example, the priority of the kth sample is: p k represents the priority of the kth sample, represents the target Q value, Current Q value, represents the state of the kth sample at time t, represents the action of the kth sample at time t; The probability p of the kth sample being sampled k 'for:

7. According to claim 6, an isolated microgrid integrated low-frequency load reduction method integrating multiple types of load correlation factors is characterized by: Under the DBR mechanism, experience data is calculated based on its immediate reward r t Classify and store in buffers D1 and D2; Considering the delay of reward feedback for reinforcement learning, set up a temporary experience playback buffer D0 with a capacity of M to store the M adjacent experience data; When the storage capacity of experience data in D0 reaches the upper limit, calculate the average instant reward of all experience data in D0 And judge the data in D0 in order of deposit time. If its reward value is r t Greater than or equal to If the learning process is completed, the remaining experience data in D0 will be stored in the buffer D1, otherwise it will be stored in the buffer D2; then the new experience data will continue to be stored in D0 until the entire learning process is completed; finally, the remaining experience data in D0 will be stored in the buffer D1 according to the reward mean. Store them in D1 and D2 respectively, and clear D0; According to the principle of experience classification storage, the sample data stored in D1 is more valuable than D2. During the training process, we hope to extract more valuable experience from D1 with a greater probability, so the priority experience playback method is used for buffer D1. At the same time, in order to ensure the diversity of sample data, sample data with small immediate rewards and low importance are extracted from the small batch D2 to form a small batch sampling. The specific sampling method is shown in the following formula: Where M1 and M2 represent the number of sample data extracted from buffers D1 and D2 respectively; η∈[0,1] represents the sampling rate of samples extracted from buffer D1; m represents the number of samples in a small batch; Prioritized experience playback is to extract experience data with higher value in the buffer first. The priority p of sample data is measured by the time difference error. The priority of the kth sample is: Where: p k represents the priority of the kth sample, represents the target Q value, Current Q value, represents the state of the kth sample at time t, Represents the action at the kth sample time t.

8. According to claim 1, a method for low-frequency load shedding of an isolated microgrid integrating multiple load-related factors, characterized in that: In step 4, the integrated load shedding model is trained offline based on the DBR-SD3 method, and the trained integrated load shedding model is used to generate the optimal integrated load shedding decision; 1) Offline training: Step 1: Initialize the Critic network and Actor network parameters w, θ and the target network parameters w', θ'; Step 2: Initialize the exploration noise and obtain the initial environmental state s0 of the isolated island microgrid; Step 3: According to the state s t Select actions and add noise; Step 4: Execute action a t , get reward r t and the next state s t+1 ; Step 5: Store the experience data into the temporary buffer D0, and store the experience data into the buffers D1 and D2 respectively according to the classification storage rules; Step 6: Calculate the time difference error of the samples extracted from D1 and update the sample priority p k ; Step 7: Calculate the next action a based on the Critic network t+1 ; Step 8: Use softmax to calculate the target Q value y for each sample t ; Step 9: Update the Critic network parameters w, Actor network parameters θ, and target network parameters w′, θ′; Step 10: Loop step, Step 11: Repeat Step 3 to Step 9 until all training sessions are completed; Step 12: Repeat Step 2-Step 9 until all rounds of training are completed; 2) Online application: Step 1: The current state value of the island microgrid system plus the random error value is used as the comprehensive state input to the Actor network; Step 2: Generate the optimal integrated load reduction decision; step3: Complete the load reduction operation.

9. According to claim 8, an isolated microgrid integrated low-frequency load reduction method integrating multiple types of load correlation factors is characterized by: Critic Network Q w The iterative optimization of parameter w is achieved by minimizing the loss function L(w), which is expressed as: Where: m represents the number of small batch samples drawn from the buffer; b represents the bth historical experience sample; represents the target Q value; TD3 uses two Critic networks on the DDPG framework to estimate the Q function and selects a smaller Q value as the estimated value, which effectively avoids the problem of overestimation of the Q value. The calculation formula is: in, By the target policy network π θ′ Calculated, Where: δ represents the introduced random noise based on normal distribution; -c and c represent the upper and lower bounds of the noise respectively; Critic Network Q w The parameter w is updated according to the gradient theory, and the specific formula is: Where: μ w Represents the Critic network learning rate.

10. The method for low-frequency load shedding of an isolated microgrid integrated with multiple load-related factors according to claim 9, characterized in that: Actor Network π θ The gradient descent method is used to optimize the parameter θ, that is: According to the deterministic policy gradient, update π θ Parameter θ: Where: μ θ Represents the Actor network learning rate; In order to reduce the deviation of the policy network update caused by the instability of Q value, TD3 adjusts the network update frequency so that the Critic network is updated at a higher frequency and the Actor network is updated at a lower frequency. Specifically, the Actor network parameters are updated after the Critic network completes d updates. The Critic network and Actor network parameter updates are performed in the form of soft updates: w n ′=τw n +(1-τ)w n ′ θ′=τθ+(1-τ)θ′ Where: τ represents the soft update coefficient.

Images