Method and system for improving transmission capacity of alternating-current and direct-current hybrid power transmission corridor

Abstract

The invention discloses a method and a system for improving the transmission capacity of an alternating-current and direct-current hybrid power transmission corridor. The method comprises the following steps: step 1), selecting a key power transmission corridor of an alternating-current and direct-current hybrid system; step 2), determining controllable equipment for power transmission of the key power transmission corridor of the alternating current-direct current hybrid system; (3) sensitivity analysis is conducted on the controllable equipment, and high-sensitivity control equipment is screened out; step 4): based on the steps 2)-3), constructing a Markov decision process model; and step 5), solving the Markov decision process model based on a safety deep reinforcement learning method of maximum entropy to obtain an optimal control strategy. According to the invention, two means of combining the direct-current line power and the generator power in the river-crossing section are used at the same time, so that the transmission capacity of the power transmission corridor is comprehensively improved.

Description

Technical Field

[0001] The present invention relates to an electric power system, and in particular to a method and system for improving the transmission capacity of an AC / DC hybrid transmission corridor. Background Art

[0002] With the large-scale grid connection of massive renewable energy and power electronics equipment, ultra-high voltage AC / DC hybrid operation, large-scale cross-regional power transmission, and the increasing proportion of new loads, the operating characteristics of new power systems have undergone profound and complex changes. The high randomness and strong uncertainty of the source and load bring huge challenges to the planning and decision-making of power systems and online safety and stability. Under extreme faults, local disturbances evolve into cascading faults, and the risk of system instability increases significantly. In response to such challenges, the existing power grid planning, operation, and control decision-making systems and methods have many limitations, which are specifically manifested in the following aspects: (1) Existing operation and control decisions are mostly based on offline simulation analysis and empirical judgment. Although a variety of scenarios are considered, it is difficult to cope with the many possibilities brought about by the large-scale fluctuations of a high proportion of new energy, the variable operation characteristics such as AC / DC hybrid operation, and it is impossible to cover all operation mode scenarios to effectively cope with extreme conditions of the power grid. (2) The operation and management data of the power grid come from different business systems, and there are still redundancy and quality problems. It is difficult to effectively extract typical and abnormal operation modes and deeply explore their potential mechanisms. Limited by the simulation analysis speed and modeling accuracy of large power grids, traditional methods are difficult to form optimal control decisions from massive operation data and simulation deduction results. (3) Regional energy management systems usually lack in-depth mining, effective accumulation and application of historical experience; at the decision-making level, there is a lack of effective use of load elasticity. Traditional algorithms mostly rely on precise models, which are inflexible and difficult to support information management and regulation of massive adjustable resources in the future. In addition, the key transmission corridors of AC / DC hybrid power grids are constrained by safety and stability, and their power transmission capacity is severely limited.

[0003] Taking the Jiangsu power grid as an example, as the installed capacity of new energy in northern Jiangsu increases, multiple DC transmission lines are fully operational. According to estimates, by 2026, the maximum transmission power of Jiangsu's cross-river channels (including AC and DC channels) is expected to reach 32.19 million kilowatts, exceeding the transmission capacity by about 7 million kilowatts. In addition, the transmission power of each cross-river channel is seriously uneven, and some cross-river channels frequently exceed the limit, causing a sharp increase in power transmission pressure; among them, the maximum transmission power of the central channel is 5.04 million kilowatts, slightly exceeding the maximum transmission capacity of the channel; the maximum transmission power of the eastern channel is 14.14 million kilowatts, exceeding the transmission capacity of the channel by about 6 million kilowatts; it is urgent to strengthen the transmission capacity of the central and eastern channels.

[0004] The transmission corridor of the traditional power system only contains AC lines, and the transmission power presents the characteristics of natural distribution. When the power flow of a line reaches the limit, it will restrict the power transmission of the entire section. In order to improve the transmission capacity of the AC section, a flexible and controllable transmission corridor power flow distribution method is needed to maximize the transmission capacity. Taking the Jiangsu power grid as an example, the Yanhuai DC, Xitai DC, Jiansu DC, and Yangzhen DC have been put into operation one after another. By utilizing the flexible and controllable characteristics of DC line power, the power flow of heavy-loaded lines in the lines is transferred, and the power transmission capacity of the internal transmission section of the AC system where the power generation center and the load center are not matched is further improved. This is an important means to improve the power transmission capacity of the AC / DC hybrid transmission corridor. However, there is currently little research on power flow control technology for improving the transmission capacity of AC / DC hybrid power grids. Existing research mainly focuses on local power flow optimization control based on FACTS power electronic equipment.

[0005] Therefore, how to fully tap and utilize limited transmission channel resources, improve the level of new energy consumption, and enhance the transmission capacity of important sections are difficult problems that urgently need to be solved in current dispatching operations and safe supply assurance. Summary of the invention

[0006] In view of the above problems, the present invention provides a method and system for improving the transmission capacity of an AC / DC hybrid transmission corridor.

[0007] The technical solution of the present invention is: a method for improving the transmission capacity of an AC / DC hybrid transmission corridor, comprising the following steps:

[0008] Step 1): Select the key transmission corridor of the AC / DC hybrid system;

[0009] Step 2): Determine the controllable equipment for power transmission in the key transmission corridor of the AC / DC hybrid system;

[0010] Step 3): Conduct sensitivity analysis on controllable devices and select high-sensitivity control devices;

[0011] Step 4): Based on steps 2)-3), construct a Markov decision process model;

[0012] Step 5): Based on the maximum entropy safe deep reinforcement learning method, solve the Markov decision process model and obtain the optimal control strategy.

[0013] In step 1), select the key transmission corridor and determine the AC line L in the corridor. AC (i,j) and DC line L DC (m,n) and AC line capacity P AC max(i,j), DC line capacity P DCmax(m,n), where i and j are the starting and ending busbars connected to the AC line, respectively; m and n are the starting and ending busbars connected to the DC line, respectively.

[0014] In step 2), the controllable equipment includes: a DC line and a controllable conventional generator set, wherein P gen (k) represents the active power output of the controllable conventional generator set;

[0015] Select all controllable DC lines as control measures and construct the DC line control set P DC .

[0016] In step 3), historical power flow data files are collected, and sensitivity calculation is performed for a set of controllable conventional generator sets, including:

[0017] 3.1) Based on the historical power flow data file, the total power of the selected key transmission corridor is obtained using the AC power flow calculation method, P 走廊 =P 交流 +P 直流 ,

[0018] Among them, P 交流 is the sum of the power of all AC lines in the transmission corridor, P 交流 =∑ i,j P AC (i,j);P AC (i, j) represents the active power of the AC line;

[0019] P 直流 is the sum of the power of all DC lines in the transmission corridor, P 直流 =∑ m,n P DC (m,n);P DC (m,n) represents the active power of the DC line;

[0020] 3.2) For the kth controllable conventional generator set, based on the current active output, increase the active output ΔP gen (k), re-solve the AC power flow equation and calculate the applied ΔP gen Total power of the transmission corridor after (k) 走廊new ; The sensitivity of the active power of the kth controllable conventional generator set to the power change in the transmission corridor is obtained by the following formula:

[0021]

[0022] 3.3) According to step 3.2), all controllable conventional generator sets are traversed to obtain a sensitivity list; a threshold value Sensitivity_Tgen is set, and generators with a sensitivity lower than this value are eliminated, thereby obtaining a controllable generator list Ggen with a sensitivity higher than Sensitivity_Tgen.

[0023] In step 4), the Markov decision process model includes:

[0024] The state space is a one-dimensional vector of system information obtained after solving the power grid operation mode flow, S = [V m ,V a ,P ac ,Q ac ,P dc ,P gen ,P renewable ], where V m is the AC bus voltage amplitude vector, V a is the AC bus voltage phase angle vector, P ac is the AC line active power vector, Q ac is the AC line reactive power vector, P dc is the DC line active power vector, P gen is the active power vector of the conventional generator set, P renewable is the active power vector of the new energy generator set;

[0025] The action space of multi-type agents is:

[0026] Type 1: The agent of the conventional generator set is the active power change vector ΔP of the conventional generator set gen =[ΔP gen(1) ,ΔP gen(2) ,…,ΔP gen(g) ],g∈Ggen;

[0027] Type 2: The DC line agent is the DC line active power change vector ΔP dc =[ΔP dc(1) ,ΔP dc(2) ,…,ΔP dc(h) ],h∈P DC ;

[0028] Reward function:

[0029] Reward=β1r1+β2r2-β3r3-β4r4-β5r5

[0030] in:

[0031] r1 represents the power value of the increased transmission capacity of the transmission corridor;

[0032] r2 represents the increase in the total power of DC lines within the transmission corridor;

[0033] r3 represents the total amount of AC line power exceeding the limit in the power system;

[0034] r4 represents the total amount of transformer power exceeding the limit in the power system;

[0035] r5 represents the penalty for non-convergence of AC power flow calculation;

[0036] β1, β2, β3, β4, and β5 are weight coefficients used to balance different types of reward values.

[0037] In step 5), for the DC line agent and the conventional generator set agent, according to the state space, action space and reward function described in step 4), the reinforcement learning agent model is trained to obtain the optimal control strategy.

[0038] The training process of the reinforcement learning agent model includes:

[0039] First, collect the historical operating condition flow files and solve the flow;

[0040] Secondly, the state space is extracted. According to the state vector, the agent outputs control actions with the goal of maximizing the reward function value.

[0041] Then, the control action is applied to the current system state, the AC power flow is solved and the agent reward function is calculated, and the samples saved in the form of tuples are stored in the cache;

[0042] Finally, determine the termination condition of the agent training. If the termination condition is met, exit the training process and save the agent; if the termination condition is not met, extract the next flow file and continue training the agent.

[0043] The weight and bias parameters in the reinforcement learning agent model are represented by θ, and the agent updates θ through learning to obtain the maximum reward value;

[0044] When solving the problem, the maximum entropy safety reinforcement learning algorithm is used. On the basis of maximum entropy reinforcement learning, safety reinforcement learning introduces the safety constraint g(s)≤0, where g(s) is a function that measures the safety of state s.

[0045] The objective function of maximum entropy security reinforcement learning is expressed as:

[0046] max_πJ(π)+αH(π)stE[g(s)]≤0

[0047] Where π is the strategy, J(π) is the expected return when applying strategy π, α is the coefficient of the entropy regularization term, H(π) is the entropy of strategy π, and E is the mathematical expectation function;

[0048] Convert the constrained optimization problem into Lagrangian form:

[0049] L(π,λ)=J(π)+αH(π)-λE[g(s)]

[0050] Where λ is the Lagrange multiplier;

[0051] Using the policy gradient method, we get the policy update rule:

[0052]

[0053] Where L is the objective function after converting the constrained optimization problem into Lagrangian form, logπ represents the logarithmic form of strategy π, θ(a|s) represents the probability of the strategy selecting action a in state s, and Q^π(s,a) represents the expected return after taking action a from state s under strategy π.

[0054] It also includes step 6), which is specifically:

[0055] During the real-time dispatching process of the power grid, the system data under the current power grid operation mode is extracted, the power flow is solved and the agent input vector is constructed. The reinforcement learning agent model is called to give the optimal decision values ​​of the DC line and conventional generator sets, and the power flow calculation is performed to verify the validity of the agent's decision results.

[0056] If the control strategy verification is passed, it will be sent to the DC station and generator set;

[0057] If the control strategy verification fails, it will not be sent to the station or generator set.

[0058] A system for improving the transmission capacity of an AC / DC hybrid transmission corridor, comprising:

[0059] A selection module is used to select key transmission corridors of AC / DC hybrid systems;

[0060] A controllable module for determining controllable devices for power delivery in key transmission corridors of AC / DC hybrid systems;

[0061] The analysis module is used to perform sensitivity analysis on controllable devices and screen out highly sensitive control devices;

[0062] Decision module, used to build Markov decision process model;

[0063] The solution module is used to solve the Markov decision process model based on the maximum entropy safe deep reinforcement learning method and derive the optimal control strategy.

[0064] In the work, the present invention selects key transmission corridors, determines controllable equipment, and selects more effective control equipment for improving the transmission capacity of key transmission corridors; and constructs a Markov decision process model and solves it to obtain the optimal control strategy. This method provides an effective solution to solve the limited transmission capacity of key transmission corridors. The trained intelligent agent based on DC line power and active power of generator sets has the advantages of fast speed and strong adaptability compared with the traditional control strategy derived from the complete system mechanism model.

[0065] The present invention simultaneously uses two means of combining the DC line power and the generator power in the cross-river section to comprehensively improve the transmission capacity of the transmission corridor. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flow chart of the method of the present invention,

[0067] Figure 2 It is a diagram of the training process of the reinforcement learning agent model.

[0068] Figure 3 is a schematic diagram of the reward value during the agent training process in the embodiment,

[0069] Figure 4 is a schematic diagram of the control effect of the intelligent agent on the total power increase of the transmission corridor in the embodiment,

[0070] Figure 5 It is a schematic diagram of the control effect of the intelligent agent on improving the transmission capacity of the transmission corridor in the embodiment. DETAILED DESCRIPTION

[0071] like Figure 1 As shown, the present invention provides a method for improving the transmission capacity of an AC / DC hybrid transmission corridor, comprising the following steps:

[0072] Step 1): Select the key transmission corridor of the AC / DC hybrid system; thus, the key transmission corridor definition is selected as the controlled object;

[0073] Step 2): Determine the controllable devices for power transmission in the key transmission corridor of the AC / DC hybrid system; for the transmission corridor defined in step 1), specify a set of controllable devices, including DC line power and generator power output;

[0074] Step 3): Conduct sensitivity analysis on controllable devices and select high-sensitivity control devices; by calculating sensitivity information, select more effective control devices for improving the transmission capacity of key transmission corridors;

[0075] Step 4): Based on steps 2)-3), construct a Markov decision process model;

[0076] Step 5): Based on the maximum entropy safe deep reinforcement learning method, solve the Markov decision process model and obtain the optimal control strategy.

[0077] The present invention uses two methods, namely, the DC line power and the generator power in the cross-river section, to comprehensively improve the transmission capacity of the transmission corridor. Compared with the traditional method, the method proposed in the present invention can instantly give an optimization strategy, demonstrating the superiority of the method.

[0078] The specific steps are described as follows:

[0079] Step 1): Select the key transmission corridor of the AC / DC hybrid system and determine the AC line L in the corridor. AC (i,j) and DC line L DC (m,n) and AC line capacity P AC max(i,j), DC line capacity P DC max(m,n). Among them, L AC (i, j) is the AC line, i and j are the head and end busbars connected to the AC line respectively; L DC (m,n) is a DC line, m and n are the head and end buses connected to the DC line. AC is alternating current; DC is direct current.

[0080] Step 2): Determine the controllable equipment for power transmission in the key transmission corridor of the AC / DC hybrid system, including: DC lines and controllable conventional generator sets, where P gen (k) represents the active power output of the controllable conventional generator set. All controllable DC lines are selected as control measures to construct the DC line control set P DC .

[0081] Step 3): For conventional generator sets, sensitivity analysis is required to select high-sensitivity generator sets as control measures. Collect historical flow data files and perform sensitivity calculations for the set of controllable conventional generator sets. The method is as follows:

[0082] 3.1) Based on the historical power flow data file, the total power of the selected key transmission corridor is obtained using the AC power flow calculation method, P 走廊 =P 交流 +P 直流 , where P 交流 is the sum of the power of all AC lines in the transmission corridor, that is, P 交流 =∑ i,j P AC (i,j);P AC (i, j) represents the active power of the AC line; P 直流 is the sum of the power of all DC lines in the transmission corridor, P 直流 =∑ m,nP DC (m,n);P DC (m,n) represents the active power of the DC line.

[0083] 3.2) For the kth controllable conventional generator set, based on the current active output, increase the active output ΔP gen (k), re-solve the AC power flow equation and calculate the applied ΔP gen Total power of the transmission corridor after (k) The sensitivity of the active power of the kth controllable conventional generator set to the power change in the transmission corridor is given by the following formula:

[0084]

[0085] 3.3) According to step 3.2), all controllable conventional generators are traversed to obtain a sensitivity list. A threshold value Sensitivity_Tgen is set, and generators with a sensitivity lower than this value are eliminated, thereby obtaining a controllable generator list Ggen with a sensitivity higher than Sensitivity_Tgen.

[0086] Step 4): Construct the problem of improving the power transmission capacity of key transmission corridors of AC / DC hybrid systems into a Markov decision process, and design a collaborative agent model for multiple types of control equipment. Construct an optimization scheduling model that takes into account the operation constraints of regulatory resources and system safety constraints, and design a multi-agent collaborative control model with constraints. The above Markov decision process model with continuous action space converts the real-time optimization scheduling of the system into a multi-agent optimization problem. Each agent uses its own strategy function as a decision variable and maximizes its own value function while satisfying the safety constraint cost constraints. The Markov decision process model includes the following main parts:

[0087] The state space is a one-dimensional vector of system information obtained after solving the power grid operation mode flow, S = [V m ,V a ,P ac ,Q ac ,P dc ,P gen ,P renewable ], where V m is the AC bus voltage amplitude vector, V a is the AC bus voltage phase angle vector, P ac is the AC line active power vector, Q ac is the AC line reactive power vector, P dc is the DC line active power vector, P gen is the active power vector of the conventional generator set, P renewable It is the active power vector of the new energy generator set.

[0088] The action space of multi-type agents is:

[0089] Type 1: The agent of the conventional generator set is the active power change vector ΔP of the conventional generator set gen =[ΔP gen(1) ,ΔP gen(2) ,…,ΔP gen(g) ],g∈Ggen.

[0090] Type 2: The DC line agent is the DC line active power change vector ΔP dc =[ΔP dc(1) ,ΔP dc(2) ,…,ΔP dc(h) ],h∈P DC .

[0091] Reward function: In order to effectively improve the power transmission capacity of key transmission corridors, the reward function design includes rewards for improved transmission capacity, rewards for improved DC power, penalties for line over-limit, penalties for non-convergence of power flow calculations, and penalties for equipment overload. The agent reward function is designed as follows:

[0092] Reward=β1r1+β2r2-β3r3-β4r4-β5r5

[0093] in:

[0094] r1=ΔC transfer , represents the power value of the transmission corridor's increased transmission capacity. The transmission corridor's transmission capacity is defined as:

[0095]

[0096] r2=ΔP 直流 , represents the increase in the total power of DC lines within the transmission corridor.

[0097] Represents the total power over-limit of AC lines in the power system, where H is the number of power over-limit lines, Pline represents the active power of the AC line, and Plinemax represents the upper limit of the active power of the AC line.

[0098] Represents the total amount of transformer power over-limit in the power system, where K is the number of transformers with over-limit power, Xfm represents the transformer active power, and Xfm_max represents the upper limit of the transformer active power.

[0099] r5 represents the penalty for non-convergence of AC power flow calculation.

[0100] β1, β2, β3, β4, and β5 are weight coefficients used to balance different types of reward values.

[0101] Step 5): A maximum entropy-based safe deep reinforcement learning method is used to solve the above Markov decision process model and derive the optimal control strategy.

[0102] When multiple agents can simultaneously meet multiple control objectives and safety constraints, the agent joint strategy can realize online optimization decision-making for improving transmission capacity.

[0103] Train the DC line agent and the conventional generator set agent. According to the state space, action space and reward function described in step 4), train the reinforcement learning agent.

[0104] like Figure 2 As shown, the training process of the reinforcement learning agent model includes:

[0105] Collect the historical operating condition flow data files. For each flow data problem, first perform AC flow calculation. After the calculation iteration converges, extract the state space, S = [V m ,V a ,P ac ,Q ac ,P dc ,P gen ,P renewable ]; According to the state vector, the agent outputs control actions with the maximum reward function value as the target, [ΔP gen ,ΔP dc ]; the control action is applied to the current system state, the AC power flow is solved and the agent reward function is calculated, and the samples saved in the form of tuples are stored in the cache to record the control effect of the agent; the neural network parameters in the agent are further updated; further, the termination condition of the agent training is determined, if the termination condition is met, the training process is exited and the agent is saved; if the termination condition is not met, the next power flow file is extracted and the training of the agent continues.

[0106] Step 6): During the real-time dispatching of the power grid, extract the system data under the current power grid operation mode, solve the power flow and construct the intelligent agent input vector S, call the intelligent agent model to give the optimal decision value of the DC line and the conventional generator set, and perform power flow calculation to verify the validity of the intelligent agent decision result. If the control strategy verification is passed, it will be sent to the DC station and generator set; if it does not pass the control strategy verification, it will not be sent to the station or generator set.

[0107] The present invention proposes a method for improving the transmission capacity of a power transmission corridor based on maximum entropy deep reinforcement learning. Reinforcement learning is a field in machine learning that emphasizes how to act based on the environment to maximize expected benefits. It can be used for online control decisions of various types of equipment in complex dynamic systems. The successful application of reinforcement learning has been reported in other fields, such as game theory, control theory, operations research, information theory, simulation optimization, multi-agent system learning, swarm intelligence, statistics, and genetic algorithms.

[0108] Deep reinforcement learning is a branch of artificial intelligence that aims to maximize the accumulation of reward values ​​for trained agents through continuous interaction with the environment in order to achieve a given control goal. In this process, given a set of states and a set of actions, the probability of transition from state s to a new state s′ after a given action a is expressed as p(s,a,s′), and the reward value r is obtained from action a and state s, expressed as r(s,a). Together with the discount factor γ for future reward values, it is represented as a tuple 〈s,a,p,r〉. The goal of agent training is to find the optimal strategy π(a|s), which maps the behavior of a given state to maximize the long-term accumulation of reward values ​​it obtains:

[0109]

[0110] V π (s) = E(G t ∣s t =s;π)

[0111] Q π (s,a)=E(G t ∣s t =s,a t =a;π)

[0112] Among them, G t is the reward value accumulation, t is the number of control iterations, r t is the reward value at step t, γ is the discount factor, and E is the mathematical expectation function. Two important concepts in deep reinforcement learning algorithms are the state value function V π (s) and Q-value function Q π (s,a). V π (s) evaluates the quality of a state by calculating the expected reward value of following a certain action from a certain state; and Q π (s,a) evaluates the reward value of a state by calculating the expected reward value of following action a from the state.

[0113] The weights and bias parameters in the reinforcement learning agent’s neural network are represented by θ. The policy can be represented as πθ(a|s), and the agent updates θ through learning to obtain the maximum reward value.

[0114] The present invention adopts the maximum entropy safety reinforcement learning algorithm (safe SAC, Soft Actor Critic).

[0115] Among them, the process of updating the optimal strategy in the traditional SAC algorithm is as follows:

[0116]

[0117] The principle of maximum entropy reinforcement learning encourages strategies to explore more possibilities and increase the randomness of strategies. t ) represents the control strategy in state s t The entropy value at the moment, H(π) = -Σπ(a|s)logπ(a|s); the α coefficient controls the balance between exploring new control strategies and adopting existing control strategies, R(s t ,a t ) represents the state s at time t t Next, control action a is adopted t The reward value after is in the probability distribution ρ π The state s at time t in space t Next, control action a is adopted t The expected value of the reward.

[0118] Based on traditional maximum entropy reinforcement learning, the present invention introduces a safety constraint g(s)≤0 in safety reinforcement learning, where g(s) is a function that measures the safety level of state s.

[0119] The objective function of maximum entropy security reinforcement learning is expressed as:

[0120] max_πJ(π)+αH(π)stE[g(s)]≤0

[0121] Where π is the strategy, J(π) is the expected return when applying strategy π, α is the coefficient of the entropy regularization term, H(π) is the entropy of strategy π, which is used to measure the randomness or uncertainty of the strategy, and E is the mathematical expectation function;

[0122] Convert the constrained optimization problem into Lagrangian form:

[0123] L(π,λ)=J(π)+αH(π)-λE[g(s)]

[0124] Where λ is the Lagrange multiplier;

[0125] Using the policy gradient method, we get the policy update rule:

[0126]

[0127] Where L is the objective function after converting the constrained optimization problem into Lagrangian form, logπ represents the logarithmic form of strategy π, θ(a|s) represents the probability of the strategy selecting action a in state s, and Q^π(s,a) represents the expected return after taking action a from state s under strategy π.

[0128] Maximum entropy safety reinforcement learning takes into account reward maximization, entropy maximization, and safety constraints. By iteratively optimizing this goal, we can obtain a strategy that can obtain high rewards, be exploratory, and maintain safety at the same time.

[0129] The present invention also provides a system for improving the transmission capacity of an AC / DC hybrid transmission corridor, comprising:

[0130] A selection module is used to select key transmission corridors of AC / DC hybrid systems;

[0131] A controllable module for determining controllable devices for power delivery in key transmission corridors of AC / DC hybrid systems;

[0132] The analysis module is used to perform sensitivity analysis on controllable devices and screen out highly sensitive control devices;

[0133] Decision module, used to build Markov decision process model;

[0134] The solution module is used to solve the Markov decision process model based on the maximum entropy safe deep reinforcement learning method and derive the optimal control strategy.

[0135] Based on the dispatching and control system, the present invention optimizes the power flow to improve the transmission capacity of the AC / DC hybrid transmission corridor, and utilizes the flexible and rapid power control characteristics of the DC transmission system to improve the power transmission capacity of the regional power grid and the new energy absorption capacity.

[0136] Example: Taking the Jiangsu power grid cross-river transmission corridor as an example, the corridor includes Yangzhen DC (±200kV) and 11 AC transmission lines (9 500kV and 2 1000kV). Collect the time series operation mode flow file for training the maximum entropy safety reinforcement learning agent for improving the transmission capacity of Jiangsu cross-river channel.

[0137] Among them, the reward value during the agent training process is as follows Figure 3 As shown in the figure, as the number of training iterations increases, the reward value of the agent continues to accumulate. After more than 2000 trainings, the reward value of the agent is stable at around 1.4, which proves the effectiveness of the agent training. Compared with the traditional method based on the complete model of the whole system and the optimization solution, the method proposed in this invention can instantly give the optimization strategy, demonstrating the superiority of this method.

[0138] like Figure 4 and Figure 5 As shown in the figure, the strategy given by the intelligent agent can effectively improve the transmission power and power transmission capacity of the cross-river corridor of Jiangsu power grid, with the total transmission power increased by an average of 473.73MW and the channel transmission capacity increased by an average of 646.06MW.

[0139] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be covered by the scope of the claims of the present invention.

Claims

1. A method for improving the transmission capacity of an AC / DC hybrid transmission corridor, characterized in that: The following steps are involved: Step 1): Select the key transmission corridor of the AC / DC hybrid system; Step 2): Determine the controllable equipment for power transmission in the key transmission corridor of the AC / DC hybrid system; Step 3): Conduct sensitivity analysis on controllable devices and select high-sensitivity control devices; Step 4): Based on steps 2)-3), construct a Markov decision process model; Step 5): Based on the maximum entropy safe deep reinforcement learning method, solve the Markov decision process model and obtain the optimal control strategy.

2. A method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 1, characterized in that: In step 1), select the key transmission corridor and determine the AC line L in the corridor. AC (i,j) and DC line L DC (m,n) and AC line capacity P AC max(i,j), DC line capacity P DC max(m,n), where i and j are the starting and ending busbars connected to the AC line, respectively; m and n are the starting and ending busbars connected to the DC line, respectively.

3. The method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 1, characterized in that: In step 2), the controllable equipment includes: a DC line and a controllable conventional generator set, wherein the active power output of the controllable conventional generator set is P gen (k); Select all controllable DC lines as control measures and construct the DC line control set P DC .

4. The method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 1, characterized in that: In step 3), historical power flow data files are collected, and sensitivity calculation is performed for a set of controllable conventional generator sets, including: 3.1) Based on the historical power flow data file, the total power of the selected key transmission corridor is obtained using the AC power flow calculation method, P 走廊 =P 交流 +P 直流 , Among them, P 交流 is the sum of the power of all AC lines in the transmission corridor, P 交流 =∑ i,j P AC (i,j);P AC (i, j) represents the active power of the AC line; P 直流 is the sum of the power of all DC lines in the transmission corridor, P 直流 =∑ m,n P DC (m,n);P DC (m,n) represents the active power of the DC line; 3.2) For the kth controllable conventional generator set, based on the current active output, increase the active output ΔP gen (k), re-solve the AC power flow equation and calculate the applied ΔP gen Total power of the transmission corridor after (k) The sensitivity of the active power of the kth controllable conventional generator set to the power change in the transmission corridor is given by the following formula: 3.3) According to step 3.2), all controllable conventional generator sets are traversed to obtain a sensitivity list; a threshold value Sensitivity_Tgen is set, and generators with a sensitivity lower than this value are eliminated, thereby obtaining a controllable generator list Ggen with a sensitivity higher than Sensitivity_Tgen.

5. A method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 4, characterized in that: In step 4), the Markov decision process model includes: The state space is the one-dimensional vector of system information S obtained after solving the power flow of the power grid operation mode. m ,V a ,P ac ,Q ac ,P dc ,P gen ,P renewable ], where V m is the AC bus voltage amplitude vector, V a is the AC bus voltage phase angle vector, P ac is the AC line active power vector, Q ac is the AC line reactive power vector, P dc is the DC line active power vector, P gen is the active power vector of the conventional generator set, P renewable is the active power vector of the new energy generator set; The action space of multi-type agents is: Type 1: The agent of the conventional generator set is the active power change vector ΔP of the conventional generator set gen =[ΔP gen(1) ,ΔP gen(2) ,…,ΔP gen(g) ],g∈Ggen; Type 2: The DC line agent is the DC line active power change vector ΔP dc =[ΔP dc(1) ,ΔP dc(2) ,…,ΔP dc(h) ],h∈P DC ; Reward function: Reward=β1r1+β2r2-β3r3-β4r4-β5r5 in: r1 represents the power value of the increased transmission capacity of the transmission corridor; r2 represents the increase in the total power of DC lines within the transmission corridor; r3 represents the total amount of AC line power exceeding the limit in the power system; r4 represents the total amount of transformer power exceeding the limit in the power system; r5 represents the penalty for non-convergence of AC power flow calculation; β1, β2, β3, β4, and β5 are weight coefficients used to balance different types of reward values.

6. A method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 5, characterized in that: In step 5), for the DC line agent and the conventional generator set agent, according to the state space, action space and reward function described in step 4), the reinforcement learning agent model is trained to obtain the optimal control strategy.

7. A method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 6, characterized in that: The training process of the reinforcement learning agent model includes: First, collect the historical operating condition flow files and solve the flow; Secondly, the state space is extracted. According to the state vector, the agent outputs control actions with the goal of maximizing the reward function value; Then, the control action is applied to the current system state, the AC power flow is solved and the agent reward function is calculated, and the samples saved in the form of tuples are stored in the cache; Finally, determine the termination condition of the agent training. If the termination condition is met, exit the training process and save the agent; if the termination condition is not met, extract the next flow file and continue training the agent.

8. A method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 7, characterized in that: The weight and bias parameters in the reinforcement learning agent model are represented by θ, and the agent updates θ through learning to obtain the maximum reward value; When solving the problem, the maximum entropy safety reinforcement learning algorithm is used. On the basis of maximum entropy reinforcement learning, safety reinforcement learning introduces the safety constraint g(s)≤0, where g(s) is a function that measures the safety of state s. The objective function of maximum entropy security reinforcement learning is expressed as: max_πJ(π)+αH(π)stE[g(s)]≤0 Where π is the strategy, J(π) is the expected return when applying strategy π, α is the coefficient of the entropy regularization term, H(π) is the entropy of strategy π, and E is the mathematical expectation function; Convert the constrained optimization problem into Lagrangian form: L(π,λ)=J(π)+αH(π)-λE[g(s)] Where λ is the Lagrange multiplier; Using the policy gradient method, we get the policy update rule: Where L is the objective function after converting the constrained optimization problem into Lagrangian form, logπ represents the logarithmic form of strategy π, θ(a|s) represents the probability of the strategy selecting action a in state s, and Q^π(s,a) represents the expected return after taking action a from state s under strategy π.

9. The method for improving the transmission capacity of an AC / DC hybrid transmission corridor according to claim 1, characterized in that: It also includes step 6), which is specifically: During the real-time dispatching process of the power grid, the system data under the current power grid operation mode is extracted, the power flow is solved and the agent input vector is constructed. The reinforcement learning agent model is called to give the optimal decision values ​​of the DC line and conventional generator sets, and the power flow calculation is performed to verify the validity of the agent's decision results. If the control strategy verification is passed, it will be sent to the DC station and generator set; If the control strategy verification fails, it will not be sent to the station or generator set.

10. A system for improving the transmission capacity of an AC / DC hybrid transmission corridor, characterized in that: include: A selection module is used to select key transmission corridors of AC / DC hybrid systems; A controllable module for determining controllable devices for power delivery in key transmission corridors of AC / DC hybrid systems; The analysis module is used to perform sensitivity analysis on controllable devices and screen out highly sensitive control devices; Decision module, used to build Markov decision process model; The solution module is used to solve the Markov decision process model based on the maximum entropy safe deep reinforcement learning method and derive the optimal control strategy.

Images