An Optimized Dispatch Method for Electric Vehicle Distribution Networks

Abstract

The present invention discloses an electric vehicle distribution network optimization scheduling method, which belongs to the field of power grid optimization scheduling, and includes the following steps: S1, considering system state variables, constructing an MDP model based on electric vehicle charging plan and distribution network voltage control problem, and realizing automatic voltage regulation and reducing electric vehicle charging costs through coordinated control of VVC equipment, electric vehicles and distributed generators; S2, constructing a deep neural network framework based on electric vehicle charging plan and distribution network voltage control problem, and using a deep deterministic policy gradient algorithm to train the deep neural network to output discrete and continuous control actions. The present invention adopts the above-mentioned electric vehicle distribution network optimization scheduling method, and enables the agent to learn an economic scheduling strategy through a two-layer scheduling control strategy, and maintain the voltage stability of the distribution network, and adopts data-driven, independent of the uncertain model in the system.

Description

Technical Field

[0001] The present invention relates to the field of power grid optimization dispatching, and in particular to an electric vehicle distribution network optimization dispatching method. Background Art

[0002] In global carbon emission statistics, the global electricity and heat production industry contributes 42% of carbon dioxide emissions, while industry and transportation contribute 18.4% and 24.6% respectively.

[0003] In order to reduce the carbon emission level of the transportation system, replacing the transportation system based on fossil fuels with the transportation system based on electricity is an effective technical path. According to the existing technical level of traditional gasoline vehicles and electric vehicles, the carbon emission per unit mileage of electric vehicles is about 50% of the emission level of traditional gasoline vehicles of the same level, based on the scenario of thermal power stations providing electricity for electric vehicles.

[0004] At the same time, driven by breakthroughs in battery technology, electric vehicles are rapidly becoming a substitute for fossil fuel vehicles. However, large-scale access to the power grid for electric vehicles will bring challenges such as frequency drift and voltage fluctuations. The voltage stability of the distribution network is an issue that needs to be focused on. The uncontrolled charging process of electric vehicles will affect the voltage stability of the distribution network. This situation will be further exacerbated when electric vehicles are connected to the grid in vehicle-to-vehicle (V2G) mode. In addition, the access of distributed generators (DG) to the power system will change the direction of the power flow. The injection of active power by distributed generators (DG) upstream will cause the voltage to rise and interfere with the voltage / reactive power control (VVC) equipment.

[0005] In traditional VVC practice, voltage regulation devices such as on-load tap changers (OLTCs), voltage regulators (VRs), and switchable capacitor banks (SCBs) are used to mitigate voltage violations. For the inverter-controlled renewable energy access problem, multi-stage methods have been widely used for VVC in distribution networks. In order to coordinate VVC devices and distributed generation, neural networks and learning-based methods are widely used, but existing learning-based methods do not consider the impact of electric vehicle charging on voltage stability, nor the possibility of electric vehicles participating in voltage control.

[0006] Electric vehicles participate in the voltage regulation of the distribution network. On the one hand, the charging behavior of electric vehicles is stimulated by electricity prices; on the other hand, electric vehicles can operate in V2G mode. By adjusting the charging power or discharging power of electric vehicles, it helps to stabilize the voltage of the distribution network. And when facing the problem of large-scale electric vehicle charging, the charging process of electric vehicles is managed by an aggregator or a central controller. However, the charging process of electric vehicles has great uncertainty, which requires the estimation and prediction of the charging demand of electric vehicles. Artificial intelligence methods have attracted attention due to their advantages in processing high-dimensional data and nonlinear problems. However, existing studies only study the charging problem of electric vehicles from the perspective of demand response (DR). The capacity and access location of electric vehicle charging loads will affect the power flow distribution of the distribution network, and disordered electric vehicle charging will reduce the voltage stability of the distribution network.

[0007] In summary, current research focuses more on VVC of EV DN (distribution network) or DR, while there are few studies that consider both and conduct coordinated optimization. Summary of the invention

[0008] To solve the above problems, the present invention provides an electric vehicle distribution network optimization scheduling method, which enables the agent to learn an economical scheduling strategy and maintain the voltage stability of the distribution network through a two-layer scheduling control strategy. It is data-driven and does not rely on the uncertain model in the system.

[0009] To achieve the above object, the present invention provides an electric vehicle distribution network optimization scheduling method, comprising the following steps:

[0010] S1. Considering the system state variables, an MDP model is constructed based on the electric vehicle charging plan and the distribution network voltage control problem. Through the coordinated control of VVC equipment, electric vehicles and distributed generators, automatic voltage regulation and reduction of electric vehicle charging costs are achieved;

[0011] S2. Based on the electric vehicle charging plan and distribution network voltage control problems, a deep neural network framework is constructed, and the deep deterministic policy gradient algorithm is used to train the deep neural network to output discrete and continuous control actions.

[0012] Preferably, the system state described in step S1 includes the uncertainty of the charging process caused by the charging behavior of the electric vehicle user and the uncertainty of uncontrollable load, system dynamic electricity price and renewable energy generation.

[0013] Preferably, step S1 specifically includes the following steps:

[0014] S11. Construct a collaborative electric vehicle charging and voltage control framework, which includes electric vehicles, distributed generators and renewable energy, and the electric vehicles are controlled by smart terminals;

[0015] S12, building a controllable unit based on a collaborative electric vehicle charging and voltage control framework;

[0016] S13. Model the electric vehicles, distributed generators and voltage control devices in the distribution network system, and set operating constraints to obtain the MDP model. The upper layer of the MDP model is set as the scheduling layer, which is used to output control signals of all dispatchable units according to the system status. The lower layer of the MDP model is the voltage control layer, which is used to receive the upper layer control signals and control the dispatchable units in the DN system.

[0017] Preferably, the controllable unit construction steps described in step S12 are as follows:

[0018] S121. Building electric vehicles:

[0019] Assuming that the control variable of the electric vehicle is the charging power, then for the node connected to the distribution network The electric vehicle, the control variable is expressed as ; In V2G mode, it is the discharge power. The value of is positive during charging and negative during discharging. The charging power constraint based on the charging strategy is:

[0020] (1)

[0021] In the formula, The maximum charging power for electric vehicles; is the maximum discharge power of the electric vehicle; For the scheduling period; is a node set; is a set of time slots;

[0022] And the charging state of the electric vehicle battery meets the following constraints during the scheduling period:

[0023] (2)

[0024] (3)

[0025] In the formula, and Electric vehicle batteries Moment and Charging status at all times; is the capacity of the electric vehicle battery; and are the charge rate and discharge rate respectively; and Respectively represent the maximum and minimum values ​​of the charging state; is the interval of one time step;

[0026] S122. Construction of distributed generators:

[0027] Assume that the control variables of the distributed generator are active power output and reactive power output, then the node Active power of distributed generators at and reactive power The following constraints are met:

[0028] (4)

[0029] (5)

[0030] In the formula, is the maximum active power; is the minimum reactive power;

[0031] S123. Construction of on-load tap changer and voltage regulator:

[0032] On-load tap changers and voltage regulators are connected to distribution network branches by changing the tap position control The control variable is expressed as , varies within the range of integers and satisfies the following constraints:

[0033] (6)

[0034] In the formula, is the maximum adjustable position of the tap; It is a collection of distribution network branches;

[0035] S124. Constructing a switchable capacitor bank:

[0036] The value of reactive power provided by the switchable capacitor bank is set by adjusting the number of operating units. Number of operating units of switchable capacitor banks at Take an integer and satisfy the following conditions:

[0037] (7)

[0038] In the formula, The maximum number of operation units.

[0039] Preferably, the operation constraints described in step S13 are as follows:

[0040] Constrain the complex power that the substation can withstand:

[0041] (8)

[0042] In the formula, , , They are active power, reactive power and maximum apparent power of the substation respectively;

[0043] Constraining node voltages :

[0044] (9)

[0045] In the formula, and are the maximum and minimum values ​​of the node voltage respectively;

[0046] Constraining branch current :

[0047] (10)

[0048] In the formula, and are the maximum and minimum values ​​of the branch current respectively.

[0049] Preferably, the MDP model construction steps described in step S13 are as follows:

[0050] The first step is to set the state variables of the distribution network system:

[0051] (11)

[0052] In the formula, For the distribution network system at any time state variables; Node Department Active power demand at each moment; Node Department Active power demand at each moment; They are Historical electricity prices at the time; and Node Wind turbine power output and photovoltaic power output at Node The charge state at Start charging time; Node The trigger time at They are The expected state of charge;

[0053] Assuming that wind turbines and photovoltaics are uncontrollable power sources, calculate the node Net load requirement :

[0054] (12)

[0055] Substituting formula (12) into formula (11) yields:

[0056] (13)

[0057] In the formula, Points Department Net load demand at the moment;

[0058] Step 2: Set the action variables of the distribution network system:

[0059] The system action variables include the active power and reactive power of the generator, the charging capacity of the electric vehicle, the tap position of the on-load tap changer, and the number of switchable capacitor banks:

[0060] (14)

[0061] In the formula, is the action variable of the distribution network system; Respectively represent nodes Active power of distributed generators at Respectively represent nodes Reactive power of distributed generators at Respectively represent nodes The number of operating units of the switchable capacitor bank at Respectively represent nodes The tap position of the on-load tapchanger at

[0062] The third step is to set the reward function by considering the operating cost of the distribution network system and the voltage stability target. The operating cost includes the cost of the distribution network interacting with the main grid through the substation, the generation cost of distributed generators, and the charging cost of electric vehicles:

[0063] (15)

[0064] In the formula, is the weight coefficient, which indicates that the reward function is a trade-off between the operating cost and the voltage stability target; The cost of purchasing electricity from the power exchange; For Node The generation cost of distributed generators at For Node The cost of charging electric vehicles in the country; Penalty for violating voltage constraints;

[0065] Among them, the cost of purchasing electricity from the power exchange The calculation formula is as follows:

[0066] (16)

[0067] In the formula, They are Historical electricity prices at the time;

[0068] node The cost of distributed generators at The calculation formula is as follows:

[0069] (17)

[0070] In the formula, , , All are power generation cost coefficients;

[0071] Charging costs for electric vehicles The calculation formula is as follows:

[0072] (18)

[0073] In the formula, For Node Historical electricity prices at

[0074] Penalty for Violating Voltage Constraints The calculation formula is as follows:

[0075] (19)

[0076] In the formula, is the penalty coefficient of voltage deviation;

[0077] Step 4: Set the objective function.

[0078] Preferably, the objective function calculation formula is as follows:

[0079] (20)

[0080] In the formula, is the expected cumulative discount return during the planning period; They are The reward for the time step; They are The discount factor for the time step; Indicates sum.

[0081] Preferably, the deep deterministic policy gradient algorithm described in step S2 is a two-layer structure, wherein the scheduling signal and the control signal are both output by the upper layer of the agent, and the lower layer obtains the complete reward value by controlling the controllable unit in the distribution network.

[0082] Preferably, step S2 specifically includes the following steps:

[0083] S21. In the reinforcement learning framework, the learning agent interacts with the constructed MDP model to transform the optimization problem into a standard reinforcement learning framework. as follows:

[0084] (twenty one)

[0085] In the formula, for status; represents the expected return; Indicates the state of Follow the strategy Expected Reward The action value function of is the set of expected returns;

[0086] S22. Solve the optimal action-value function by recursively solving the Bellman equation , and based on the optimal strategy obtained , describing the optimization problem:

[0087] (twenty two)

[0088] In the formula, Represents the optimal strategy Expected value; represents the reward at time t; represents the discount factor; Indicates the state of Optimal strategy Expected Reward The optimal action value function of

[0089] S23. Construct a deep deterministic policy gradient algorithm to determine the critic parameters:

[0090] (twenty three)

[0091] In the formula, Indicates the batch size of samples sampled from the replay buffer; Represents the target of the loss function; represents an approximate function; Indicates the current The target value of the value;

[0092] S24. Update the parameters of the participant value network based on the deep deterministic policy gradient algorithm:

[0093] (twenty four)

[0094] In the formula, Represents a function approximator In the deterministic target strategy The gradient operator under ; Indicates the expected return from starting distribution; represents the updated expected value; represents the participant function; express; represents the gradient of the weight; Indicates behavior parameters; Indicates the status parameter.

[0095] S25. Soft-update the target network using the updated critic and actor networks until a converged strategy is obtained:

[0096] (25)

[0097] In the formula, Represents the updated behavior parameters; Indicates the updated state parameters; Represents the soft update coefficient.

[0098] Preferably, after step S24, step S25 is also included;

[0099] It specifically includes the following steps:

[0100] Soft-update the target network using the updated critic and actor networks until a converged policy is obtained:

[0101] (25)

[0102] In the formula, Represents the updated behavior parameters; Indicates the updated state parameters; Represents the soft update coefficient.

[0103] The present invention has the following beneficial effects:

[0104] 1. Considering the voltage stability of the distribution network, an MDP model with unknown transfer probability is established to solve the problem of electric vehicle charging. And by balancing the electric vehicle charging target and the voltage stability target through the designed reward function, it is possible to learn an economic dispatch strategy and maintain the voltage stability of the distribution network;

[0105] 2. The model-free DDPG algorithm is introduced to solve the coordination optimization problem, which can output mixed continuous scheduling signals and discrete control signals. It is data-driven and does not rely on the uncertain model in the system.

[0106] 3. A two-layer coordinated EV charging and voltage control framework is adopted to minimize EV charging cost and stabilize distribution network voltage.

[0107] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 A flow chart of an electric vehicle distribution network optimization dispatching method of the present invention;

[0109] Figure 2 A coordinated electric vehicle charging and voltage control framework diagram of an electric vehicle distribution network optimization scheduling method of the present invention;

[0110] Figure 3 IEEE33-bus distribution network layout diagram for the simulation experiment of the present invention;

[0111] Figure 4 This is a diagram of node voltage simulation results of the simulation experiment of the present invention. DETAILED DESCRIPTION

[0112] In order to make the purpose, technical scheme and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0113] It should be noted that the terms "include", "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or devices.

[0114] Like reference numerals and letters denote similar items in the following drawings, and thus, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0115] like Figure 1 As shown, a method for optimizing the dispatching of an electric vehicle distribution network comprises the following steps:

[0116] S1. Considering the system state variables, an MDP model is constructed based on the electric vehicle charging plan and the distribution network voltage control problem. Through the coordinated control of VVC equipment, electric vehicles and distributed generators, automatic voltage regulation and reduction of electric vehicle charging costs are achieved;

[0117] The system state described in step S1 includes the uncertainty of the charging process caused by the charging behavior of the electric vehicle user, as well as the uncertainty of uncontrollable load, system dynamic electricity price and renewable energy generation.

[0118] Step S1 specifically includes the following steps:

[0119] S11, such as Figure 2 As shown, a collaborative electric vehicle charging and voltage control framework is constructed, the collaborative electric vehicle charging and voltage control framework includes electric vehicles, distributed generators and renewable energy, and the electric vehicles are controlled by intelligent terminals;

[0120] S12, building a controllable unit based on a collaborative electric vehicle charging and voltage control framework;

[0121] The controllable unit construction steps described in step S12 are as follows:

[0122] S121. Building electric vehicles:

[0123] Assuming that the control variable of the electric vehicle is the charging power, then for the node connected to the distribution network The electric vehicle, the control variable is expressed as ; In V2G mode, it is the discharge power. The value of is positive during charging and negative during discharging. The charging power constraint based on the charging strategy is:

[0124] (1)

[0125] In the formula, The maximum charging power for electric vehicles; is the maximum discharge power of the electric vehicle; For the scheduling period; is a node set; is a set of time slots;

[0126] And the charging state of the electric vehicle battery meets the following constraints during the scheduling period:

[0127] (2)

[0128] (3)

[0129] In the formula, and Electric vehicle batteries Moment and Charging status at all times; is the capacity of the electric vehicle battery; and are the charge rate and discharge rate respectively; and Respectively represent the maximum and minimum values ​​of the charging state; is the interval of one time step;

[0130] S122. Construction of distributed generators:

[0131] Assume that the control variables of the distributed generator are active power output and reactive power output, then the node Active power of distributed generators at and reactive power The following constraints are met:

[0132] (4)

[0133] (5)

[0134] In the formula, is the maximum active power; is the minimum reactive power;

[0135] S123. Construction of on-load tap changer and voltage regulator:

[0136] On-load tap changers and voltage regulators are connected to distribution network branches by changing the tap position control The control variable is expressed as , varies within the range of integers and satisfies the following constraints:

[0137] (6)

[0138] In the formula, is the maximum adjustable position of the tap; It is a collection of distribution network branches;

[0139] S124. Constructing a switchable capacitor bank:

[0140] The value of reactive power provided by the switchable capacitor bank is set by adjusting the number of operating units. Number of operating units of switchable capacitor banks at Take an integer and satisfy the following conditions:

[0141] (7)

[0142] In the formula, The maximum number of operation units.

[0143] S13. Model the electric vehicles, distributed generators and voltage control devices in the distribution network system, and set operating constraints to obtain the MDP model. The upper layer of the MDP model is set as the scheduling layer, which is used to output control signals of all dispatchable units according to the system status. The lower layer of the MDP model is the voltage control layer, which is used to receive the upper layer control signals and control the dispatchable units in the DN system.

[0144] The operation constraints described in step S13 are as follows:

[0145] Constrain the complex power that the substation can withstand:

[0146] (8)

[0147] In the formula, , , They are active power, reactive power and maximum apparent power of the substation respectively;

[0148] Constraining node voltages :

[0149] (9)

[0150] In the formula, and are the maximum and minimum values ​​of the node voltage respectively;

[0151] Constraining branch current :

[0152] (10)

[0153] In the formula, and are the maximum and minimum values ​​of the branch current respectively.

[0154] Preferably, the MDP model construction steps described in step S13 are as follows:

[0155] The first step is to set the state variables of the distribution network system:

[0156] (11)

[0157] In the formula, For the distribution network system at any time state variables; Node Department Active power demand at each moment; Node Department Active power demand at each moment; They are Historical electricity prices at the time; and Node Wind turbine power output and photovoltaic power output at Node The charge state at Start charging time; Node The trigger time at They are The expected state of charge;

[0158] Assuming that wind turbines and photovoltaics are uncontrollable power sources, calculate the node Net load requirement :

[0159] (12)

[0160] Substituting formula (12) into formula (11) yields:

[0161] (13)

[0162] In the formula, Points Department Net load demand at the moment;

[0163] Step 2: Set the action variables of the distribution network system:

[0164] The system action variables include the active power and reactive power of the generator, the charging capacity of the electric vehicle, the tap position of the on-load tap changer, and the number of switchable capacitor banks:

[0165] (14)

[0166] In the formula, is the action variable of the distribution network system; Respectively represent nodes Active power of distributed generators at Respectively represent nodes Reactive power of distributed generators at Respectively represent nodes The number of operating units of the switchable capacitor bank at Respectively represent nodes The tap position of the on-load tapchanger at

[0167] The third step is to set the reward function by considering the operating cost of the distribution network system and the voltage stability target. The operating cost includes the cost of the distribution network interacting with the main grid through the substation, the generation cost of distributed generators, and the charging cost of electric vehicles:

[0168] (15)

[0169] In the formula, is the weight coefficient, which indicates that the reward function is a trade-off between the operating cost and the voltage stability target; The cost of purchasing electricity from the power exchange; For Node The generation cost of distributed generators at For Node The cost of charging electric vehicles in the country; Penalty for violating voltage constraints;

[0170] Among them, the cost of purchasing electricity from the power exchange The calculation formula is as follows:

[0171] (16)

[0172] In the formula, They are Historical electricity prices at the time;

[0173] node The cost of distributed generators at The calculation formula is as follows:

[0174] (17)

[0175] In the formula, , , All are power generation cost coefficients;

[0176] Charging costs for electric vehicles The calculation formula is as follows:

[0177] (18)

[0178] In the formula, For Node Historical electricity prices at

[0179] Penalty for Violating Voltage Constraints The calculation formula is as follows:

[0180] (19)

[0181] In the formula, is the penalty coefficient of voltage deviation;

[0182] Step 4: Set the objective function.

[0183] The objective function calculation formula is as follows:

[0184] (20)

[0185] In the formula, is the expected cumulative discount return during the planning period; They are The reward for the time step; They are The discount factor for the time step; Indicates sum.

[0186] S2. Based on the electric vehicle charging plan and distribution network voltage control problems, a deep neural network framework is constructed, and the deep deterministic policy gradient algorithm is used to train the deep neural network to output discrete and continuous control actions.

[0187] The deep deterministic policy gradient algorithm described in step S2 is a two-layer structure, in which the scheduling signal and the control signal are both output by the upper layer of the agent, and the lower layer obtains the complete reward value by controlling the controllable units in the distribution network.

[0188] Step S2 specifically includes the following steps:

[0189] S21. In the reinforcement learning framework, the learning agent interacts with the constructed MDP model to transform the optimization problem into a standard reinforcement learning framework. as follows:

[0190] (twenty one)

[0191] In the formula, for status; represents the expected return; Indicates the state of Follow the strategy Expected Reward The action value function of is the set of expected returns;

[0192] S22. Solve the optimal action-value function by recursively solving the Bellman equation , and based on the optimal strategy obtained , describing the optimization problem:

[0193] (twenty two)

[0194] In the formula, Represents the optimal strategy Expected value; represents the reward at time t; represents the discount factor; Indicates the state of Optimal strategy Expected Reward The optimal action value function of

[0195] S23. Construct a deep deterministic policy gradient algorithm to determine the critic parameters:

[0196] (twenty three)

[0197] In the formula, Indicates the batch size of samples sampled from the replay buffer; Represents the target of the loss function; represents an approximate function; Indicates the current The target value of the value;

[0198] S24. Update the parameters of the participant value network based on the deep deterministic policy gradient algorithm:

[0199] (twenty four)

[0200] In the formula, Represents a function approximator In the deterministic target strategy The gradient operator under ; Indicates the expected return from starting distribution; represents the updated expected value; represents the participant function; express; represents the gradient of the weight; Indicates behavior parameters; Indicates the status parameter.

[0201] S25. Soft-update the target network using the updated critic and actor networks until a converged strategy is obtained:

[0202] (25)

[0203] In the formula, Represents the updated behavior parameters; Indicates the updated state parameters; Represents the soft update coefficient.

[0204] Preferably, after step S24, step S25 is also included;

[0205] It specifically includes the following steps:

[0206] Soft-update the target network using the updated critic and actor networks until a converged policy is obtained:

[0207] (25)

[0208] In the formula, Represents the updated behavior parameters; Indicates the updated state parameters; Represents the soft update coefficient.

[0209] Simulation experiment:

[0210] A personal computer with an NVIDIA RTX-3070 GPU and an Intel (R) Core (TM) i7-10700K CPU was used for training and testing. The code was written in Python 3.7.8, and the reinforcement learning algorithm was implemented using the deep learning package TensorFlow 1.14.0. Figure 3 The test was conducted on the IEEE-33 node system shown in the figure. Electric vehicle charging stations were set up on nodes 8, 13, 19, 22, and 29, and each node can be connected to a different number of electric vehicles. The maximum charging and discharging power of each vehicle is set to 6 kW, and the battery capacity is set to 24 kWh. The charging and discharging efficiency of the electric vehicle is set to 0.98 and 0.95 respectively.

[0211] Test results such as Figure 4 As shown, it can be seen that the voltage of the system decreases as the load of electric vehicles in the system increases. Controlling the voltage while dispatching electric vehicles can keep the voltage of each node in the system within a safe range. This embodiment can well take into account the charging needs of electric vehicles and the voltage stability of the distribution network.

[0212] Therefore, the present invention adopts the above-mentioned electric vehicle distribution network optimization scheduling method, and enables the agent to learn an economical scheduling strategy through a two-layer scheduling control strategy, and maintain the voltage stability of the distribution network. It is data-driven and does not rely on the uncertain model in the system.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for optimizing the dispatching of an electric vehicle distribution network, characterized in that: The following steps are involved: S1. Considering the system state variables, an MDP model is constructed based on the electric vehicle charging plan and the distribution network voltage control problem. Through the coordinated control of VVC equipment, electric vehicles and distributed generators, automatic voltage regulation and reduction of electric vehicle charging costs are achieved; The system state described in step S1 includes the uncertainty of the charging process caused by the charging behavior of the electric vehicle user and the uncertainty of the uncontrollable load, the system dynamic electricity price and the renewable energy generation; Step S1 specifically includes the following steps: S11. Construct a collaborative electric vehicle charging and voltage control framework, which includes electric vehicles, distributed generators and renewable energy, and the electric vehicles are controlled by smart terminals; S12, building a controllable unit based on a collaborative electric vehicle charging and voltage control framework; The controllable unit construction steps described in step S12 are as follows: S121. Building electric vehicles: Assuming that the control variable of the electric vehicle is the charging power, then for the node connected to the distribution network The control variables are expressed as ; In V2G mode, it is the discharge power. The value of is positive during charging and negative during discharging. The charging power constraint based on the charging strategy is: (1) In the formula, The maximum charging power for electric vehicles; is the maximum discharge power of the electric vehicle; For the scheduling period; is a node set; is a set of time slots; And the charging state of the electric vehicle battery meets the following constraints during the scheduling period: (2) (3) In the formula, and Electric vehicle batteries Moment and Charging status at all times; is the capacity of the electric vehicle battery; and are the charge rate and discharge rate respectively; and Respectively represent the maximum and minimum values ​​of the charging state; is the interval of one time step; S122. Construction of distributed generators: Assume that the control variables of the distributed generator are active power output and reactive power output, then the node Active power of distributed generators at and reactive power The following constraints are met: (4) (5) In the formula, is the maximum active power; is the minimum reactive power; S123. Construction of on-load tap changer and voltage regulator: On-load tap changers and voltage regulators are connected to distribution network branches by changing the tap position control The control variable is expressed as , varies within the range of integers and satisfies the following constraints: (6) In the formula, is the maximum adjustable position of the tap; It is a collection of distribution network branches; S124. Constructing a switchable capacitor bank: The reactive power value provided by the switchable capacitor bank is set by adjusting the number of operating units. Number of operating units of switchable capacitor banks at Take an integer and satisfy the following conditions: (7) In the formula, is the maximum number of operating units; S13, modeling electric vehicles, distributed generators and voltage control devices in the distribution network system, and setting operation constraints to obtain an MDP model, wherein the upper layer of the MDP model is set as a scheduling layer, which is used to output control signals of all dispatchable units according to the system state, and the lower layer of the MDP model is a voltage control layer, which is used to receive upper layer control signals and control dispatchable units in the DN system; S2. Based on the electric vehicle charging plan and distribution network voltage control problems, a deep neural network framework is constructed, and the deep deterministic policy gradient algorithm is used to train the deep neural network to output discrete and continuous control actions.

2. The method for optimizing and dispatching an electric vehicle distribution network according to claim 1, characterized in that: The operation constraints described in step S13 are as follows: Constrain the complex power that the substation can withstand: (8) In the formula, , , They are active power, reactive power and maximum apparent power of the substation respectively; Constraining node voltages : (9) In the formula, and are the maximum and minimum values ​​of the node voltage respectively; Constraining branch current : (10) In the formula, and are the maximum and minimum values ​​of the branch current respectively.

3. The method for optimizing and dispatching an electric vehicle distribution network according to claim 2, characterized in that: The steps of constructing the MDP model described in step S13 are as follows: The first step is to set the state variables of the distribution network system: （11） In the formula, For the distribution network system at any time state variables; Node Department Active power demand at each moment; Node Department Active power demand at each moment; They are Historical electricity prices at the time; and Node Wind turbine power output and photovoltaic power output at Node The charge state at Start charging time; Node The trigger time at They are The expected state of charge; Assuming that wind turbines and photovoltaics are uncontrollable power sources, calculate the node Net load requirement : （12） Substituting formula (12) into formula (11) yields: （13） In the formula, Points Department Net load demand at the moment; Step 2: Set the action variables of the distribution network system: The system action variables include the active power and reactive power of the generator, the charging capacity of the electric vehicle, the tap position of the on-load tap changer, and the number of switchable capacitor banks: （14） In the formula, is the action variable of the distribution network system; Respectively represent nodes Active power of distributed generators at Respectively represent nodes Reactive power of distributed generators at Respectively represent nodes The number of operating units of the switchable capacitor bank at Respectively represent nodes The tap position of the on-load tapchanger at The third step is to set the reward function by considering the operating cost of the distribution network system and the voltage stability target. The operating cost includes the cost of the distribution network interacting with the main grid through the substation, the generation cost of distributed generators, and the charging cost of electric vehicles: （15） In the formula, is the weight coefficient, which indicates that the reward function is a trade-off between the operating cost and the voltage stability target; The cost of purchasing electricity from the power exchange; For Node The generation cost of distributed generators at For Node The cost of charging electric vehicles in the country; Penalty for violating voltage constraints; Among them, the cost of purchasing electricity from the power exchange The calculation formula is as follows: （16） In the formula, They are Historical electricity prices at the time; node The cost of distributed generators at The calculation formula is as follows: （17） In the formula, , , All are power generation cost coefficients; Charging costs for electric vehicles The calculation formula is as follows: （18） In the formula, For Node Historical electricity prices at Penalty for Violating Voltage Constraints The calculation formula is as follows: （19） In the formula, is the penalty coefficient of voltage deviation; Step 4: Set the objective function.

4. The method for optimizing the dispatching of an electric vehicle distribution network according to claim 3, characterized in that: The objective function calculation formula is as follows: （20） In the formula, is the expected cumulative discount return during the planning period; They are The reward for the time step; They are The discount factor for the time step; Indicates sum.

5. The method for optimizing and dispatching an electric vehicle distribution network according to claim 4, characterized in that: The deep deterministic policy gradient algorithm described in step S2 is a two-layer structure, in which the scheduling signal and the control signal are both output by the upper layer of the agent, and the lower layer obtains the complete reward value by controlling the controllable unit in the distribution network.

6. The method for optimizing and dispatching an electric vehicle distribution network according to claim 5, characterized in that: Step S2 specifically includes the following steps: S21. In the reinforcement learning framework, the learning agent interacts with the constructed MDP model to transform the optimization problem into a standard reinforcement learning framework. as follows: （21） In the formula, for status; represents the expected return; Indicates the state of Follow the strategy Expected Reward The action value function of is the set of expected returns; S22. Solve the optimal action-value function by recursively solving the Bellman equation , and based on the optimal strategy obtained , describing the optimization problem: （22） In the formula, Represents the optimal strategy Expected value; represents the reward at time t; represents the discount factor; Indicates the state of Optimal strategy Expected Reward The optimal action value function of S23. Construct a deep deterministic policy gradient algorithm to determine the critic parameters: （23） In the formula, Indicates the batch size of samples sampled from the replay buffer; represents the target of the loss function; represents an approximate function; Indicates the current The target value of the value; S24. Update the parameters of the participant value network based on the deep deterministic policy gradient algorithm: （24） In the formula, Represents a function approximator In the deterministic target strategy The gradient operator under ; Indicates the expected return from starting distribution; represents the updated expected value; represents the participant function; express; represents the gradient of the weight; Indicates behavior parameters; Indicates the state parameter; S25. Soft-update the target network using the updated critic and actor networks until a converged strategy is obtained: （25） In the formula, Represents the updated behavior parameters; Indicates the updated state parameters; Represents the soft update coefficient.

7. The method for optimizing and dispatching an electric vehicle distribution network according to claim 6, characterized in that: After step S24, step S25 is also included; It specifically includes the following steps: Soft-update the target network using the updated critic and actor networks until a converged policy is obtained: （25） In the formula, Represents the updated behavior parameters; Indicates the updated state parameters; Represents the soft update coefficient.

Images