Control system and method for accessing urban rail transit direct current traction network

Abstract

The embodiment of the invention provides a control system and method for accessing multiple photovoltaic energy storage systems to an urban rail transit direct current traction network, and the system comprises a photovoltaic unit which is connected to the direct current traction network through a Boost type converter, and is used for adjusting the working efficiency according to the working mode of the system, and enabling the photovoltaic unit to be connected to the direct current traction network through a Boost type converter. The solar energy is converted into electric energy and sent to the energy storage unit; the energy storage unit is connected with the photovoltaic unit and the train load unit and is used for absorbing or transmitting electric energy to a direct-current traction network through a Buck / Boost bidirectional converter according to a system mode; the traction substation is used for carrying out power transformation processing on the electric energy of the drainage traction network and providing the electric energy subjected to the power transformation processing to the train load unit; and the train load unit is used for receiving the electric energy of the energy storage unit and the traction substation for train operation.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a control system and method for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit. Background Art

[0002] Supplying the electricity generated by the photovoltaic power generation system to the urban rail transit system can not only promote the consumption of renewable energy, but also alleviate the power supply pressure of the urban rail transit system, and promote the construction of "green transportation" while reducing operating costs, which is in line with the basic national policy of establishing a low-carbon transportation model.

[0003] Domestic and foreign research and practical applications of photovoltaic systems connected to urban rail transit are mainly focused on the power supply of station lighting or signal systems, and mainly consider 400V, 35kV / 10kV AC grid connection. There is still little design research on the connection of photovoltaic units to the DC side of the rail transit traction power supply system to directly supply energy to urban rail trains. In addition, the complexity of the circuit topology of the traction power supply system, the time-varying nature of the train load power, photovoltaic output power, and traction substation voltage, have brought difficulties to the analytical solution of the optimal control strategy. Summary of the invention

[0004] The object of the present invention is to provide a control system and method for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit, aiming to solve the above-mentioned problems in the prior art.

[0005] The present invention provides a control system for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit, comprising:

[0006] Photovoltaic unit, which is connected to the DC traction grid through a Boost converter and is used to adjust the working efficiency according to the system working mode, convert solar energy into electrical energy and send it to the energy storage unit;

[0007] The energy storage unit is connected to the photovoltaic unit and the train load unit and is used to absorb or transmit electric energy to the DC traction grid through the Buck / Boost bidirectional converter according to the system mode;

[0008] The traction substation is used to transform the electric energy of the traction network and provide the transformed electric energy to the train load unit;

[0009] The train load unit is used to receive electric energy from the energy storage unit and the traction substation for train operation.

[0010] The present invention provides a control system for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit, and the control system for connecting the multi-photovoltaic energy storage system to a DC traction network of urban rail transit comprises:

[0011] The photovoltaic unit is connected to the DC traction grid through a Boost-type converter, which is used to adjust the working efficiency according to the system working mode, convert solar energy into electrical energy and send it to the energy storage unit;

[0012] The energy storage unit absorbs or transmits electric energy to the DC traction grid through the Buck / Boost bidirectional converter according to the system mode;

[0013] The traction substation transforms the electric energy from the traction network and provides the transformed electric energy to the train load unit;

[0014] The train load unit receives electric energy from the energy storage unit and the traction substation for train operation.

[0015] By adopting the embodiments of the present invention, it is possible to control the reasonable charging and discharging of each energy storage unit, maintain the SOC balance of each energy storage unit while maintaining the voltage of the DC traction network, and avoid the situation of overcharging or over-discharging of energy storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 It is a schematic diagram of a control system of a multi-photovoltaic energy storage system connected to a DC traction network of urban rail transit according to an embodiment of the present invention;

[0018] Figure 2 is a schematic diagram of a multi-photovoltaic-energy storage system topology structure according to an embodiment of the present invention;

[0019] Figure 3 is a diagram showing the relationship between the energy management layer and the converter control layer of an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of a reinforcement learning framework for a multi-energy coupling system according to an embodiment of the present invention;

[0021] Figure 5 It is a flow chart of a control method for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0023] System Example

[0024] According to an embodiment of the present invention, a control system for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit is provided. Figure 1 Schematic diagram of a control system for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit according to an embodiment of the present invention. Figure 1 As shown, the control system for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit according to an embodiment of the present invention specifically includes:

[0025] Photovoltaic unit 10, the photovoltaic unit is connected to the DC traction grid through a Boost type converter, and is used to adjust the working efficiency according to the system working mode, convert solar energy into electrical energy and send it to the energy storage unit; specifically used for:

[0026] According to the balance relationship between energy supply and demand of each unit in the system during steady-state operation in different time periods as shown in Formula 1, the system operating mode is determined, wherein the system operating mode specifically includes: light abandonment mode and maximum power point tracking mode:

[0027]

[0028] Among them, P 列车牵引(t) Indicates the train load traction power, P ESch(t) Represents the power of the energy storage unit, P PV(t) Indicates the power of the traction substation, P 列车制动(t) Indicates the regenerative braking power of the train load, t0 indicates the start time of the abandoned light mode, t1 indicates the end time of the abandoned light mode and the start time of the high power point tracking mode, and t2 indicates the end time of the high power point tracking mode; the system working mode from t0 to t1 is the abandoned light mode, and the system working mode from t1 to t2 is the high power point tracking mode;

[0029] In the abandonment mode, the system power generation is reduced, and in the maximum power point tracking mode, the maximum power generation is achieved.

[0030] The energy storage unit 12 is connected to the photovoltaic unit and the train load unit, and is used to absorb or transmit electric energy to the DC traction network through the Buck / Boost bidirectional converter according to the system mode; specifically used for:

[0031] Charging according to the voltage threshold when the system is in the light abandonment mode;

[0032] When the system is in maximum power point tracking mode, it is charged and discharged according to the voltage threshold: when the photovoltaic unit is in maximum power point tracking mode, it is set to traction sub-mode or braking sub-mode according to the state of the train load unit. In traction sub-mode, it is in discharge state, and the photovoltaic unit and traction substation supply power to the train load unit; in braking sub-mode, it is in charging state, and absorbs the electric energy of the photovoltaic unit and traction substation at the same time.

[0033] The control strategies are determined to be the system energy management layer strategy and the DC / DC converter control layer strategy. The system energy management layer strategy adopts the energy management strategy to generate control instructions for charging and discharging the energy storage unit according to the voltage of the traction substation, the train load unit, the photovoltaic unit, and the energy storage unit status in the multi-energy coupling system; the DC / DC converter control layer strategy receives the control instructions and acts on the Buck / Boost bidirectional converter of the energy storage unit, and executes the control instructions by controlling the opening and closing of the Buck / Boost bidirectional converter switch tube.

[0034] A dynamic charge and discharge threshold management strategy for energy storage systems based on deep reinforcement learning is adopted, in which the optimization objective of the dynamic charge and discharge threshold management strategy for energy storage systems is max As shown in Formula 2-Formula 4, the constraints include: The constraints include the Kirchhoff circuit equation constraints satisfied by the traction power supply system, the voltage-current constraint equations of each circuit element, the total power utilization constraint, and the photovoltaic utilization efficiency constraint:

[0035] J max =v%+e% Formula 2;

[0036]

[0037]

[0038] Among them, e% represents the energy saving rate, v% represents the voltage regulation rate, upv0 and ipv0 represent the voltage and current of the photovoltaic device when it is always connected, and u pvs 、i pvs Indicates the actual photovoltaic voltage and current after adopting the control strategy, u s is the actual voltage of the traction substation, u0 is the rated voltage of the traction substation, and T represents the cumulative time that the PV unit is connected to the DC traction grid;

[0039] Establish a DC traction network multi-energy coupling system reinforcement learning framework, perform deep reinforcement learning training based on the DC traction network multi-energy coupling system reinforcement learning framework, and obtain a trained energy storage system dynamic charge and discharge threshold management model;

[0040] Energy management is performed through the energy storage system dynamic charge and discharge threshold management model.

[0041] Establish a reinforcement learning framework for the DC traction network multi-energy coupling system: determine the state space s according to formula 5, and normalize the state quantity according to formula 6:

[0042] s=[p,SOC,P pv ,u rec ] Formula 5;

[0043]

[0044] Where p represents the train load, P pv Represents photovoltaic output power, u rec represents the voltage of the rectifier unit of the substation, SOC represents the state of charge of the energy storage unit, z represents the value after normalization of the state quantity, x refers to all state variables, such as power, voltage, etc., μ and σ are the mean and variance of the data set respectively;

[0045] According to formula 9, the action space a is determined, and the action amount is the charge and discharge threshold of the energy storage unit. By changing the adjustment factor k ch , k dis The value is taken to realize dynamic adjustment of the threshold, where the relationship between the adjustment factor and the threshold is shown in Formula 7-Formula 8:

[0046] u ch =u s0 +k ch ·△u Formula 7;

[0047] u dis =u s0 -k dis ·△u Formula 8;

[0048] a=[k ch , k dis ] Formula 9;

[0049] Among them, u s0 =750V, △u=12.5, k ch , k dis Dynamically changes according to energy management strategies in [1, 5];

[0050] Determine the dimension of the input and output variables of the neural network in DQN, where the number of fully connected hidden layers of the neural network is h, and the neuron structure of each layer is u = [u1, u2...uh ], the activation function of the neuron selects the linear correction function (RELU), as shown in formula 10:

[0051] f(x)=max(0,x) Formula 10;

[0052] Among them, x refers to the input variable of the neural network;

[0053] Determine the reward function r according to formula 11: the reward function is the weighted sum of the energy saving rate and the voltage stabilization rate within the unit time step △t, and the weight coefficient is 0.5:

[0054] r = -0.5△v - 0.5△e Formula 11;

[0055] Among them, △v represents the voltage change, and △e represents the power change.

[0056] According to the DC traction network multi-energy coupling system reinforcement learning framework, deep reinforcement learning training is performed to obtain a trained energy storage system dynamic charging and discharging threshold management model: in a round of learning, the agent extracts the system state s from the environment as the input of the action-value function composed of a deep neural network inside the agent. The agent compares the Q values ​​corresponding to all possible action quantities a, and selects the action corresponding to the maximum Q value to output to the environment, changing the state quantity of the environment. The agent obtains the corresponding reward r for this round of learning, and changes the internal parameters of the deep neural network according to the reward to carry out the next learning process. In the repeated learning and training process, the agent modifies the strategy according to the feedback reward value to maximize the cumulative reward.

[0057] The traction substation 14 is used to transform the electric energy of the traction network and provide the transformed electric energy to the train load unit;

[0058] The train load unit 16 is used to receive electric energy from the energy storage unit and the traction substation for train operation.

[0059] In summary, the embodiment of the present invention proposes a novel photovoltaic-storage-conversion system control method for the DC traction network. The photovoltaic system participates in the DC traction power supply, and the energy storage unit can absorb excess photovoltaic power and recycle the regenerative braking energy of the train. Reinforcement learning does not rely on given label data. It obtains data through interaction with the environment and performs agent strategies, making it a new idea for energy management of urban rail transit systems with photovoltaic storage. The embodiment of the present invention takes energy saving rate and voltage regulation rate as optimization targets, establishes a photovoltaic-energy storage system reinforcement learning framework, and performs strategy learning and optimization through the agent's "trial and error" and "feedback" mechanism to solve the source-storage-load energy optimization problem of the urban rail transit system.

[0060] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings.

[0061] Figure 2 The photovoltaic-energy storage system is connected to the DC traction network topology of urban rail transit. The system includes photovoltaic units, energy storage units, traction substations, train load units, etc. Among them, the photovoltaic unit is connected to the DC traction network through a Boost converter, and the energy storage unit absorbs or transmits energy to the DC traction network through a Buck / Boost bidirectional converter.

[0062] 1. Coordinated control method of multiple photovoltaic-energy storage systems

[0063] Photovoltaic and energy storage units have different working characteristics. Specifically, the photovoltaic unit can operate in maximum power point tracking (MPPT) mode and abandonment mode, and the energy storage unit operates in charging and discharging mode, and shuts down when the battery state of charge (SOC) setting range is exceeded.

[0064]

[0065] Formula (2-2-1) is the balance relationship of energy supply and demand of each unit in the system during steady-state operation in different time periods. The system works in mode 1 from t0 to t1, and in mode 2 from t1 to t2. It can be seen from this formula that the energy change of a unit in the system will break the energy balance relationship of the system during steady-state operation, which is mainly manifested in the changes in the energy consumed by train load traction or generated by braking, the changes in photovoltaic units due to voltage, light intensity, temperature, etc., the changes in charging and discharging power of energy storage due to SOC, etc. The traction network voltage will rise or fall as a result, and each unit will switch the control strategy accordingly, and the system will switch the operation mode accordingly. The following is a description of each mode:

[0066] Operation mode 1: Because the energy emitted by the photovoltaic unit is greater than the sum of the load and energy storage consumption, or the load regenerative braking energy and the photovoltaic energy emitted are greater than the energy storage consumption, the traction network voltage rises above the limit. At this time, the photovoltaic unit operates in the abandonment mode, reducing the system power generation, and the energy storage unit is charged according to the voltage threshold.

[0067] Operation mode 2: When the photovoltaic energy decreases or the load consumption increases, and the load regenerative braking energy decreases, the traction network voltage does not exceed the limit. At this time, the photovoltaic unit works in the MPPT (Maximum Power Point Tracking) mode to achieve maximum energy utilization, and the energy storage unit is charged and discharged according to the voltage threshold. In this mode, according to the state of the train load, it can be divided into two sub-modes: traction and braking. During traction, the energy storage unit is in a discharging state, and the photovoltaic unit and traction substation supply power to the train load; during braking, the energy storage unit is in a charging state, and the photovoltaic unit and traction substation also charge the energy storage.

[0068] 2. Energy management strategy under multi-source coupling (energy management layer)

[0069] The control strategy of the energy storage unit is divided into two parts: the system energy management layer and the DC / DC converter control layer. The relationship between the two parts is as follows: Figure 3 As shown. The system energy management layer generates control instructions for energy storage charging and discharging according to the adopted energy management strategy, the traction voltage, train load, photovoltaic unit, and energy storage unit status in the multi-energy coupling system; the DC / DC converter control layer is responsible for receiving control instructions and acting on the energy storage unit converter, and tracking the instructions by controlling the opening and closing of the converter switch tube.

[0070] The embodiment of the present invention also proposes a dynamic charge and discharge threshold management strategy for energy storage systems based on deep reinforcement learning. The optimization objectives and constraints of the strategy are as follows:

[0071] 1. Optimization goals

[0072] J max =v%+e%#(2-3-4)

[0073] The optimization target is as shown in formula (2-3-4), where e% represents the energy saving rate and v% represents the voltage stabilization rate. The weights of both are equal to the target J. The evaluation indicators of energy saving rate and voltage stabilization rate are as follows: pv0 、i pv0 Indicates that the photovoltaic power plant is always connected to the lower voltage and current, u pvs i pvs Indicates the actual photovoltaic voltage and current after adopting the control strategy, u s is the actual voltage of the traction substation, and u0 is the rated voltage of the traction substation.

[0074]

[0075] 2. Constraints

[0076] The constraints include Kirchhoff circuit equation constraints satisfied by the traction power supply system, voltage-current constraint equations of each circuit element, total power utilization constraints, and photovoltaic utilization efficiency constraints.

[0077] Establish a reinforcement learning framework for multi-energy coupling systems in DC traction grids, such as Figure 4 As shown in the figure, the energy management system is an agent that performs the learning and training functions of the energy management strategy algorithm based on deep reinforcement learning. The multi-energy coupled DC traction network is the agent's environment. The state space, action space, and reward function are defined as follows.

[0078] (1) State space s

[0079] s includes the train load, the state of charge (SOC) of the energy storage unit, the output power of the photovoltaic unit, and the voltage of the traction substation where the system is located. s is expressed as:

[0080] s=[p,SOC,P pv ,u rec ]#(2-3-7)

[0081] Where p represents the train load, P pv Represents photovoltaic output power, u rec Represents the rectifier unit voltage of the substation. In order to facilitate the convergence of the energy management strategy algorithm based on deep reinforcement learning, the state quantity is normalized according to formula (2-3-8), where μ and σ are the mean and variance of the data set respectively.

[0082]

[0083] (2) Action space a

[0084] The action amount is the charge and discharge threshold of the energy storage unit, which is adjusted by changing the adjustment factor k ch , k dis The value is used to dynamically adjust the threshold. The relationship between the adjustment factor and the threshold is as follows:

[0085] u ch =u s0 +k ch ·Δu#(2-3-9)

[0086] u dis =u s0 -k dis ·Δu#(2-3-10)

[0087] where u s0 =750V, Δu=12.5, action a is expressed as follows, k ch , k dis It changes dynamically according to the energy management strategy in [1,5].

[0088] a=[k ch ,k dis ]#(2-3-11)

[0089] After the state space and action space are determined, the dimension of the input and output variables of the neural network in DQN can be determined. The number of fully connected hidden layers of the neural network is h, and the neuron structure of each layer is u = [u1,u2...u h ]. The activation function of the neuron is the rectified linear unit (RELU), as shown below.

[0090] f(x)=max(0,x)#(2-3-12)

[0091] (3) Reward function r

[0092] The reward function is the weighted sum of the energy saving rate and voltage stabilization rate within the unit time step Δt, and the weight coefficient is 0.5:

[0093] r=-0.5Δv-0.5Δe#(2-3-13)

[0094] The training process of the energy management method based on the energy management strategy of deep reinforcement learning is shown in Table 1. In a round of learning, the agent extracts the state quantity s of the system from the environment as the input quantity of the action-value function composed of the deep neural network inside the agent. The agent compares the Q values ​​corresponding to all possible action quantities a, and selects the action corresponding to the maximum Q value to output to the environment, changing the state quantity of the environment. The agent obtains the corresponding reward r of this round of learning, and changes the internal parameters of the deep neural network according to the reward, and proceeds to the next learning process. In the repeated learning and training process, the agent modifies the strategy according to the feedback reward value to maximize the cumulative reward.

[0095] Table 1 Energy management algorithm based on DQN

[0096]

[0097] In summary, it can be seen from the above technical solutions that the setting of the charge and discharge threshold of the energy storage unit affects the voltage distribution of the traction network and the energy flow between the energy storage unit, photovoltaic unit, traction substation, and train load. The traditional fixed threshold control strategy is difficult to handle the instability of photovoltaic units and locomotive loads in the multi-energy coupling system, and the multi-directionality of energy flow. In order to optimize the energy saving and voltage stabilization effects of the system and improve the energy flow efficiency of the traction network, a dynamic charge and discharge threshold management strategy based on deep reinforcement learning is proposed. The proposed dynamic charge and discharge threshold management method based on deep reinforcement learning is used for energy management. Since the charging threshold can be changed dynamically, its ability to suppress voltage is stronger and the power quality is better; and the photovoltaic utilization rate under the dynamic charge and discharge threshold management method based on deep reinforcement learning is better than the fixed threshold method. Photovoltaic can bear more traction substation output power, reduce the power grid purchase cost, and have better economy.

[0098] Method Embodiment

[0099] According to an embodiment of the present invention, a control method for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit is provided, which is used in a control system for connecting the multi-photovoltaic energy storage system to a DC traction network of urban rail transit. Figure 5 is a flow chart of a control method for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit according to an embodiment of the present invention. Figure 5 As shown, the control method for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit according to an embodiment of the present invention specifically includes:

[0100] Step S501, the photovoltaic unit is connected to the DC traction grid through a Boost converter, and is used to adjust the working efficiency according to the system working mode, convert solar energy into electrical energy and send it to the energy storage unit; the photovoltaic unit is connected to the DC traction grid through a Boost converter, and is used to adjust the working efficiency according to the system working mode, specifically including:

[0101] According to the balance relationship between energy supply and demand of each unit in the system during steady-state operation in different time periods as shown in Formula 1, the system operating mode is determined, wherein the system operating mode specifically includes: light abandonment mode and maximum power point tracking mode:

[0102]

[0103] Among them, P 列车牵引(t) Indicates the train load traction power, P ESch(t) Represents the power of the energy storage unit, P PV(t) Indicates the power of the traction substation, P 列车制动(t) Indicates the regenerative braking power of the train load, t0 indicates the start time of the abandoned light mode, t1 indicates the end time of the abandoned light mode and the start time of the high power point tracking mode, and t2 indicates the end time of the high power point tracking mode; the system working mode from t0 to t1 is the abandoned light mode, and the system working mode from t1 to t2 is the high power point tracking mode;

[0104] In the abandoned light mode, the system power generation is reduced, and in the maximum power point tracking mode, the maximum power generation is executed;

[0105] Step S502: The energy storage unit absorbs or transmits electric energy to the DC traction network through the Buck / Boost bidirectional converter according to the system mode; the energy storage unit absorbs or transmits electric energy to the DC traction network through the Buck / Boost bidirectional converter according to the system mode specifically includes:

[0106] Charging according to the voltage threshold when the system is in the light abandonment mode;

[0107] When the system is in the maximum power point tracking mode, it is charged and discharged according to the voltage threshold: when the photovoltaic unit is in the maximum power point tracking mode, it is set to the traction sub-mode or the braking sub-mode according to the state of the train load unit. In the traction sub-mode, it is in the discharging state, and the photovoltaic unit and the traction substation supply power to the train load unit; in the braking sub-mode, it is in the charging state, and the power of the photovoltaic unit and the traction substation is absorbed at the same time;

[0108] The control strategies are determined to be the system energy management layer strategy and the DC / DC converter control layer strategy. The system energy management layer strategy adopts the energy management strategy to generate control instructions for charging and discharging the energy storage unit according to the voltage of the traction substation, the train load unit, the photovoltaic unit, and the energy storage unit status in the multi-energy coupling system; the DC / DC converter control layer strategy receives the control instructions and acts on the Buck / Boost bidirectional converter of the energy storage unit, and executes the control instructions by controlling the opening and closing of the Buck / Boost bidirectional converter switch tube;

[0109] A dynamic charge and discharge threshold management strategy for energy storage systems based on deep reinforcement learning is adopted, in which the optimization objective of the dynamic charge and discharge threshold management strategy for energy storage systems is max As shown in Formula 2-Formula 4, the constraints include: The constraints include the Kirchhoff circuit equation constraints satisfied by the traction power supply system, the voltage-current constraint equations of each circuit element, the total power utilization constraint, and the photovoltaic utilization efficiency constraint:

[0110] J max =v%+e% Formula 2;

[0111]

[0112] Among them, e% represents the energy saving rate, v% represents the voltage stabilization rate, and u% represents the voltage stabilization rate. pv0 、i pv0 Indicates that the photovoltaic power plant is always connected to the lower voltage and current, u pvs 、i pvs Indicates the actual photovoltaic voltage and current after adopting the control strategy, u s is the actual voltage of the traction substation, u0 is the rated voltage of the traction substation, and T represents the cumulative time that the PV unit is connected to the DC traction grid;

[0113] Establish a DC traction network multi-energy coupling system reinforcement learning framework, perform deep reinforcement learning training based on the DC traction network multi-energy coupling system reinforcement learning framework, and obtain a trained energy storage system dynamic charge and discharge threshold management model;

[0114] Energy management is performed through the energy storage system dynamic charge and discharge threshold management model.

[0115] Among them, the reinforcement learning framework of DC traction network multi-energy coupling system is established for:

[0116] The state space s is determined according to Formula 5, and the state quantity is normalized according to Formula 6:

[0117] s=[p,SOC,P pv ,u rec ] Formula 5;

[0118]

[0119] Where p represents the train load, P pv Represents photovoltaic output power, u rec represents the voltage of the rectifier unit of the substation, SOC represents the state of charge of the energy storage unit, z represents the value after normalization of the state quantity, x refers to all state variables such as power and voltage, and μ and σ are the mean and variance of the data set respectively;

[0120] According to formula 9, the action space a is determined, and the action amount is the charge and discharge threshold of the energy storage unit. By changing the adjustment factor k ch , k dis The value is taken to realize dynamic adjustment of the threshold, where the relationship between the adjustment factor and the threshold is shown in Formula 7-Formula 8:

[0121] u ch =u s0 +k ch ·△u Formula 7;

[0122] u dis =u s0 -k dis ·△u Formula 8;

[0123] a=[k ch , k dis ] Formula 9;

[0124] Among them, u s0 =750V, Δu=12.5, k ch , k dis Dynamically changes according to energy management strategies in [1, 5];

[0125] Determine the dimension of the input and output variables of the neural network in DQN, where the number of fully connected hidden layers of the neural network is h, and the neuron structure of each layer is u = [u1, u2...u h ], the activation function of the neuron selects the linear correction function (RELU), as shown in formula 10:

[0126] f(x)=max(0,x) Formula 10;

[0127] Among them, x refers to the input variable of the neural network;

[0128] Determine the reward function r according to formula 11: the reward function is the weighted sum of the energy saving rate and the voltage stabilization rate within the unit time step △t, and the weight coefficient is 0.5:

[0129] r = -0.5△v - 0.5△e Formula 11;

[0130] Among them, Δv represents the voltage change, and Δe represents the power change;

[0131] The deep reinforcement learning training according to the DC traction network multi-energy coupling system reinforcement learning framework specifically includes:

[0132] According to the DC traction network multi-energy coupling system reinforcement learning framework, deep reinforcement learning training is performed to obtain a trained energy storage system dynamic charging and discharging threshold management model: in a round of learning, the agent extracts the system state s from the environment as the input of the action-value function composed of a deep neural network inside the agent. The agent compares the Q values ​​corresponding to all possible action quantities a, and selects the action corresponding to the maximum Q value to output to the environment, changing the state quantity of the environment. The agent obtains the corresponding reward r for this round of learning, and changes the internal parameters of the deep neural network according to the reward to carry out the next learning process. In the repeated learning and training process, the agent modifies the strategy according to the feedback reward value to maximize the cumulative reward.

[0133] Step S503, the traction substation transforms the electric energy of the traction network and provides the transformed electric energy to the train load unit;

[0134] Step S504: the train load unit receives electric energy from the energy storage unit and the traction substation to operate the train.

[0135] The embodiment of the present invention is a method embodiment corresponding to the above-mentioned device embodiment. The specific operation of each step can be understood by referring to the description of the device embodiment, which will not be repeated here.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control system for connecting multiple photovoltaic energy storage systems to a DC traction network of urban rail transit, characterized in that: include: Photovoltaic unit, which is connected to the DC traction grid through a Boost converter and is used to adjust the working efficiency according to the system working mode, convert solar energy into electrical energy and send it to the energy storage unit; The energy storage unit is connected to the photovoltaic unit and the train load unit and is used to absorb or transmit electric energy to the DC traction grid through the Buck / Boost bidirectional converter according to the system mode; The traction substation is used to transform the electric energy of the traction network and provide the transformed electric energy to the train load unit; The train load unit is used to receive electric energy from the energy storage unit and the traction substation for train operation.

2. The system according to claim 1, characterized in that The photovoltaic unit is specifically used for: According to the balance relationship between energy supply and demand of each unit in the system during steady-state operation in different time periods as shown in Formula 1, the system operating mode is determined, wherein the system operating mode specifically includes: light abandonment mode and maximum power point tracking mode: Among them, P 列车牵引(t) Indicates the train load traction power, P ESch(t) Represents the power of the energy storage unit, P PV(t) Indicates the power of the traction substation, P 列车制动(t) Indicates the regenerative braking power of the train load, t0 indicates the start time of the abandoned light mode, t1 indicates the end time of the abandoned light mode and the start time of the high power point tracking mode, and t2 indicates the end time of the high power point tracking mode; the system working mode from t0 to t1 is the abandoned light mode, and the system working mode from t1 to t2 is the high power point tracking mode; In the abandonment mode, the system power generation is reduced, and in the maximum power point tracking mode, the maximum power generation is achieved.

3. The method according to claim 1, characterized in that: The energy storage unit is specifically used for: Charging according to the voltage threshold when the system is in the light abandonment mode; When the system is in maximum power point tracking mode, it is charged and discharged according to the voltage threshold: when the photovoltaic unit is in maximum power point tracking mode, it is set to traction sub-mode or braking sub-mode according to the state of the train load unit. In traction sub-mode, it is in discharge state, and the photovoltaic unit and traction substation supply power to the train load unit; in braking sub-mode, it is in charging state, and absorbs the electric energy of the photovoltaic unit and traction substation at the same time.

4. The system according to claim 1, characterized in that The energy storage unit is specifically used for: The control strategies are determined to be the system energy management layer strategy and the DC / DC converter control layer strategy. The system energy management layer strategy adopts the energy management strategy to generate control instructions for charging and discharging the energy storage unit according to the voltage of the traction substation, the train load unit, the photovoltaic unit, and the energy storage unit status in the multi-energy coupling system; the DC / DC converter control layer strategy receives the control instructions and acts on the Buck / Boost bidirectional converter of the energy storage unit, and executes the control instructions by controlling the opening and closing of the Buck / Boost bidirectional converter switch tube.

5. The system according to claim 1, characterized in that The energy storage unit is specifically used for: A dynamic charge and discharge threshold management strategy for energy storage systems based on deep reinforcement learning is adopted, in which the optimization objective of the dynamic charge and discharge threshold management strategy for energy storage systems is max As shown in Formula 2-Formula 4, the constraints include: The constraints include the Kirchhoff circuit equation constraints satisfied by the traction power supply system, the voltage-current constraint equations of each circuit element, the total power utilization constraint, and the photovoltaic utilization efficiency constraint: J max = v% + e% Formula 2; Among them, e% represents the energy saving rate, v% represents the voltage stabilization rate, and u pv0 、i pv0 Indicates that the photovoltaic power plant is always connected to the lower voltage and current, u pvs 、i pvs Indicates the actual photovoltaic voltage and current after adopting the control strategy, u s is the actual voltage of the traction substation, u0 is the rated voltage of the traction substation, and T represents the cumulative time of the PV unit connected to the DC traction network; Establish a DC traction network multi-energy coupling system reinforcement learning framework, perform deep reinforcement learning training based on the DC traction network multi-energy coupling system reinforcement learning framework, and obtain a trained energy storage system dynamic charge and discharge threshold management model; Energy management is performed through the energy storage system dynamic charge and discharge threshold management model.

6. The system according to claim 5, characterized in that The energy storage unit is specifically used for: Establish a reinforcement learning framework for the DC traction network multi-energy coupling system: determine the state space s according to formula 5, and normalize the state quantity according to formula 6: s=[p,SOC,P pv ,u rec ] Formula 5; Where p represents the train load, P pv Represents photovoltaic output power, u rec represents the rectifier unit voltage of the substation, SOC represents the state of charge of the energy storage unit, z represents the normalized value of the state quantity, x refers to all state variables, μ and σ are the mean and variance of the data set respectively; According to formula 9, the action space a is determined, and the action amount is the charge and discharge threshold of the energy storage unit. By changing the adjustment factor k ch , k dis The value is taken to realize dynamic adjustment of the threshold, where the relationship between the adjustment factor and the threshold is shown in Formula 7-Formula 8: u ch =u s0 +k ch Δu Formula 7; u dis =u s0 -k dis Δu Formula 8; a=[k ch , k dis ] Formula 9; Among them, u s0 =750V, Δu=12.5, k ch , k dis Dynamically changes according to energy management strategies in [1,5]; Determine the dimension of the input and output variables of the neural network in DQN, where the number of fully connected hidden layers of the neural network is h, and the neuron structure of each layer is u = [u1,u2...u h ], the activation function of the neuron selects the linear correction function (RELU), as shown in formula 10: f(x)=max(0,x) Formula 10; Among them, x refers to the input variable of the neural network; Determine the reward function r according to formula 11: the reward function is the weighted sum of the energy saving rate and the voltage stabilization rate within the unit time step Δt, and the weight coefficient is 0.5: r=-0.5Δv-0.5Δe Formula 11; Among them, Δv represents the voltage change, and Δe represents the power change.

7. The system according to claim 5 or 6, characterized in that: The energy storage unit is specifically used for: According to the DC traction network multi-energy coupling system reinforcement learning framework, deep reinforcement learning training is performed to obtain a trained energy storage system dynamic charging and discharging threshold management model: in a round of learning, the agent extracts the system state s from the environment as the input of the action-value function composed of a deep neural network inside the agent. The agent compares the Q values ​​corresponding to all possible action quantities a, and selects the action corresponding to the maximum Q value to output to the environment, changing the state quantity of the environment. The agent obtains the corresponding reward r for this round of learning, and changes the internal parameters of the deep neural network according to the reward to carry out the next learning process. In the repeated learning and training process, the agent modifies the strategy according to the feedback reward value to maximize the cumulative reward.

8. A control method for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit, characterized in that: A control system for connecting a multi-photovoltaic energy storage system to a DC traction network of urban rail transit according to any one of claims 1 to 7, the method comprising: The photovoltaic unit is connected to the DC traction grid through a Boost-type converter, which is used to adjust the working efficiency according to the system working mode, convert solar energy into electrical energy and send it to the energy storage unit; The energy storage unit absorbs or transmits electric energy to the DC traction grid through the Buck / Boost bidirectional converter according to the system mode; The traction substation transforms the electric energy from the traction network and provides the transformed electric energy to the train load unit; The train load unit receives electric energy from the energy storage unit and the traction substation for train operation.

9. The method according to claim 8, characterized in that The photovoltaic unit is connected to the DC traction grid through a Boost converter, which is used to adjust the working efficiency according to the system working mode. Specifically, it includes: According to the balance relationship between energy supply and demand of each unit in the system during steady-state operation in different time periods as shown in Formula 1, the system operating mode is determined, wherein the system operating mode specifically includes: light abandonment mode and maximum power point tracking mode: P 列车牵引(t) Indicates the train load traction power, P ESch(t) Represents the power of the energy storage unit, P PV(t) Indicates the power of the traction substation, P 列车制动(t) Indicates the regenerative braking power of the train load, t0 indicates the start time of the abandoned light mode, t1 indicates the end time of the abandoned light mode and the start time of the high power point tracking mode, and t2 indicates the end time of the high power point tracking mode; the system working mode from t0 to t1 is the abandoned light mode, and the system working mode from t1 to t2 is the high power point tracking mode; In the abandoned light mode, the system power generation is reduced, and in the maximum power point tracking mode, the maximum power generation is achieved; The energy storage unit absorbs or transmits electric energy to the DC traction grid through the Buck / Boost bidirectional converter according to the system mode. Specifically, it includes: Charging according to the voltage threshold when the system is in the light abandonment mode; When the system is in the maximum power point tracking mode, it is charged and discharged according to the voltage threshold: when the photovoltaic unit is in the maximum power point tracking mode, it is set to the traction sub-mode or the braking sub-mode according to the state of the train load unit. In the traction sub-mode, it is in the discharging state, and the photovoltaic unit and the traction substation supply power to the train load unit; in the braking sub-mode, it is in the charging state, and the power of the photovoltaic unit and the traction substation is absorbed at the same time; The control strategies are determined to be the system energy management layer strategy and the DC / DC converter control layer strategy. The system energy management layer strategy adopts the energy management strategy to generate control instructions for charging and discharging the energy storage unit according to the voltage of the traction substation, the train load unit, the photovoltaic unit, and the energy storage unit status in the multi-energy coupling system; the DC / DC converter control layer strategy receives the control instructions and acts on the Buck / Boost bidirectional converter of the energy storage unit, and executes the control instructions by controlling the opening and closing of the Buck / Boost bidirectional converter switch tube; A dynamic charge and discharge threshold management strategy for energy storage systems based on deep reinforcement learning is adopted, in which the optimization objective of the dynamic charge and discharge threshold management strategy for energy storage systems is max As shown in Formula 2-Formula 4, the constraints include: The constraints include the Kirchhoff circuit equation constraints satisfied by the traction power supply system, the voltage-current constraint equations of each circuit element, the total power utilization constraint, and the photovoltaic utilization efficiency constraint: J max = v% + e% Formula 2; Among them, e% represents the energy saving rate, v% represents the voltage stabilization rate, and u pv0 、i pv0 Indicates that the photovoltaic power plant is always connected to the lower voltage and current, u pvs 、i pvs Indicates the actual photovoltaic voltage and current after adopting the control strategy, u s is the actual voltage of the traction substation, u0 is the rated voltage of the traction substation, and T represents the cumulative time of the PV unit connected to the DC traction network; Establish a DC traction network multi-energy coupling system reinforcement learning framework, perform deep reinforcement learning training based on the DC traction network multi-energy coupling system reinforcement learning framework, and obtain a trained energy storage system dynamic charge and discharge threshold management model; Energy management is performed through the energy storage system dynamic charge and discharge threshold management model.

10. The method according to claim 8, characterized in that The reinforcement learning framework for DC traction network multi-energy coupling system is established for: The state space s is determined according to Formula 5, and the state quantity is normalized according to Formula 6: s=[p,SOCM,P pv ,u rec ] Formula 5; Where p represents the train load, P pv Represents photovoltaic output power, u rec represents the rectifier unit voltage of the substation, SOC represents the state of charge of the energy storage unit, z represents the normalized value of the state quantity, x refers to all state variables, μ and σ are the mean and variance of the data set respectively; According to formula 9, the action space a is determined, and the action amount is the charge and discharge threshold of the energy storage unit. By changing the adjustment factor k ch , k dis The value is taken to realize dynamic adjustment of the threshold, where the relationship between the adjustment factor and the threshold is shown in Formula 7-Formula 8: u ch =u s0 +k ch Δu Formula 7; u dis =u s0 -k dis Δu Formula 8; a=[k ch , k dis ] Formula 9; Among them, u s0 =750V, Δu=12.5, k ch , k dis Dynamically changes according to energy management strategies in [1,5]; Determine the dimension of the input and output variables of the neural network in DQN, where the number of fully connected hidden layers of the neural network is h, and the neuron structure of each layer is u = [u1,u2...u h ], the activation function of the neuron selects the linear correction function (RELU), as shown in formula 10: f(x)=max(0,x) Formula 10; Among them, x refers to the input variable of the neural network; Determine the reward function r according to formula 11: the reward function is the weighted sum of the energy saving rate and the voltage stabilization rate within the unit time step Δt, and the weight coefficient is 0.5: r=-0.5Δv-0.5Δe Formula 11; Among them, Δv represents the voltage change, and Δe represents the power change; The deep reinforcement learning training according to the DC traction network multi-energy coupling system reinforcement learning framework specifically includes: According to the DC traction network multi-energy coupling system reinforcement learning framework, deep reinforcement learning training is performed to obtain a trained energy storage system dynamic charging and discharging threshold management model: in a round of learning, the agent extracts the system state s from the environment as the input of the action-value function composed of a deep neural network inside the agent. The agent compares the Q values ​​corresponding to all possible action quantities a, and selects the action corresponding to the maximum Q value to output to the environment, changing the state quantity of the environment. The agent obtains the corresponding reward r for this round of learning, and changes the internal parameters of the deep neural network according to the reward to carry out the next learning process. In the repeated learning and training process, the agent modifies the strategy according to the feedback reward value to maximize the cumulative reward.

Images