Rail transit collaborative energy supply system energy management method based on dynamic planning

Abstract

The invention discloses a rail transit collaborative energy supply system energy management method based on dynamic planning, and the method comprises the steps: building an energy storage system-new energy power generation system-traction power supply system collaborative multi-target energy management optimization model through considering the comprehensive energy loss and traction energy consumption of a system; offline training and online operation are carried out by defining state variables, actions, rewards, returns and strategies of a system, a dynamic planning method is adopted in the offline training, strategy evaluation, strategy improvement and strategy iteration are adopted, an optimal action strategy is gradually approached, and the obtained optimal action strategy is loaded into an online operation database and transmitted to an online operation link. Adopting a greedy strategy to realize action output so as to optimize an energy management process; the method is used in a power fusion type rail transit cooperative energy supply system to realize adaptive control of an energy storage system and a new energy power generation system, the energy utilization level of the rail transit cooperative energy supply system is improved, and the energy loss of the new energy power generation system is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of electrified railways, and in particular relates to an energy management method for a rail transit collaborative energy supply system based on dynamic programming. Background Art

[0002] As an important part of the current rail transportation system, electrified railways bear a large number of passenger and freight transportation needs. When electric locomotives are transporting, all the electricity comes from the traction power supply system. With the expansion of the scale of electrified railways and the number of electric locomotives, a large amount of electricity will be consumed for electric traction every year, and its high energy consumption characteristics are becoming increasingly prominent. In order to reduce electricity consumption, how to effectively utilize the regenerative braking energy of locomotives and the energy of new energy power generation systems along the line has become one of the important measures for the railway system to promote the "dual carbon" strategy. During the braking process of the locomotive, the regenerative braking strategy is usually preferred to generate a large amount of regenerative braking energy. However, as an energy consumer of the external power system, the traditional traction power supply system is difficult to store the regenerative braking energy of the electric locomotive due to the lack of energy storage equipment, and can only be consumed by other electric locomotives in the traction state under the same power supply arm. If the energy is returned to the power grid, it will cause energy waste due to non-billing or reverse billing methods. In addition, the new energy power generation system and the energy storage system usually adopt a DC power supply mode, and the existing traction power supply system is difficult to access these devices due to the lack of a DC interface, which limits the efficiency of energy utilization across time and space. The rail transit collaborative energy supply system adopts a power integration architecture, that is, power integration devices such as railway power conditioners are added to the power distribution offices and sub-divisions within the station, which not only realizes the cross-phase flow of energy, but also constructs a DC link to facilitate the access of energy storage equipment and new energy power generation systems. For this new power supply system structure, the energy of electric locomotives comes not only from the external power system, but also from the internal energy storage system and new energy power generation system, making the energy flow distribution far more complex than the traditional system. As the decision-making core of the system energy regulation, the energy management system is responsible for regulating the output of each system at every moment. The efficiency of its energy management directly determines the energy consumption level of the collaborative energy supply system.

[0003] At present, rule-based energy management strategies have the characteristics of simple setting and strong robustness. Usually, multiple thresholds are set according to traction power or contact network voltage to adapt to various working conditions of electric locomotives, and are widely used in the field of energy management. In the field of DC power supply for urban rail transit, there is an approximately linear relationship between the traction network voltage and the power of electric locomotives. The energy management strategy with voltage stability as the goal can achieve better control effect. In the field of AC traction power supply system for mainline electrified railways, the energy storage system can be controlled by setting the corresponding thresholds based on the real-time power of the electric locomotive. However, the driving organization mode of the electric locomotive changes with the system operation status in real time, and the setting of the threshold needs to rely on the designer's experience, which makes the system lack of adaptive ability. Therefore, the energy management strategy based on optimization has gradually become the mainstream. However, although the optimization methods represented by model predictive control and reinforcement learning strategies can achieve a certain degree of adaptive energy management, their decisions can only obtain local optimal solutions, and it is difficult to obtain global optimal solutions, which limits the technical and economic performance of optimization strategies.

[0004] According to public patent information, the current research on energy management methods for rail transit collaborative energy supply systems is still in its infancy. The patent "A control method based on AC electrified energy storage system" (publication number: CN112994094B) proposes an energy management strategy based on power thresholds, sets a variety of energy storage system operating conditions according to the load power of the electric locomotive, and then exchanges energy according to the charging and discharging rules, but the method is not flexible enough and has no adaptive ability to reduce peak power, which can easily cause excessive discharge of the energy storage system; the patent "An energy management method for electrified railway energy storage system based on fuzzy control" (publication number: CN114498692B) is based on fuzzy logic control algorithms, and achieves a certain degree of adaptive control through fuzzification, fuzzy reasoning, reverse reasoning and other operations, but the parameters represented by fuzzy membership functions and fuzzy rules need to be set manually, and the optimality of the results cannot be guaranteed. The patent "An Energy Management Method for Electrified Railway Energy Storage System Based on Reinforcement Learning" (Publication No.: CN116316755B) established a Markov decision model for the energy management process. Through the reinforcement learning algorithm, adaptive energy exchange between the energy storage system and the traction power supply system can be achieved, but there is a lack of interaction mechanism for new energy systems. Summary of the invention

[0005] In order to overcome the defects of the prior art, the present invention proposes an energy management method for a rail transit collaborative energy supply system based on a dynamic programming algorithm. By considering the multi-objective energy management mathematical optimization model of the system's comprehensive energy loss and traction energy consumption, and using a dynamic programming algorithm to solve it, the new energy power generation system and the energy storage system are controlled to perform adaptive energy exchange, so as to achieve efficient operation of the rail transit collaborative energy supply system.

[0006] The energy management method of the rail transit coordinated energy supply system based on dynamic programming, according to the established multi-objective energy management optimization model that considers the system comprehensive energy loss and traction energy consumption, and the coordination of energy storage system-new energy power generation system-traction power supply system, the energy management optimization model is converted into a dynamic programming model, and offline training and online operation are performed by defining the state variables, actions, rewards, returns and strategies of the system. The offline training link adopts a dynamic programming method, and gradually approaches the optimal action strategy through strategy evaluation, strategy improvement and strategy iteration, which specifically includes the following steps:

[0007] S1: Loading historical power data into the multi-objective energy management optimization model, including traction power and typical daily power historical data of new energy power generation system;

[0008] S2: Based on historical power data, mathematical statistics methods are used to obtain the state transition probability under different states;

[0009] S3: Initialize system parameters, initialize system strategies, and obtain initial strategies;

[0010] S4: After the parameter initialization is completed, the system starts training and calculates the current system state;

[0011] S5: Based on the current state, calculate the state value function and perform strategy evaluation;

[0012] S6: Based on the current strategy, randomly sample the action space to obtain the current action;

[0013] S7: Calculate the value of the reward function and the value of the state-action value function;

[0014] S8: Calculate the action corresponding to the optimal action-state value function in the current state;

[0015] S9: Improve the strategy and iteratively update the system parameters until the convergence conditions are met;

[0016] S10: If the convergence condition is met, the obtained optimal action strategy is loaded into the online operation database, passed to the online operation link, and the greedy strategy is used to implement the action output to optimize the energy management process;

[0017] If the convergence condition is not met, return to S4;

[0018] The energy management method is used in a power-integrated rail transit collaborative energy supply system to achieve adaptive control of the energy storage system and the new energy power generation system.

[0019] Furthermore, the multi-objective energy management optimization model includes a cooperative energy supply system objective function and cooperative energy supply system constraints, wherein the cooperative energy supply system optimization objective includes three sub-objectives: minimizing the energy loss absorbed by the energy storage system, minimizing the traction energy of the traction power supply system, and minimizing the energy loss of the photovoltaic system. The cooperative energy supply system objective function is described as formula (1):

[0020]

[0021] In formula (1), and c 1 are the energy loss absorbed by the energy storage system and its weight factor, and c 2 are the traction energy of the traction power supply system and its weight factor, and c 3 are the photovoltaic system loss energy and its weight factor respectively;

[0022] The constraints of the collaborative energy supply system include energy balance constraints, maximum power constraints, power ramp constraints and state of charge constraints.

[0023] Furthermore, the state variables, actions, rewards, returns and strategies are described by the system parameters at time k respectively, and the element s[k] of the state variable S is described as follows:

[0024]

[0025] In formula (2), P lc [k] is the power of the electric locomotive at time k, is the maximum power output of the new energy power generation system at time k, P ess [k-1] is the real-time exchange power of the energy storage system at time k-1, SOC[k] is the state of charge of the energy storage system at time k, and S is the set of state variables;

[0026] The element a[k] of the action set A is described as formula (3):

[0027]

[0028] In formula (3), P pv [k] is the total real-time exchange power of the new energy power generation system at time k, P ess [k] is the exchange power of the energy storage system at time k, P pv is the action space of the new energy power generation system, P ess It is the action space of the energy storage system;

[0029] The reward R is used to evaluate the value of taking action a[k] under the current state s[k]. According to the objective function of the collaborative energy supply system, the system reward can be described as follows:

[0030]

[0031] In formula (4), r is the reward factor;

[0032] The return G is the discounted sum of the rewards, as described in formula (5):

[0033]

[0034] In formula (5), γ is the discount factor and K is the termination time of the reward calculation.

[0035] Furthermore, the state transition probability is the probability that the system state s[k] is transferred to s[k+1] when action a[k] is taken, and this parameter is obtained based on mathematical statistics. In addition, historical data is statistically analyzed, and the locomotive power of the traction power supply system is P lc [k] Transfer to P lc The transition statistical probability p(P lc [k] → P lc [k+1]), the power generated by the new energy power generation system is P pv [k] Transfer to P pv The transition statistical probability p(P pv [k] → P pv [k+1]) and the charge state estimated at the next moment, the state probability p(a[k]|s[k]→s[k+1]) can be obtained.

[0036] Furthermore, the strategy π is the sampling probability distribution of action a[k] under state s[k], which satisfies the uniform distribution in the range of 0 to 1, and can be expressed as:

[0037] π(A|S)=U(0,1)(p(a[k]|s[k])=1 / D ess D pv )(6)

[0038] In formula (6), p(a[k]|s[k]) is the sampling probability of action a[k] under strategy π(A|S) and state s[k], D ess is the number of discrete working conditions of the maximum exchange power of the energy storage system, D pv It is the number of discrete working conditions of the maximum exchange power of the renewable energy power generation system.

[0039] The state value function V describes the expected discounted reward and obtained by taking actions according to the strategy π(A|S) starting from the current state s[k], as shown in formula (7):

[0040]

[0041] In the optimal strategy π * (A|S), the optimal action-state value function V can be obtained by using the dynamic programming algorithm * (s[k]), that is, the maximum value of the state value function, the optimal action-state value function is calculated as shown in formula (8):

[0042]

[0043] Furthermore, the strategy evaluation is used to measure the value of the state s[k] under the current strategy π(A|S), and the optimal action-state value function V * The solution problem is transformed into the recursive update equation of the state value function. Let n be the number of iterations. The recursive update equation of the state value function is as follows:

[0044]

[0045] When the policy π remains unchanged, the action-state value function will iteratively converge to V π .

[0046] Furthermore, the strategy improvement improves the original strategy π to maximize the state value function, and defines the action-state value function of taking action a[k] in state s[k] under strategy π as Q π (s[k], a[k]), as shown in formula (10):

[0047]

[0048] According to the greedy strategy, the action with the maximum action-value function value is selected to complete the strategy improvement, as shown in formula (11):

[0049]

[0050] Furthermore, the policy iteration is to repeat the policy evaluation and policy iteration according to the chain method, gradually approaching the Bellman optimal equation. The policy iteration description is as shown in formula (12):

[0051]

[0052] In formula (12), π * 、V * are the optimal strategy and optimal value function respectively, π 0 , are the initial strategy and the corresponding state value function, respectively. represents the strategy evaluation, Indicates strategy improvement.

[0053] Furthermore, the policy iteration algorithm will continue to repeat the policy evaluation-policy iteration process until the termination condition is reached, that is, the policy converges or the value function converges. Assume that the algorithm converges after m policy iterations. The convergence conditions include equations (13) and (14):

[0054] π m (a|s)=π m-1 (a|s)(a∈A,s∈S)(13)

[0055] |V m (s)-V m-1 (s)|≤ΔV (s∈S) (14)

[0056] In formula (14), ΔV represents the value function fluctuation convergence criterion.

[0057] Furthermore, based on the optimal strategy π obtained in the offline training phase * , through the greedy strategy, the action with the largest action-value function value under the current state s[k] is selected for output, that is:

[0058]

[0059] a π* is the action corresponding to the maximum action-value function value in the current state.

[0060] The purpose of the present invention can be achieved by the following technical solutions:

[0061] Compared with the prior art, the present invention has the following technical effects:

[0062] (1) It can realize the efficient operation of the rail transit coordinated energy supply system, so that the energy storage system and the new energy power generation system can operate in coordination under various working conditions, thereby improving the energy utilization efficiency;

[0063] (2) It can control the adaptive energy exchange between the renewable energy power generation system and the energy storage system, absorb the regenerative braking energy and the energy of the renewable energy power generation system, realize the compensation of traction power, and play the role of peak shaving and valley filling;

[0064] (3) It can improve the energy efficiency of the energy management system and gradually approach the optimal action strategy through continuous strategy evaluation and improvement iteration, and has good adaptability and learning ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1This is a structural diagram of the rail transit collaborative energy supply system of the present invention;

[0066] Figure 2 The flowchart is a method for energy management of a rail transit collaborative energy supply system based on dynamic programming;

[0067] Figure 3 This is the original electric locomotive power diagram;

[0068] Figure 4 It is the maximum output power curve of photovoltaic power generation system;

[0069] Figure 5 Exchange power curve diagram for energy storage system;

[0070] Figure 6 This is the power curve of the photovoltaic power generation system. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the invention, all other embodiments obtained by ordinary technicians in this field without creative work, any modifications, equivalent substitutions, improvements, etc., should be included in the protection scope of the present invention.

[0072] An energy management method for a rail transit collaborative energy supply system based on a dynamic programming algorithm, the energy management method is used in a power-smoothing rail transit collaborative energy supply system, such as Figure 1 As shown, the structure of the electrified railway energy storage system includes a traction power supply system, an external power system, a power conditioner RPC device, a new energy system, an energy storage system and an energy management system. The system structure is based on the traditional traction power supply system structure, and a railway power conditioner RPC device connected to the left and right power supply arms is added inside the traction substation system, so that energy can be exchanged through the RPC device bypassing the electrical phase separation, and the energy storage system and the new energy power generation system are connected to the internal DC bus of the RPC device after completing the power conversion through the DC / DC converter.

[0073] The energy management method of the rail transit coordinated energy supply system based on dynamic programming, according to the established multi-objective energy management optimization model that considers the system comprehensive energy loss and traction energy consumption, and the coordination of energy storage system-new energy power generation system-traction power supply system, the energy management optimization model is converted into a dynamic programming model, and offline training and online operation are performed by defining the state variables, actions, rewards, returns and strategies of the system. The offline training link adopts a dynamic programming method, and gradually approaches the optimal action strategy through strategy evaluation, strategy improvement and strategy iteration, which specifically includes the following steps:

[0074] S1: Loading historical power data into the multi-objective energy management optimization model, including traction power and typical daily power historical data of new energy power generation system;

[0075] The multi-objective energy management optimization model includes a collaborative energy supply system objective function and collaborative energy supply system constraints, wherein the collaborative energy supply system optimization objective includes three sub-objectives: minimizing the energy loss absorbed by the energy storage system, minimizing the traction energy of the traction power supply system, and minimizing the energy loss of the photovoltaic system. The collaborative energy supply system objective function is described as formula (S1):

[0076]

[0077] In formula (S1), and c 1 are the energy loss absorbed by the energy storage system and its weight factor, and c 2 are the traction energy of the traction power supply system and its weight factor, and c 3 are the photovoltaic system loss energy and its weight factor respectively;

[0078] Let the discrete data point number be k, and the total number of data points be K. is the regenerative braking power of the electric locomotive in the traction power supply system at time k, η ce For the energy storage system charging efficiency, is the charging power of the energy storage system at time k, is the maximum power output of the new energy power generation system at time k, then the energy storage system absorbs energy loss objective function Calculate as formula (S2):

[0079]

[0080] set up is the traction power of the electric locomotive in the traction power supply system at time k, is the discharge power of the energy storage system at time k, The power transferred from the new energy power generation system to the traction power supply system at time k, and the energy feedback efficiency is η de , then the objective function of traction energy minimization of the traction power supply system is Calculate as formula (S3):

[0081]

[0082] set up The objective function of minimizing the energy loss of the photovoltaic system is that the new energy generation system transfers power to the traction power supply system at time k. Calculate as formula (S4):

[0083]

[0084] The constraints of the collaborative energy supply system include energy balance constraints, maximum power constraints, power ramp constraints and state of charge constraints.

[0085] Assume that the power exchanged between the rail transit coordinated energy supply system and the external power system at time k is P gd [k], the total real-time exchange power of the energy storage system at time k is P ess [k], the total power of the electric locomotive at time k is P lc [k], the total real-time exchange power of the new energy power generation system at time k is P pv [k], the power loss is The time interval between discrete sampling points is Δt, and the energy balance constraint can be expressed as follows:

[0086] sP gd [k]Δt=(P ess [k]+P lc [k]+P pv [k])Δt (S5)

[0087]

[0088]

[0089] Assume that the maximum charging power of the energy storage system is The maximum discharge power of the energy storage system is Then the maximum power constraint can be expressed as in equations (S8) and (S9):

[0090]

[0091]

[0092] Assume that the short-term maximum power change of the energy storage system is The short-term maximum power change of the new energy power generation system is The power ramp constraint can be expressed as:

[0093]

[0094]

[0095] Assume that the upper and lower limits of the state of charge of the energy storage system are SOC max , SOC min , the state of charge of the energy storage system at time t is SOC[t], then the state of charge constraint is:

[0096] SOC min ≤SOC[t]≤SOCmax (S12)

[0097] The state variable is the basic information of decision-making, which is used to reflect the current state of the system. The state variable, action, reward, return and strategy are described by the system parameters at time k. The element s[k] of the state variable S is described as follows:

[0098]

[0099] In formula (S13), P lc [k] is the power of the electric locomotive at time k, is the maximum power output of the new energy power generation system at time k, P ess [k-1] is the real-time exchange power of the energy storage system at time k-1, SOC[k] is the state of charge of the energy storage system at time k, and S is the set of state variables;

[0100] The element a[k] of the action set A is described as formula (S14):

[0101] a[k]={P pv [k],P ess [k]}(a[k]∈A,P pv [k]∈P pv ,P ess [k]∈P ess )(S14)

[0102] In formula (S14), P pv [k] is the total real-time exchange power of the new energy power generation system at time k, P ess [k] is the exchange power of the energy storage system at time k, P pv is the action space of the new energy power generation system; it is described as formula (S15):

[0103]

[0104] P ess is the action space of the energy storage system, described as formula (S16):

[0105]

[0106] In formulas (S15) and (S16), D ess is the number of discrete working conditions of the maximum exchange power of the energy storage system, D pv is the number of discretization conditions for the exchange power of the renewable energy power generation system.

[0107] The reward R is used to evaluate the value of taking action a[k] under the current state s[k]. According to the objective function of the collaborative energy supply system, the system reward can be described as follows (S17):

[0108]

[0109] In formula (S17), r is the reward factor;

[0110] The reward G is the discounted sum of the rewards, as described in formula (S18):

[0111]

[0112] In formula (S18), γ is the discount factor and K is the termination time of the reward calculation.

[0113] S2: Based on historical power data, mathematical statistics methods are used to obtain the state transition probability under different states;

[0114] The state transition probability is the probability that the system state s[k] will transfer to s[k+1] when action a[k] is taken. This parameter is obtained based on mathematical statistics. After taking action a[k] at the current state of charge SOC[k], the state of charge at the next moment can be estimated. Let E ess [k] is the terminal voltage of the energy storage system, Q ess is the total charge of the energy storage system, then the state of charge SOC[k+1] at the next moment is calculated as follows:

[0115]

[0116] In addition, the historical data is statistically analyzed. For the traction power supply system, the locomotive power is P lc [k] Transfer to P lc The number of random events of [k+1] is n(P lc [k+1]), i is the sequence number, P lc,i In the action space S ess The action element with internal serial number i is based on the locomotive power of the traction power supply system from P lc [k] Transfer to P lc The transition statistical probability p(P lc [k] → P lc [k+1]) is calculated as formula (S20):

[0117]

[0118] For the new energy power generation system, the power generation power is P pv [k] Transfer to P pv The number of random events of [k+1] is n(P pv [k+1]), j is the sequence number, P pv,j In the action space S pv The action element with internal serial number j, the power generation of the new energy power generation system is Ppv [k] Transfer to P pv The transition statistical probability p(P pv [k] → P pv [k+1]) is calculated as formula (S21):

[0119]

[0120] Based on the above transition statistical probability and the charge state estimated at the next moment, the state probability p(a[k]|s[k]→s[k+1]) can be obtained.

[0121] S3: Initialize system parameters, initialize system strategies, and obtain initial strategies;

[0122] The strategy π is the sampling probability distribution of action a[k] under state s[k], which satisfies the uniform distribution in the range of 0 to 1, and can be expressed as:

[0123] π(A|S)=U(0,1)(p(a[k]|s[k])=1 / D ess D pv )(S22)

[0124] In formula (S22), p(a[k]|s[k]) is the sampling probability of action a[k] under strategy π(A|S) and state s[k], D pv It is the number of discrete working conditions of the maximum exchange power of the renewable energy power generation system.

[0125] S4: After the parameters are initialized, the system starts training and calculates the current system state;

[0126] S5: Based on the current state, calculate the state value function and perform strategy evaluation;

[0127] The state value function V describes the expected discounted reward and obtained by taking actions according to the strategy π(A|S) starting from the current state s[k], as shown in formula (S23):

[0128]

[0129] S6: Based on the current strategy, randomly sample the action space to obtain the current action;

[0130] S7: Calculate the value of the reward function and the value of the state-action value function;

[0131] S8: Calculate the action corresponding to the optimal action-state value function in the current state;

[0132] In the optimal strategy π *(A|S), the optimal action-state value function V can be obtained by using the dynamic programming algorithm * (s[k]), that is, the maximum value of the state value function, the optimal action-state value function is calculated as shown in formula (S24):

[0133]

[0134] S9: Improve the strategy and iteratively update the system parameters until the convergence conditions are met;

[0135] The strategy evaluation is used to measure the value of the state s[k] under the current strategy π(A|S), and the optimal action-state value function V * The solution problem is transformed into the recursive update equation of the state value function. Let n be the number of iterations. The recursive update equation of the state value function is as follows (S25):

[0136]

[0137] When the policy π remains unchanged, the action-state value function will iteratively converge to V π .

[0138] The strategy improvement improves the original strategy π to maximize the state value function. The action-state value function of taking action a[k] in state s[k] under strategy π is defined as Q π (s[k], a[k]), as shown in formula (S26):

[0139]

[0140] For two determined strategies π′ and π, the set of actions under strategy π′ is A π′ , action is a π′ , for any state s∈S, if the action-state value function Q of the strategy π′ π′ (s,a π′ ) is always greater than the state value function V of the strategy π in this state π (s), then strategy π′ is better than strategy π. If they are equal, they are equally good. π′ is called the better strategy, satisfying formula (S27):

[0141] Q π′ (s,a π′ )≥V π (s)(s∈S,a π′ ∈A π′ )(S27)

[0142] The action completion strategy is improved by selecting the action with the maximum action-value function value according to the greedy strategy, as shown in formula (S28):

[0143]

[0144] The strategy iteration is to repeat the strategy evaluation and strategy iteration according to the chain method, gradually approaching the Bellman optimal equation. The strategy iteration equation is described as formula (S29):

[0145]

[0146] In formula (S29), π * 、V * are the optimal strategy and optimal value function respectively, π 0 , are the initial strategy and the corresponding state value function, represents the strategy evaluation, Indicates strategy improvement.

[0147] The policy iteration algorithm will continuously repeat the policy evaluation-policy iteration process until the termination condition is reached, that is, the policy converges or the value function converges. Assume that the algorithm converges after m policy iterations. The convergence condition includes equations (S30) and (S31):

[0148] π m (a|s)=π m-1 (a|s)(a∈A,s∈S) (S30)

[0149] |V m (s)-V m-1 (s)|≤ΔV (s∈S) (S31)

[0150] In formula (S31), ΔV represents the value function fluctuation convergence criterion.

[0151] S10: If the convergence condition is met, the obtained optimal action strategy is loaded into the online operation database, passed to the online operation link, and the greedy strategy is used to implement the action output to optimize the energy management process;

[0152] The optimal strategy π obtained based on the offline training phase * , through the greedy strategy, the action with the largest action-value function value under the current state s[k] is selected for output, that is:

[0153]

[0154] is the action corresponding to the maximum action-value function value in the current state.

[0155] If the convergence condition is not met, return to S4.

[0156] The energy management method is used in a power-integrated rail transit collaborative energy supply system to achieve adaptive control of the energy storage system and the new energy power generation system.

[0157] against Figure 1 The structure of the rail transit coordinated energy supply system is shown in FIG. 1 , where the new energy power generation system is a photovoltaic power generation system. The typical daily power curve of the electric locomotive and the typical daily power curve of the photovoltaic power generation system are shown in FIG. Figure 3 , Figure 4 The basic parameters of typical daily power of electric locomotive are shown in Table 1:

[0158] Table 1 Basic parameters of typical daily power of electric locomotive

[0159]

[0160] The basic parameters of the photovoltaic power generation system and energy storage system are shown in Table 2:

[0161] Table 2 Basic parameters of photovoltaic power generation system and energy storage system

[0162]

[0163] Using the above data for testing, the energy storage system exchange power curve is as follows: Figure 5 As shown, it can be seen that the energy storage system will adaptively charge and discharge according to the power situation of the electric locomotive, and will absorb the energy of the photovoltaic power generation system during idle periods to achieve consumption.

[0164] Table 3 Parameter comparison table

[0165]

[0166] The power curve of photovoltaic power generation system is as follows: Figure 6 As shown, it can be seen that the photovoltaic power generation system also achieves good and accurate coordination with the traction power supply system, compensating the traction power curve as much as possible. Under this strategy, the specific comparison parameters are shown in Table 3. It can be seen that the peak power after power compensation is significantly reduced, effectively playing the role of peak shaving and valley filling.

Claims

1. An energy management method for a rail transit collaborative energy supply system based on dynamic programming, characterized in that: The energy management method is based on a multi-objective energy management optimization model that considers the system's comprehensive energy loss and traction energy consumption, and the coordination of the energy storage system, the new energy generation system, and the traction power supply system. The energy management optimization model is converted into a dynamic programming model, and offline training and online operation are performed by defining the system's state variables, actions, rewards, returns, and strategies. The offline training link adopts a dynamic programming method, and gradually approaches the optimal action strategy through strategy evaluation, strategy improvement, and strategy iteration. Specifically, the following steps are included: S1: Loading historical power data into the multi-objective energy management optimization model; S2: Based on historical power data, mathematical statistics methods are used to obtain the state transition probability under different states; S3: Initialize system parameters and system strategies; S4: Calculate the current system state; S5: Calculate the state value function and perform strategy evaluation; S6: Based on the current strategy, randomly sample the action space to obtain the current action; S7: Calculate the value of the reward function and the value of the state-action value function; S8: Calculate the action corresponding to the optimal action-state value function in the current state; S9: Improve the strategy and iteratively update the system parameters until the convergence conditions are met; S10: If the convergence condition is met, the obtained optimal action strategy is loaded into the online operation database, passed to the online operation link, and the greedy strategy is used to implement the action output to optimize the energy management process; If the convergence condition is not met, return to S4; The energy management method is used in a power-integrated rail transit collaborative energy supply system to achieve adaptive control of an energy storage system and a new energy power generation system.

2. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 1 is characterized in that: The multi-objective energy management optimization model includes a collaborative energy supply system objective function and collaborative energy supply system constraints. The collaborative energy supply system objective function is described as follows: In formula (1), and c1 are the energy loss absorbed by the energy storage system and its weight factor, and c2 are the traction energy of the traction power supply system and its weight factor, and c3 are the photovoltaic system loss energy and its weight factor respectively; The constraints of the collaborative energy supply system include energy balance constraints, maximum power constraints, power ramp constraints and state of charge constraints.

3. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 2 is characterized in that: The state variables, actions, rewards, returns and strategies are described by the system parameters at time k respectively. The element s[k] of the state variable S is described as follows: In formula (2), P lc [k] is the power of the electric locomotive at time k, is the maximum power output of the new energy power generation system at time k, P ess [k-1] is the real-time exchange power of the energy storage system at time k-1, SOC[k] is the state of charge of the energy storage system at time k, and S is the set of state variables; The element a[k] of the action set A is described as formula (3): a[k]={P pv [k],P ess [k]} (a[k]∈A,P pv [k]∈P pv ,P ess [k]∈P ess ) (3) In formula (3), P pv [k] is the total real-time exchange power of the new energy power generation system at time k, P ess [k] is the exchange power of the energy storage system at time k, P pv is the action space of the new energy power generation system, P ess It is the action space of the energy storage system; The reward R is used to evaluate the value of taking action a[k] under the current state s[k]. According to the objective function of the collaborative energy supply system, the system reward can be described as follows: In formula (4), r is the reward factor; The return G is the discounted sum of the rewards, as described in formula (5): In formula (5), γ is the discount factor and K is the termination time of the reward calculation.

4. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 3 is characterized in that: The state transition probability is the probability that the system state s[k] transfers to s[k+1] when action a[k] is taken, based on the locomotive power of the traction power supply system being P lc [k] Transfer to P lc The transition statistical probability p(P lc [k] → P lc [k+1]), the power generated by the new energy power generation system is P pv [k] Transfer to P pv The transition statistical probability p(P pv [k] → P pv [k+1]) and the charge state estimated at the next moment, the state probability p(a[k]|s[k]→s[k+1]) can be obtained.

5. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 4 is characterized in that: The strategy π is the sampling probability distribution of action a[k] under state s[k]: π(A|S)=U(0,1) (p(a[k]|s[k])=1 / D ess D pv ) (6) In formula (6), D ess is the number of discrete working conditions of the maximum exchange power of the energy storage system, D pv The number of discrete working conditions for the maximum exchange power of the renewable energy power generation system; The state value function V describes the expected discounted reward and obtained by taking actions according to the strategy π(A|S) starting from the current state s[k], as shown in formula (7): The optimal action-state value function is calculated as shown in formula (8):

6. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 5 is characterized in that: The strategy evaluation is used to measure the value of the state s[k] under the current strategy π(A|S), and the optimal action-state value function V * The solution problem is transformed into the recursive update equation of the state value function, as shown in formula (9): When the policy π remains unchanged, the action-state value function will iteratively converge to V π .

7. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 6 is characterized in that: The strategy improvement improves the original strategy π to maximize the state value function. The action-state value function of taking action a[k] in state s[k] under strategy π is defined as Q π (s[k], a[k]), as shown in formula (10): According to the greedy strategy, the action with the maximum action-value function value is selected to complete the strategy improvement, as shown in formula (11):

8. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 7 is characterized in that: The strategy iteration is to repeat the strategy evaluation and strategy iteration according to the chain method, gradually approaching the Bellman optimal equation. The strategy iteration is described as formula (12): In formula (12), π * 、V * are the optimal strategy and optimal value function, π0, are the initial strategy and the corresponding state value function, represents the strategy evaluation, Indicates strategy improvement.

9. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 8 is characterized in that: The convergence conditions include equations (13) and (14): p m (a|s)=π m-1 (a|s) (a∈A,s∈S) (13) |Vm(s)-Vm-1(s)|≤ΔV (s∈S) (14)In formula (14), ΔV represents the value function fluctuation convergence criterion.

10. The energy management method for the rail transit coordinated energy supply system based on dynamic programming according to claim 9 is characterized in that: The optimal strategy π obtained based on the offline training phase * , through the greedy strategy, the action with the largest action-value function value under the current state s[k] is selected for output, that is: is the action corresponding to the maximum action-value function value in the current state.

Images