Adaptive sensing heavy-load train group operation energy management method and control device

Through the adaptive sensing energy management method, the TD3-BC algorithm and the bidirectional DC/DC converter are used to solve the problem of large traction load power and strong volatility in heavy-load trains, and the efficient power distribution of lithium batteries and supercapacitors is realized. Protect the battery and improve system performance and reliability.

CN118859730BActive Publication Date: 2024-11-26CENT SOUTH UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411342067.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-25
Publication Date
2024-11-26
Estimated Expiration
2044-09-25

AI Technical Summary

Technical Problem

Heavy-load trains have large traction load power and strong volatility. It is difficult for the existing technology to effectively manage hybrid energy storage systems, resulting in overcharge and discharge of lithium batteries and low supercapacitance utilization, affecting system performance and battery life.

Method used

Adaptive sensing of heavy-load train group operation energy management method, the low-pass filter cutoff frequency is adjusted in real time through TD3-BC offline learning algorithm, and the load requirements are decomposed into high-frequency and low-frequency components, which are used in supercapacitors and lithium batteries respectively. Optimize the duty cycle of the bidirectional DC/DC converter, combining the state of charge of lithium batteries and supercapacitors for power distribution.

Benefits of technology

Effectively avoid overcharging and discharging of lithium batteries, improve the utilization rate of supercapacitors, adapt to load changes under complex operating conditions, optimize energy distribution, reduce energy waste, protect battery, and improve system reliability and energy utilization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118859730B_ABST
    Figure CN118859730B_ABST
Patent Text Reader

Abstract

The present invention discloses an adaptive sensing heavy-load train group operation energy management method and control device, the method is: according to the bus voltage and bus reference voltage at the current moment, generate the total reference current of the load demand; according to the voltage of the lithium battery and super capacitor at the current moment, respectively calculate the charge state of the lithium battery and the super capacitor; according to the charge state of the lithium battery and the super capacitor and the total reference current, obtain the cutoff frequency of the low-pass filter through the intelligent agent based on the TD3-BC offline learning algorithm; use the low-pass filter with the cutoff frequency to filter the total reference current, and obtain the high-frequency component and low-frequency component of the total reference current as the reference current of the super capacitor and the lithium battery respectively; according to the reference current of the lithium battery and the super capacitor, respectively generate the duty cycle, and drive and control the corresponding bidirectional DC / DC converter based on the duty cycle. The present invention can protect the life of the lithium battery and improve the dynamic response capability of the heavy-load train.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle power control, and in particular relates to an adaptive sensing heavy-load train group operation energy management method and control device. Background Art

[0002] By the end of 2023, the total operating mileage of electrified railways in my country has exceeded 159,000 kilometers. While electrified railways are developing rapidly, heavy-haul trains have been widely used due to their efficient transportation capacity and outstanding cost-effectiveness. However, with the increase in energy consumption and pressure on energy conservation and emission reduction, heavy-haul trains face major challenges. In particular, heavy-haul trains generate a large amount of regenerative braking energy during operation, but this energy is currently not effectively recycled in most cases.

[0003] In order to deal with this problem, measures have been proposed to introduce energy storage devices in heavy-duty trains. By installing energy storage devices on the train, the regenerative energy generated during braking can be effectively captured and stored. These energy storage devices convert the energy generated during braking into electrical energy and store it for subsequent use. This approach can not only significantly reduce the energy consumption of the train and reduce operating costs, but also reduce dependence on external power grids and enhance the train's autonomous operation capabilities and flexibility. In addition, by effectively recovering regenerative energy, it can also achieve energy reuse and help achieve the "dual carbon" goal.

[0004] However, in the "grid-source-storage-vehicle" coordinated power supply system, the traction load of heavy-duty trains is large and volatile, and it is difficult to match the source and load in time and space. In order to ensure reliable power supply for trains and the safe operation of the system as a whole, there is a great demand for real-time control of the heavy-duty train hybrid energy storage management system; in addition, on the one hand, lithium batteries have the advantages of high energy density and low self-discharge, and on the other hand, supercapacitors have the advantages of high power density, wide operating temperature range, fast charging and discharging, and long service life. Therefore, using supercapacitors to supply or store peak power or rapidly changing power in the heavy-duty train hybrid energy storage system can not only improve the quality of energy supply, but also help reduce the current pressure of the battery and extend the battery life.

[0005] The energy distribution strategies in the hybrid energy storage system of heavy-duty trains mainly include rule-based control methods, optimization-based methods, and frequency-division-based control methods. The rule-based control method formulates control rules according to the required power of the hybrid energy source, the state of charge of the lithium battery and the supercapacitor. This method is simple and easy to implement, but it is not applicable to the complex working conditions of heavy-duty trains with large traction load power and strong volatility. In contrast, the optimization-based method can optimize energy distribution through complex algorithms, reduce battery aging and system energy consumption, but it usually requires a lot of computing time and resources and is difficult to apply in real time. The frequency-division-based control method generally uses a filter to distribute the load power to high-frequency components and low-frequency components. This method can achieve real-time energy distribution, but the fixed low-pass filter cutoff frequency does not consider the state of charge of the hybrid energy source, and does not consider the frequency distribution of the load power demand under different road conditions. After working for a long time, the supercapacitor is prone to overcharge or over-discharge, which makes the utilization rate of the supercapacitor low and does not play a role in protecting the battery, reducing the performance of the overall system.

[0006] There are some existing patents, such as the Chinese patent publication number CN116632886A, the publication date is August 22, 2023, and the invention name is "A real-time energy optimization management method for heavy-duty electrified railway stations". The invention aims to minimize the real-time operating cost of the system and minimize the real-time three-phase voltage imbalance on the public grid side of the traction substation, establish a multi-objective optimization function; solve the simplified global optimization model of the collaborative power supply system to complete energy optimization management. However, no specific power allocation method is proposed. The Chinese patent publication number is CN116853005A, the publication date is October 10, 2023, and the invention name is "Energy management method for urban rail transit energy storage device based on train running status". The invention train running status urban rail transit energy storage device energy management method can increase the charge and discharge threshold when the train is traction, so that the energy storage system can be discharged in time, effectively avoiding the problem of the energy storage system being unable to discharge in time and the energy storage system being unevenly utilized and poor energy saving effect, but a single energy storage device is not suitable for the complex operating conditions of heavy-duty trains. The Chinese patent publication number is CN112116156A, and the publication date is December 22, 2020. The name of the invention is "Energy management method and system for hybrid trains based on deep reinforcement learning". The invention obtains an optimized energy management strategy model through offline training through deep reinforcement learning; the optimized energy management strategy model is input with real-time data of the hybrid train to obtain an optimized energy management strategy. However, the energy management strategy based on learning and deviating from the mathematical model may have an impact on the stable operation of heavy-load trains. Summary of the invention

[0007] In order to overcome the technical problems of large and volatile traction load power of heavy-load trains, the present invention provides an adaptive sensing heavy-load train group operation energy management method and control device, which can not only protect the life of lithium batteries but also improve the dynamic response capability of heavy-load trains.

[0008] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0009] An adaptive sensing heavy-load train group operation energy management method, comprising:

[0010] According to the bus voltage at the current moment and the given bus reference voltage, a voltage closed-loop circuit is used to generate a total reference current required by the load;

[0011] According to the voltage and current of the lithium battery and the supercapacitor at the current moment, the charge state of the lithium battery and the supercapacitor are calculated respectively;

[0012] According to the charge state of the lithium battery and supercapacitor and the total reference current required by the load, the cutoff frequency of the low-pass filter is obtained through an intelligent agent based on the TD3-BC offline learning algorithm;

[0013] The total reference current is filtered using a low-pass filter with an adjusted cutoff frequency to obtain a high-frequency component and a low-frequency component of the total reference current, and the high-frequency component is used as the reference current of the supercapacitor, and the low-frequency component is used as the reference current of the lithium battery;

[0014] According to the reference current of the lithium battery and the supercapacitor, a current closed-loop circuit is used to generate the duty cycle of each bidirectional DC / DC converter;

[0015] The corresponding bidirectional DC / DC converter is driven and controlled based on the obtained duty cycle.

[0016] Furthermore, lithium batteries are Determine its state of charge, and the method for calculating its state of charge is:

[0017] ;

[0018] in, Indicates that lithium batteries are The state of charge at the moment, is the nominal charge of the lithium battery, For lithium batteries The amount of charge at a given moment, Indicates that lithium batteries are The state of charge at the moment, For lithium batteries The current at the moment, Indicates the duration of adjacent moments.

[0019] Furthermore, supercapacitors are Determine its state of charge, or update the supercapacitor's state of charge using the following calculation:

[0020] ;

[0021] in, Indicates that the supercapacitor The state of charge at the moment, Indicates that the supercapacitor The terminal voltage at the moment, represents the nominal voltage of the supercapacitor, Indicates that the supercapacitor The state of charge at the moment, Indicates the duration of adjacent moments. For supercapacitors The current at the moment, is the maximum voltage of the supercapacitor, is the capacitance of the supercapacitor.

[0022] Furthermore, the training method of the intelligent agent based on the TD3-BC offline learning algorithm is:

[0023] (1) Experience is extracted from historical operation data and stored in the experience replay pool. Each experience includes: current state s, actual action a, reward r for executing action a, and next state s'; where: the state is composed of the charge state of the lithium battery and supercapacitor and the total reference current required by the load. Each type of state data is normalized; the action refers to the cutoff frequency of the low-pass filter;

[0024] (2) Initialize the agent based on the TD3-BC offline learning algorithm, including the policy network , two Q networks and , Strategy Target Network , two Q target networks and ;in:

[0025] Policy Network It is used to generate the cutoff frequency of the low-pass filter in a given state, and θ is a parameter of the policy network;

[0026] Two Q networks and , used to estimate the value of the current state and action, and are the parameters of the two Q networks;

[0027] Policy Target Network For smooth generation of target motion, are the parameters of the policy target network;

[0028] Two Q target networks and , used to smoothly generate the target Q value, and are the parameters of the two Q target networks;

[0029] (3) Randomly sample a batch of experience data from the experience replay pool and train the intelligent agent based on the TD3-BC offline learning algorithm.

[0030] Furthermore, using empirical data to train an agent based on the TD3-BC offline learning algorithm specifically includes:

[0031] (1) Using the policy target network Generate the target action corresponding to the next state s' , and calculate the target Q value:

[0032] ;

[0033] ;

[0034] in, is the noise, c is the clipping range of the noise; To limit noise The magnitude is y is the target Q value; r is the immediate reward for the current action a in state s; γ is a discount factor used to weigh the importance of future rewards and current rewards; and : Two Q target networks and Estimates of the next state s′ and target action a′;

[0035] (2) By minimizing the loss function To update the Q network:

[0036] ;

[0037] Where i∈{1,2}, corresponding to two Q networks respectively; N: the number of samples drawn from the experience replay pool each time; Represents the i-th Q network for state s and actual action a Q value;

[0038] (3) Update the strategy network:

[0039] On the one hand, calculating the TD3 strategy loss :

[0040] ;

[0041] in, :Q Network Output: Take action in state s Q value at ; : Policy Network Actions generated based on the current state s;

[0042] On the other hand, the behavior cloning loss is calculated :

[0043] ;

[0044] Among them, a is the actual action corresponding to each experience and state s in the experience replay pool;

[0045] Then, the TD3 strategy loss and behavior cloning loss are weighted together to obtain the hybrid loss function of the strategy network: :

[0046] ;

[0047] Among them, α is a hyperparameter used to adjust the weights of behavior cloning loss and TD3 loss;

[0048] (4) Target network soft update:

[0049] ;

[0050] ;

[0051] ;

[0052] in, is the soft update coefficient.

[0053] Furthermore, the cutoff frequency of the low-pass filter is in the range of [1,5 Hz].

[0054] An adaptive sensing heavy-load train group operation energy management device, characterized in that it includes a voltage and current acquisition module, a control module, a drive module and two bidirectional DC / DC converters; the two bidirectional DC / DC converters are respectively arranged at the output ends of a lithium battery and a super capacitor, and the parallel output provides power for a bus load;

[0055] The voltage and current acquisition module is provided with a voltage sensor and a current sensor, the voltage sensor is used to acquire the bus voltage, super capacitor voltage, and lithium battery voltage, and the current sensor is used to acquire the lithium battery current and super capacitor current;

[0056] The control module is used to: generate the duty cycle of each bidirectional DC / DC converter based on the data collected by the collection circuit module and using any of the above-mentioned adaptive sensing heavy-load train group operation energy management methods;

[0057] The driving module is used to drive and control a corresponding bidirectional DC / DC converter based on a duty cycle.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The adaptive sensing heavy-load train group operation energy management method of the present invention calculates the current state of charge of the lithium battery and supercapacitor through the voltage and current of the lithium battery and supercapacitor collected in real time. , and according to the lithium battery and super capacitor The total reference current of the load demand is adjusted through the TD3-BC offline learning algorithm to adjust the low-pass filter cutoff frequency, and obtain high-frequency components and low-frequency components respectively. This not only effectively avoids the over-charging and discharging of lithium batteries, but also enables supercapacitors to bear larger peak currents or high-frequency power, greatly improving the utilization rate of supercapacitors and overcoming the problem of large power and strong volatility of heavy-load train traction loads.

[0060] 2. The adaptively perceived energy management method for heavy-load train group operation of the present invention, facing the load power P and the SOC of lithium batteries and supercapacitors under different working conditions, the TD3-BC offline learning algorithm adjusts the cutoff frequency of the low-pass filter and is adaptive to different road conditions. Since reinforcement learning continuously updates the strategy and value function, and introduces behavioral imitation, when facing a complex dynamic environment, TD3-BC can quickly adjust the control strategy through past experience and current status to adapt to real-time changing energy needs. Secondly, the algorithm continuously optimizes the energy management strategy through reinforcement learning, making energy distribution more accurate and efficient, reducing energy waste and improving energy utilization, ensuring that the system can quickly restore to a stable state when the load changes dramatically, and significantly improving the reliability of the system. In addition, TD3-BC introduces a dual Q network to solve the problem of overestimation of the Q value, and due to the introduction of behavioral simulation, the speed of algorithm training has been greatly improved.

[0061] 3. The adaptive sensing heavy-load train group operation energy management method of the present invention combines the charge state of lithium batteries and supercapacitors Power allocation is performed with deep reinforcement learning, taking into account possible acceleration and emergency braking conditions in future road conditions, and maintaining the charge state of the supercapacitor over the entire road time scale. Maintained at the middle level, the supercapacitor Both are maintained in the range of [35%, 75%], which greatly improves the utilization rate of supercapacitors and makes the use scenarios of supercapacitors more targeted at the high-frequency components in the load power, so that the frequency of the low-frequency components borne by the lithium battery is lower and the current fluctuation of the lithium battery is also smaller, which effectively protects the battery and reduces the energy consumption of the entire hybrid energy storage system.

[0062] 4. The adaptive sensing heavy-load train group operation energy management method of the present invention has a simple control method and an ideal control effect, and can effectively overcome the problem of large power and strong volatility of the traction load of heavy-load trains. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] FIG1 is a main circuit topology diagram of the train hybrid energy storage control device of the present invention; wherein, is the current of the lithium battery, is the current of the supercapacitor, It is the current provided by the supercapacitor after conversion by the DC / DC converter. is the total reference current required by the load;

[0064] FIG2 is a circuit diagram of a voltage and current acquisition module in the train hybrid energy storage control device of the present invention;

[0065] FIG3 is a principle block diagram of the train hybrid energy storage management method of the present invention. DETAILED DESCRIPTION

[0066] The following is a detailed description of an embodiment of the present invention. This embodiment is based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process to further explain the technical solution of the present invention.

[0067] Example 1

[0068] The embodiment of the present invention provides an adaptive sensing heavy-load train group operation energy management method, which is applied to the hybrid energy storage of lithium batteries and supercapacitors. This embodiment takes the hybrid energy storage on a long and large-scale heavy-load train as an example to explain the technical solution of the present invention. Figure 1 The main circuit topology diagram of the train hybrid energy storage control device is shown, which includes a lithium battery, a super capacitor, two DC / DC converters, a filter capacitor C and a bus load, and a voltage and current acquisition module, a control module and a drive module are not drawn. The two bidirectional DC / DC converters are respectively set at the output end of the lithium battery and the super capacitor, and after being connected in parallel, the output is output through the filter capacitor C to provide power for the bus load. Figure 2The voltage and current acquisition module circuit shown is provided with a voltage sensor and a current sensor. The voltage sensor is used to collect bus voltage, supercapacitor voltage, and lithium battery voltage, and the current sensor is used to collect lithium battery current and supercapacitor current. The voltage and current data obtained by the voltage and current acquisition module are transmitted to the control module, and the control module generates a duty cycle signal for the two bidirectional DC / DC converters based on the received voltage and current data. Then, the driving module generates two complementary driving signals with dead time based on the duty cycle signal, which act on the power tube of the bidirectional DC / DC converter.

[0069] The adaptive sensing heavy-load train group operation energy management method is referenced Figure 3 As shown, the following steps are included:

[0070] Step 1: According to the bus voltage at the current moment and a given bus reference voltage, a voltage closed-loop circuit is used to generate a total reference current required by the load.

[0071] Step 2: Calculate the charge state of the lithium battery and the super capacitor respectively according to the voltage of the lithium battery and the super capacitor at the current moment.

[0072] Lithium battery according to Determine its state of charge, and the method for calculating its state of charge is:

[0073] ;

[0074] in, Indicates that lithium batteries are The state of charge at the moment, is the nominal charge of the lithium battery, For lithium batteries The amount of charge at a given moment, Indicates that lithium batteries are The state of charge at the moment, For lithium batteries The current at the moment, Indicates the duration of adjacent moments.

[0075] Supercapacitors based on Determine its state of charge, or update the supercapacitor's state of charge using the following calculation:

[0076] ;

[0077] in, Indicates that the supercapacitor The state of charge at the moment, Indicates that the supercapacitor The terminal voltage at the moment, represents the nominal voltage of the supercapacitor, Indicates that the supercapacitor The state of charge at the moment, Indicates the duration of adjacent moments. For supercapacitors The current at the moment, is the maximum voltage of the supercapacitor, is the capacitance of the supercapacitor.

[0078] When the SoC of the supercapacitor is higher, the system will tend to let the supercapacitor bear more high-frequency load requirements because the supercapacitor can provide fast energy response;

[0079] If the SoC of the supercapacitor is low, the agent may reduce the high-frequency components allocated to the supercapacitor, and more load may be allocated to the lithium battery for processing.

[0080] Step 3: According to the charge state of the lithium battery and the supercapacitor and the total reference current required by the load, the cutoff frequency of the low-pass filter is obtained through an intelligent agent based on the TD3-BC offline learning algorithm.

[0081] The training method of the intelligent agent based on the TD3-BC offline learning algorithm is:

[0082] (1) Experience is extracted from historical operation data and stored in the experience replay pool. Each experience includes: current state s, actual action a, reward r for executing action a, and next state s'; where: the state is composed of the charge state of the lithium battery and supercapacitor and the total reference current required by the load. Each type of state data is normalized; the action refers to the cutoff frequency of the low-pass filter;

[0083] (2) Initialize the agent based on the TD3-BC offline learning algorithm, including the policy network , two Q networks and , Strategy Target Network , two Q target networks and ;in:

[0084] Policy Network It is used to generate the cutoff frequency of the low-pass filter in a given state, and θ is a parameter of the policy network;

[0085] Two Q networks and , used to estimate the value of the current state and action, and are the parameters of the two Q networks;

[0086] Policy Target Network Smoothly generate target actions, It is the parameter of the strategy target network, which helps the system smoothly adjust the cutoff frequency of the low-pass filter under different load requirements;

[0087] Two Q target networks and , used to smoothly generate the target Q value, ensuring that the target value does not fluctuate drastically when updating the Q network, and are the parameters of the two Q target networks;

[0088] (3) Randomly sample a batch of experience data from the experience replay pool and train the agent based on the TD3-BC offline learning algorithm through steps (4) to (7).

[0089] (4) Update the Critic network (Q network):

[0090] Using policy target network Generate the next state s ’ Corresponding target action :

[0091] ;

[0092] in is a small noise, and c is the clipping range of the noise. Clip is used to limit the amplitude of the noise to By introducing noise to smooth the target action, overfitting can be prevented.

[0093] Calculate the target Q value:

[0094] ;

[0095] y: target Q value.

[0096] r: The immediate reward obtained by the current action a in state s.

[0097] γ: Discount factor, used to weigh the importance of future rewards and current rewards. Usually the value is between [0,1]. A larger γ value means more emphasis on future rewards.

[0098] and : The two Q target networks’ estimates of the next state s′ and target action a′.

[0099] Update the Critic network: Update the Q network by minimizing the loss function:

[0100] ;

[0101] i∈{1,2}, corresponding to two Q networks respectively. Represents the Q value of the i-th Q network for state s and actual action a.

[0102] N: This refers to the number of samples drawn from the experience replay pool each time.

[0103] In the step of updating the Q network, TD3-BC adopts a dual Q network to avoid overestimation by selecting the minimum of the two Q values, which is a key improvement of TD3.

[0104] In addition, TD3-BC adds noise to the action and smoothes the target Q value, making the policy update process smoother.

[0105] (5) Update of the policy network (Actor):

[0106] On the one hand, calculate the TD3 strategy loss:

[0107] ;

[0108] By minimizing the negative Q-value, the policy network is updated to maximize the Q-value, allowing the Actor network to learn how to choose the best action in a given state.

[0109] :Q Network Output: Take action in state s The Q value at that time.

[0110] : That is, the action a generated by the policy network, based on the current state s and the parameters 𝜃 of the policy network.

[0111] On the other hand, the behavioral cloning (BC) loss is calculated:

[0112] Compare the actions generated by the policy network with the actual actions in the offline data and minimize the difference between the two through the behavior cloning loss:

[0113] ;

[0114] : The action output by the policy network, which is generated according to the current state s.

[0115] It is the actual action corresponding to state s in each experience in the experience replay pool.

[0116] Mixed loss function:

[0117] The final policy update loss is a weighted combination of the TD3 policy loss and the behavior cloning loss:

[0118] ;

[0119] Among them, α is a hyperparameter used to adjust the weights of behavior cloning loss and TD3 loss.

[0120] In the step of updating the policy network, TD3-BC introduces behavior cloning (BC) to make its actions closer to the actual actions in the offline data, generate more conservative policies, and avoid the policy network from being overly optimistic or exploring unknown areas.

[0121] At the same time, due to the introduction of BC loss, TD3-BC does not require complex exploration mechanisms and online interactions, but only needs to extract valuable strategies from offline datasets;

[0122] (6) Target network soft update:

[0123] ;

[0124] ;

[0125] ;

[0126] θ′: Parameters of the policy target network.

[0127] : Parameters of the policy network.

[0128] : Soft update coefficient, usually between [0,1]. Smaller The value makes the update of the target network smoother and reduces the instability caused by frequent changes;

[0129] (7) Training is completed.

[0130] Step 4, filtering the total reference current using a low-pass filter with an adjusted cutoff frequency to obtain a high-frequency component and a low-frequency component of the total reference current, and using the high-frequency component as the reference current of the supercapacitor and the low-frequency component as the reference current of the lithium battery;

[0131] Step 5, according to the reference current of the lithium battery and the super capacitor, respectively use the current closed loop circuit to generate the duty cycle of the respective bidirectional DC / DC converters;

[0132] Step 6: driving and controlling the power tube of the corresponding bidirectional DC / DC converter based on the obtained duty cycle.

[0133] The present invention uses the SOC of lithium batteries and supercapacitors, and the total reference current of load demand as inputs of the TD3-BC offline learning algorithm, adjusts the cutoff frequency of the low-pass filter in real time, and decomposes the total reference current of the load demand into low-frequency components and high-frequency components through the low-pass filter. The components are used as reference currents of the current closed-loop circuits of the lithium battery and supercapacitor respectively. On the premise of meeting the load demand power in real time, the supercapacitor SOC is maintained to fluctuate at around 50%, which not only improves the utilization rate of the supercapacitor, but also has sufficient capacity preparation for high-frequency positive and negative power demands that may appear in the future, and effectively protects the lithium battery.

[0134] Example 2

[0135] This embodiment provides an adaptive sensing heavy-load train group operation energy management control device, including a voltage and current acquisition module, a control module, a drive module and two bidirectional DC / DC converters; the two bidirectional DC / DC converters are respectively arranged at the output ends of the lithium battery and the super capacitor, and the parallel output provides power for the bus load;

[0136] The voltage and current acquisition module is provided with a voltage sensor and a current sensor, the voltage sensor is used to acquire the bus voltage, super capacitor voltage, and lithium battery voltage, and the current sensor is used to acquire the lithium battery current and super capacitor current;

[0137] The control module is used to: generate the duty cycle of each bidirectional DC / DC converter based on the data collected by the collection circuit module and using the adaptive sensing heavy-load train group operation energy management method described in Example 1;

[0138] The driving module is used to drive and control a corresponding bidirectional DC / DC converter based on a duty cycle.

[0139] The above embodiments are preferred embodiments of the present application. Ordinary technicians in this field can also make various changes or improvements on this basis. Without departing from the overall concept of the present application, these changes or improvements should fall within the scope of protection required by the present application.

Claims

1. An adaptive sensing method for heavy-load train group operation energy management, It is characterized in that include: According to the bus voltage at the current moment and the given bus reference voltage, a voltage closed-loop circuit is used to generate a total reference current required by the load; According to the voltage and current of the lithium battery and the supercapacitor at the current moment, the charge state of the lithium battery and the supercapacitor are calculated respectively; According to the charge state of the lithium battery and supercapacitor and the total reference current required by the load, the cutoff frequency of the low-pass filter is obtained through an intelligent agent based on the TD3-BC offline learning algorithm; The training method of the intelligent agent based on the TD3-BC offline learning algorithm is as follows: (1) Experience is extracted from historical operation data and stored in the experience replay pool. Each experience includes: current state s, actual action a, reward r for executing action a, and next state s'; where: the state is composed of the charge state of the lithium battery and supercapacitor and the total reference current required by the load. Each type of state data is normalized; the action refers to the cutoff frequency of the low-pass filter; (2) Initialize the agent based on the TD3-BC offline learning algorithm, including the policy network , two Q networks and , Strategy Target Network , two Q target networks and ;in, Policy Network It is used to generate the cutoff frequency of the low-pass filter in a given state, and θ is a parameter of the policy network; Two Q networks and , used to estimate the value of the current state and action, and are the parameters of the two Q networks; Policy Target Network For smooth generation of target motion, are the parameters of the policy target network; Two Q target networks and , used to smoothly generate the target Q value, and are the parameters of the two Q target networks; (3) Randomly sample a batch of experience data from the experience replay pool and train the agent based on the TD3-BC offline learning algorithm; The total reference current is filtered using a low-pass filter with an adjusted cutoff frequency to obtain a high-frequency component and a low-frequency component of the total reference current, and the high-frequency component is used as the reference current of the supercapacitor, and the low-frequency component is used as the reference current of the lithium battery; According to the reference current of the lithium battery and the supercapacitor, a current closed-loop circuit is used to generate the duty cycle of each bidirectional DC / DC converter; The corresponding bidirectional DC / DC converter is driven and controlled based on the obtained duty cycle.

2. The method for managing the operation energy of heavy-load train groups based on adaptive perception according to claim 1, It is characterized in that Lithium battery according to Determine its state of charge, and its state of charge update calculation method is: ; in, Indicates that lithium batteries are The state of charge at the moment, is the nominal charge of the lithium battery, For lithium batteries The amount of charge at a given moment, Indicates that lithium batteries are The state of charge at the moment, For lithium batteries The current at the moment, Indicates the duration of adjacent moments.

3. The adaptive sensing heavy-load train group operation energy management method according to claim 1, It is characterized in that Supercapacitors based on Determine its state of charge and update the supercapacitor's state of charge using the following calculation: ; in, Indicates that the supercapacitor The state of charge at the moment, Indicates that the supercapacitor The terminal voltage at the moment, represents the nominal voltage of the supercapacitor, Indicates that the supercapacitor The state of charge at the moment, Indicates the duration of adjacent moments. For supercapacitors The current at the moment, is the maximum voltage of the supercapacitor, is the capacitance of the supercapacitor.

4. The method for managing the operation energy of heavy-load train groups based on adaptive perception according to claim 1, It is characterized in that Using empirical data to train an agent based on the TD3-BC offline learning algorithm specifically includes: (1) Use the policy target network Generate the target action corresponding to the next state s' , and calculate the target Q value: ; ; in, is the noise, c is the clipping range of the noise; To limit noise The magnitude is y is the target Q value; r is the immediate reward for the current action a in state s; γ is a discount factor used to weigh the importance of future rewards and current rewards; and : Two Q target networks and Estimates of the next state s′ and target action a′; (2) By minimizing the loss function To update the Q network: ; Where i∈{1,2}, corresponding to two Q networks respectively; N: the number of samples drawn from the experience replay pool each time; Represents the i-th Q network for state s and actual action a Q value; (3) Update the strategy network: On the one hand, calculating the TD3 strategy loss : ; in, :Q Network Output: Take action in state s Q value at ; : Policy Network Actions generated based on the current state s; On the other hand, the behavior cloning loss is calculated : ; in, is the actual action corresponding to each experience and state s in the experience replay pool; Then, the TD3 strategy loss and behavior cloning loss are weighted together to obtain the hybrid loss function of the strategy network: : ; Among them, α is a hyperparameter used to adjust the weights of behavior cloning loss and TD3 loss; (4) Target network soft update: ; ; ; in, is the soft update coefficient.

5. The adaptive sensing heavy-load train group operation energy management method according to claim 1, It is characterized in that The cutoff frequency of the low-pass filter is in the range of [1,5Hz].

6. An adaptive sensing heavy-load train group operation energy management device, It is characterized in that It includes voltage and current acquisition module, control module, drive module and 2 bidirectional DC / DC converters; The two bidirectional DC / DC converters are respectively arranged at the output ends of the lithium battery and the super capacitor, and are connected in parallel to output power for the bus load; The voltage and current acquisition module is provided with a voltage sensor and a current sensor, the voltage sensor is used to acquire the bus voltage, super capacitor voltage, and lithium battery voltage, and the current sensor is used to acquire the lithium battery current and super capacitor current; The control module is used to: generate the duty cycle of each bidirectional DC / DC converter based on the data collected by the collection circuit module and using the adaptive sensing heavy-load train group operation energy management method according to any one of claims 1 to 5; The driving module is used to drive and control a corresponding bidirectional DC / DC converter based on a duty cycle.