Vehicle ammonia-hydrogen hybrid power device and energy regulation and management method

Abstract

The present invention proposes a vehicle ammonia-hydrogen hybrid power device and an energy regulation and management method, which are divided into two parts: a vehicle ammonia-hydrogen hybrid power device and an energy regulation and management method. The hybrid power device consists of a liquid ammonia tank, a hydrogen tank, an ammonia-hydrogen engine, an ammonia electrolyzer, a fuel cell, a generator, a motor, a lithium battery, a transmission, etc. On this basis, an energy regulation and management method is proposed. According to the required power of the motor and the state of charge of the battery, the vehicle driving modes are divided into: braking, pure electric operation, hybrid charging, and hybrid discharging four modes. In the hybrid charging and discharging modes, energy management optimization objectives for improving the efficiency of the ammonia-hydrogen hybrid system and maintaining the state of charge of the battery are proposed. During the training process of the double-delay deep deterministic policy gradient algorithm, the trajectory of the state of charge change of the lithium battery generated by fuzzy logic control is added. This method can be used in the ammonia-hydrogen hybrid power device and the energy regulation and management method for formulating the management framework, selecting optimization objectives and methods.

Description

Technical Field

[0001] The present invention belongs to the technical field of new energy, and particularly relates to a vehicle ammonia-hydrogen hybrid power system and an energy regulation and management method. Background Art

[0002] Hydrogen power is considered promising as a new future vehicle power, especially suitable for heavy trucks. The safety and economy of on-vehicle hydrogen storage have always been difficult problems that hydrogen-powered vehicles must face. Converting hydrogen into an "energy carrier" with high energy density and easy to transport and store is one of the important ways to solve this problem. Currently, ammonia is considered one of the most promising hydrogen energy carriers, and it can also be used as a zero-carbon fuel itself. Using hydrogen as an "oxidizer" for ammonia combustion can achieve efficient ammonia-hydrogen combustion. Achieving efficient ammonia-hydrogen combustion through on-line ammonia hydrogen production is a feasible route for the application of ammonia internal combustion engines in heavy trucks. Electro-catalytic ammonia decomposition for hydrogen production can be carried out around room temperature, with high hydrogen purity and easy control of the hydrogen production process, which is more suitable for on-vehicle hydrogen production. The hydrogen produced by ammonia decomposition can be directly used in fuel cells in addition to being an "oxidizer" for ammonia engines, jointly constituting a zero-carbon, heavy-duty vehicle ammonia-hydrogen hybrid power system with engines, electric motors, and power batteries.

[0003] Different energy conversion devices have different energy efficiency characteristics. To maximize the advantages of different power devices and ensure the efficient and economical operation of vehicles, it is very important to design an advanced power energy management strategy. The present invention proposes three power sources: an ammonia-hydrogen engine, a fuel cell system, and a lithium-ion battery, combined with on-vehicle hydrogen production, which makes power energy management quite complex. Based on this, the present invention proposes an energy management technology and method for such a new type of hybrid power device. Summary of the Invention

[0004] The present invention proposes a vehicle ammonia-hydrogen hybrid power device and an energy regulation and management method. Based on the vehicle ammonia-hydrogen hybrid power device, by adopting the energy regulation and management method, the energy of ammonia-hydrogen hybrid vehicles can be optimized, and at the same time, the defects in the energy distribution of complex ammonia-hydrogen hybrid power systems can be made up for.

[0005] The vehicle ammonia-hydrogen hybrid power device includes: a liquid ammonia tank, a hydrogen tank, an ammonia-hydrogen engine, an ammonia electrolyzer, a fuel cell, a generator, a motor, a lithium battery, a transmission, and a main reducer. The device composition scheme is as follows: The liquid ammonia tank and the hydrogen tank are respectively connected to the ammonia-hydrogen engine, the ammonia electrolyzer, and the fuel cell. The fuel cell is respectively connected to the motor and the lithium battery. The motor serves as the power source directly driving the vehicle. The motor transmits torque to the wheels through the transmission and the main reducer. The ammonia-hydrogen engine converts mechanical energy into electrical energy through the generator and supplies it to the motor. The fuel cell can charge the lithium battery. Ammonia in the liquid ammonia tank is used for combustion in the ammonia-hydrogen engine on the one hand and for hydrogen production in the ammonia electrolyzer on the other hand. When the hydrogen produced by the ammonia electrolyzer is insufficient to meet the demand of the ammonia-hydrogen engine, the insufficient hydrogen is supplemented by the hydrogen tank. When the hydrogen produced by the electrolyzer exceeds the demand of the ammonia-hydrogen engine, the excess hydrogen is stored in the hydrogen tank. The lithium battery provides the electrical energy required for the hydrogen production process of the electrolyzer and can also supply power directly to the motor. The lithium battery can recover electrical energy when the vehicle brakes.

[0006] The energy regulation and management method of this power device: According to the required power of the motor and the state of charge of the lithium battery, the driving modes of the vehicle are divided into: braking recovery mode, pure electric operation mode, hybrid charging mode, and hybrid discharging mode. In the hybrid charging mode and the hybrid discharging mode, an energy regulation and management strategy applying the double-delay deep deterministic policy gradient algorithm and fuzzy logic control is used, and the characteristics of these four modes are respectively defined.

[0007] The characteristics and beneficial effects of the present invention are as follows: The proposed vehicle ammonia-hydrogen hybrid power device provides a new way to solve problems such as high cost of on-vehicle hydrogen storage and poor ammonia combustion characteristics, and provides a new solution for the decarbonization application of ammonia-hydrogen integration technology in the field of heavy transportation. The proposed energy regulation and management method provides a new research method for the energy management of the current complex multi-energy hybrid system. By combining fuzzy logic control and deep reinforcement learning, on the one hand, it solves the problem that the energy management strategy based on fuzzy logic control overly relies on technical experience and lacks optimality; on the other hand, it makes up for the deficiencies of the energy management strategy based on deep reinforcement learning, such as long training time and difficult convergence. Description of the Drawings

[0008] Figure 1 It is the structure diagram of the ammonia-hydrogen hybrid power system in the embodiment. In the figure: The long dashed arrows represent fuel transmission; the short dashed arrows represent electrical transmission; the solid arrows represent torque transmission.

[0009] Figure 2 It is the flow chart of the vehicle operation mode in the embodiment.

[0010] Figure 3 It is the energy management framework based on fuzzy logic and deep reinforcement learning in the embodiment.

[0011] Figure 4 This is the learning curve graph of different methods in the embodiments of the present invention.

[0012] Figure 5 This is the test performance of different EMSs in the embodiments of the present invention Detailed implementation manners

[0013] The following further describes the structural composition of the device of the present invention and the method for energy regulation and management in conjunction with the accompanying drawings and through specific embodiments. It should be noted that this embodiment is narrative rather than restrictive, and does not limit the protection scope of the present invention.

[0014] The following embodiments provide a vehicle ammonia-hydrogen hybrid power device and an energy regulation and management method to achieve energy distribution for ammonia-hydrogen hybrid vehicles. The structure of the ammonia-hydrogen hybrid system is as Figure 1 shown, and on the basis of the structure shown in Figure 1 the division of the operating modes of ammonia-hydrogen hybrid vehicles will be described in combination with Figure 2 the following.

[0015] As shown in Figure 1 the technical structure of the vehicle ammonia-hydrogen hybrid power device is: the liquid ammonia tank 1 and the hydrogen tank 2 are respectively connected to the ammonia-hydrogen engine 3, the ammonia electrolyzer 5, and the fuel cell 4. The fuel cell is respectively connected to the motor 7 and the lithium battery 8. The motor serves as the power source for directly driving the vehicle. The motor transmits torque to the wheels through the transmission 9 and the main reducer 10. The ammonia-hydrogen engine converts mechanical energy into electrical energy through the generator 6 and supplies it to the motor. The fuel cell can charge the lithium battery. Ammonia in the liquid ammonia tank is used for combustion in the ammonia-hydrogen engine on the one hand and for hydrogen production in the ammonia electrolyzer on the other hand. When the hydrogen generated by the ammonia electrolyzer is insufficient to meet the demands of the ammonia-hydrogen engine and the fuel cell, the insufficient hydrogen is supplemented by the hydrogen tank; when the hydrogen generated by the electrolyzer exceeds the demands of the ammonia-hydrogen engine and the fuel cell, the excess hydrogen is stored in the hydrogen tank. The lithium battery provides the electrical energy required for the hydrogen production process of the electrolyzer and can also directly supply power to the motor. The lithium battery recovers electrical energy during vehicle braking.

[0016] The energy regulation and management method of the vehicle ammonia-hydrogen hybrid power device is specifically as follows: According to the demand power of the motor and the state of charge of the lithium battery, the driving modes of the vehicle are divided into: braking energy recovery mode, pure electric operation mode, hybrid charging mode, and hybrid discharging mode. In the hybrid charging mode and the hybrid discharging mode, an energy regulation and management strategy applying the double-delay deep deterministic policy gradient algorithm and fuzzy logic control is used.

[0017] The characteristics of the following four modes will be described respectively. In the text, the "state of charge" is represented by SOC, and the same hereinafter.

[0018] The operating modes are divided according to the vehicle driving demand power and the lithium battery SOC. When the motor demand power is less than 0, the vehicle is in the braking energy recovery mode. When the motor demand power is greater than 0 and less than the lower limit of the efficient operation of the fuel cell and the ammonia-hydrogen engine, the vehicle is in the pure electric operation mode. When the lithium battery SOC < 20%, the vehicle turns on the hybrid charging mode. When the lithium battery SOC = 30%, the hybrid charging mode is turned off and the vehicle is in the hybrid discharging mode.

[0019] In the braking energy recovery mode, the recovery of electrical energy is limited by the maximum braking torque of the motor and the maximum charging current of the lithium battery to recover the amount of electricity. The recovery stops when the vehicle speed is too low (< 5 m / s) or the lithium battery is fully charged (SOC > 95%).

[0020] In the pure electric operation mode, only the lithium battery supplies power to the motor.

[0021] In the hybrid charging mode, the ammonia-hydrogen engine and the fuel cell meet the motor's demand, and the fuel cell also charges the lithium battery. In this mode, the hydrogen tank is the only source of hydrogen. When the battery SOC < 20%, the ammonia-hydrogen engine and the fuel cell system allocate the motor's demand, and at the same time the fuel cell charges the lithium battery. When the battery SOC increases by 10%, the hybrid discharging mode is turned on again.

[0022] In the hybrid discharging mode, the ammonia-hydrogen engine and the fuel cell meet the motor's demand, and the lithium battery only supplies power to the hydrogen production process of the electrolyzer. In this mode, the electrolyzer is the main source of hydrogen, and the hydrogen tank plays an auxiliary role. When the motor demand power is greater than the lower limit of the high-efficiency operation of the fuel cell and the ammonia-hydrogen engine, and the battery SOC > 20%, the ammonia-hydrogen engine and the fuel cell system allocate the motor's demand.

[0023] During the training process of the double-delayed deep deterministic policy gradient algorithm, the change trajectory of the lithium battery state of charge generated by fuzzy logic control is added.

[0024] In the hybrid charging and hybrid discharging modes, the power of the ammonia-hydrogen engine and the fuel cell is allocated according to the motor demand power and the battery SOC. Specifically:

[0025] Develop an energy regulation and management strategy based on fuzzy logic control, with the motor demand power and the battery SOC as inputs and the allocation coefficient as the output. The allocation coefficient ranges from 0 to 1 and is used to adjust the power output of the ammonia-hydrogen engine and the fuel cell system in different states. The SOC trajectory generated by the energy regulation and management strategy based on fuzzy logic control is used to construct the reinforcement learning reward function.

[0026] The energy regulation optimization objectives in the present invention include the improvement of the operating efficiency of the energy system and the maintenance of the battery SOC. The ammonia-hydrogen engine and the fuel cell in the vehicle ammonia-hydrogen hybrid device have different efficiency characteristics, and the device is coupled with ammonia electrolysis for hydrogen production. Under dynamic driving loads, it is an important optimization objective to adjust the outputs of the ammonia-hydrogen engine and the fuel cell to ensure the highest overall efficiency in the hybrid mode. Secondly, ammonia electrolysis for hydrogen production is the main hydrogen source in the energy system. Whether it is the ammonia-hydrogen engine or the fuel cell, hydrogen is required as fuel. Therefore, maintaining the lithium battery power within a reasonable range is an important condition for ensuring the normal operation of the device.

[0027] The energy management strategy based on fuzzy logic control relies on the experience of engineers, and the control effect lacks optimality and adaptability. Deep reinforcement learning can achieve the optimal control strategy through continuous interaction with the environment. However, in the initial stage of exploring the environment, it is carried out in a trial-and-error manner, which means that the agent needs to be repeatedly trained to learn the above "common sense" cognition for engineers, and may even learn control strategies that violate the wishes of engineers. Therefore, the present invention combines fuzzy logic control and deep reinforcement learning in the optimization problem of energy management in the ammonia-hydrogen hybrid system, as follows:

[0028] (1) Design an energy management strategy based on fuzzy logic control

[0029] Take the motor demand power as input variable 1, and the fuzzy sets are VL representing very low, L representing low, M representing medium, and H representing high. Take the battery SOC as input variable 2, and the fuzzy sets are L, M, H. Take the distribution coefficient DF as the output variable, and the fuzzy sets are VL, L, M, H.

[0030] To ensure that the power commands of the ammonia-hydrogen engine and the fuel cell do not exceed the peak power, if the peak power of the ammonia-hydrogen engine is less than that of the fuel cell, the distribution coefficient will act on the power system through equations (1-1) and (1-2):

[0031]

[0032]

[0033] In the formula: P AHG_cmd , P FCS_cmd are the power commands of the ammonia-hydrogen engine and the fuel cell system respectively; are the peak powers of the ammonia-hydrogen engine and the fuel cell system respectively; P req is the motor demand power; P threshold is the hybrid mode activation threshold; sig is the charging signal; is the charging power of the fuel cell for the lithium battery, which is calculated by equation (1-3).

[0034]

[0035] Where: P rem is the value obtained by subtracting the power of the fuel cell driving the motor from the peak power of the fuel cell; is the rated power of the fuel cell.

[0036] The fuzzy inference rules are as follows:

[0037] If the motor demand power P req is VL, then the distribution coefficient DF is VL;

[0038] If the motor demand power P req is L and the battery SOC is L, then the distribution coefficient DF is M;

[0039] If the motor demand power P req is L and the battery SOC is M, then the distribution coefficient DF is M;

[0040] If the motor demand power P req is L and the battery SOC is H, then the distribution coefficient DF is L

[0041] If the motor demand power P req is M and the battery SOC is L, then the distribution coefficient DF is H;

[0042] If the motor demand power P req is M and the battery SOC is M, then the distribution coefficient DF is M;

[0043] If the motor demand power P req is M and the battery SOC is H, then the distribution coefficient DF is M;

[0044] If the motor demand power P req is H and the battery SOC is L, then the distribution coefficient DF is H;

[0045] If the motor demand power P req is H and the battery SOC is M, then the distribution coefficient DF is H;

[0046] If the motor demand power P req is H and the battery SOC is H, then the distribution coefficient DF is M.

[0047] (2) Online Deep Reinforcement Learning Energy Management Strategy Combining Fuzzy Logic Control

[0048] The Double Delayed Deep Deterministic Policy Gradient algorithm is a deep reinforcement learning algorithm based on the actor-critic framework, used to handle control problems in continuous state and action spaces. By adopting the delayed updates of two critic networks, the actor network, and the target network, the risk of overestimating the action value is reduced. At each moment, the agent needs to receive three states from the environment, as shown in Equation 2-1:

[0049]

[0050] To unify the different magnitudes of the states, the motor demand power is normalized, and P max is the motor peak power. The agent outputs an action to the environment according to the state, and this action is the distribution coefficient DF.

[0051] The environment updates the state according to the action and feeds back the instantaneous reward to the agent. The reward functions are shown in Equations (2-2) and (2-3):

[0052] sig = 1

[0053]

[0054] sig = 0

[0055]

[0056] In the formula: w1, w2, w3, w4 represent the weight factors, used to balance the two energy management objectives of improving efficiency and maintaining the SOC under different states; SOC FLC is the battery SOC trajectory generated by the energy management strategy based on fuzzy logic control, and η chr represents the system efficiency of the vehicle in the hybrid charging mode, and η dis represents the system efficiency in the hybrid discharging mode. As shown in Equations 2-4 and 2-5:

[0057]

[0058]

[0059] Combined with Figure 3Describe the energy management strategy for ammonia-hydrogen hybrid vehicles. The online energy management strategy of the ammonia-hydrogen hybrid system contains two environmental representative power systems, corresponding to the energy management unit based on the Twin Delayed Deep Deterministic Policy Gradient algorithm (TD3) and the energy management unit based on Fuzzy Logic Control (FLC), respectively. The energy regulation management method is suitable for two application scenarios. One is that TD3 corresponds to the environment from an actual vehicle, and FLC corresponds to the controlled object from a simulation model; the other is that the controlled objects corresponding to both are the system simulation model. In these two scenarios, the proposed method can achieve online training of the deep reinforcement learning agent.

[0060] At each moment, the reward obtained by the agent includes both a penalty term for the deviation of the reference SOC and a reward term for the improvement of system efficiency. Using the SOC trajectory generated by the energy regulation management strategy based on fuzzy logic control to construct the reinforcement learning reward function can reduce the policy learning time of the agent and unreasonable exploration actions. Compared with using a fixed reference SOC, this method can reduce the variance of the agent's policy update, stabilize the learning process of the agent, and improve the energy management performance. Compared with using the globally optimal SOC trajectory, this method does not require a large amount of offline calculation and advance knowledge of driving information, and can achieve online learning.

[0061] According to the above method, a calculation example is provided. Select the driving condition (CHTC-HT) of a truck (GVW > 5500 kg) as the training condition, and randomly set the initial SOC to one of 0.2, 0.4, 0.6, and 0.8 before each training starts. Compare three methods, namely the EMS based on TD3, the EMS based on fuzzy logic control, and the EMS based on TD3-FLC. For the EMS based on TD3, the reference SOC in its reward function is piecewise constant, as shown in Equation (2-5). For the EMS based on TD3-FLC, the reference SOC in its reward function is the SOC trajectory generated by FLC.

[0062]

[0063] Figure 4Shows the rewards obtained by TD3-based EMS and TD3-FLC-based EMS in 50 rounds of training. The episode reward is the cumulative reward obtained in each round of training, and the average reward refers to the average episode reward in the current cumulative number of episodes. As can be seen from the figure, the training method using FLC as the reference SOC converges when the number of training rounds reaches about 10 rounds, while the method using the piecewise constant SOC trajectory does not converge after 50 rounds of training. Due to the randomness of the initial SOC before each training, the cumulative rewards of different training sets will fluctuate. This fluctuation is particularly obvious in the method using a fixed reference SOC. This is because the rewards of this method vary greatly in different states, resulting in a larger update variance of the action-value function and ultimately learning a low-quality policy.

[0064] Figure 5 Shows the SOC trajectories of three energy management strategies (EMS) under test conditions. The test condition is the global transient vehicle cycle (C-WTVC), with a total of 10 cycles. As can be seen from the three curves in the figure, the FLC-based EMS can easily maintain the SOC at a relatively high level according to human experience, but it cannot guarantee efficient operation; while the TD3-based EMS focuses on improving efficiency but ignores the SOC maintenance that cannot be ignored during long-distance transportation. The TD3-FLC-based EMS proposed in the present invention uses a large amount of lithium batteries in the first half of the vehicle operation, that is, when the remaining power is relatively high, and slows down the speed of SOC consumption as the SOC decreases, taking into account both the optimization goals of efficiency improvement and SOC maintenance.

Claims

1. An ammonia-hydrogen hybrid power device for vehicles, comprising an ammonia tank, a hydrogen tank, an ammonia-hydrogen engine, an ammonia electrolyzer, a fuel cell, a generator, a motor, a lithium battery, a transmission, and a main reducer, characterized in that: The liquid ammonia tank (1) and the hydrogen tank (2) are respectively connected to the ammonia-hydrogen engine (3), the ammonia electrolyzer (5), and the fuel cell (4). The fuel cell is respectively connected to the electric motor (7) and the lithium battery (8). The electric motor serves as the power source for directly driving the vehicle. The electric motor transmits torque to the wheels through the transmission (9) and the main reducer (10). The ammonia-hydrogen engine converts mechanical energy into electrical energy through the generator (6) and supplies it to the electric motor. The fuel cell can charge the lithium battery. Ammonia in the liquid ammonia tank is used for combustion in the ammonia-hydrogen engine on the one hand; on the other hand, it is used for hydrogen production in the ammonia electrolyzer. When the hydrogen produced by the ammonia electrolyzer is not enough to meet the needs of the ammonia-hydrogen engine and the fuel cell, the insufficient hydrogen is supplemented by the hydrogen tank; when the hydrogen produced by the electrolyzer exceeds the needs of the ammonia-hydrogen engine and the fuel cell, the excess hydrogen is stored in the hydrogen tank. The lithium battery provides the electrical energy required for the hydrogen production process of the electrolyzer and can also directly supply power to the electric motor. The lithium battery recovers electrical energy when the vehicle brakes. According to the demand power of the electric motor and the state of charge of the lithium battery, the driving modes of the vehicle are divided into: braking recovery mode, pure electric operation mode, hybrid charging mode, and hybrid discharging mode. In the hybrid charging mode and the hybrid discharging mode, an energy regulation and management strategy applying the double-delay deep deterministic policy gradient algorithm and fuzzy logic control is adopted.

2. The energy regulation and management method of the vehicle ammonia-hydrogen hybrid power device according to claim 1, characterized in that: The operating mode is divided according to the demand power of the vehicle drive and the state of charge of the lithium battery. When the demand power of the electric motor is less than 0, the vehicle is in the braking recovery mode; when the demand power of the electric motor is greater than 0 and less than the lower limit of the efficient operation of the fuel cell and the ammonia-hydrogen engine, the vehicle is in the pure electric operation mode; when the state of charge of the lithium battery < 20%, the vehicle starts the hybrid charging mode. When the state of charge of the lithium battery = 30%, the hybrid charging mode is closed, and the vehicle is in the hybrid discharging mode.

3. The energy regulation and management method of the vehicle ammonia-hydrogen hybrid power device according to claim 1, characterized in that: In the braking recovery mode, the recovery of electrical energy is limited by the maximum braking torque of the electric motor and the maximum charging current of the lithium battery to recover the amount of electricity.

4. The energy regulation and management method of the vehicle ammonia-hydrogen hybrid power device according to claim 1, characterized in that: In the pure electric operation mode, only the lithium battery supplies power to the electric motor.

5. The energy regulation and management method of the vehicle ammonia-hydrogen hybrid power device according to claim 1, characterized in that: In the hybrid charging mode, the ammonia-hydrogen engine and the fuel cell meet the demand of the electric motor, and the fuel cell also charges the lithium battery at the same time. In this mode, the hydrogen tank is the only source of hydrogen.

6. The energy regulation and management method of the vehicle ammonia-hydrogen hybrid power device according to claim 1, characterized in that: In the hybrid discharging mode, the ammonia-hydrogen engine and the fuel cell meet the demand of the electric motor, and the lithium battery only supplies power for the hydrogen production process of the electrolyzer. In this mode, the electrolyzer is the main source of hydrogen, and the hydrogen tank plays an auxiliary role.

7. The energy regulation and management method of the vehicle ammonia-hydrogen hybrid power device according to claim 1, characterized in that: During the training process of the double-delay deep deterministic policy gradient algorithm, the trajectory of the change in the state of charge of the lithium battery generated by the fuzzy logic control is added.

Images