Multi-element hydrogen storage form hydrogen energy unmanned aerial vehicle energy management optimization method and system

Abstract

The invention discloses an energy management optimization method and system for a hydrogen energy unmanned aerial vehicle in multiple hydrogen storage forms. The method comprises the following steps: constructing three hydrogen storage forms, a physical property model of a fuel cell and a lithium battery and a charge state model of the lithium battery; constructing a target function by taking the minimum hydrogen charging cost and the fuel cell life loss cost as targets, and constructing a safe operation constraint and an energy balance constraint; constructing a sequential decision model comprising a state space, an action space and a reward function; predicting future required power of the unmanned aerial vehicle through the constructed power prediction model; and based on the predicted unmanned aerial vehicle demand power and the selected action, a reward function value under the selected action is calculated, the sequential decision model is optimized, and an action selection strategy with the maximum enhanced reward value is solved. Through unified modeling and intelligent optimization scheduling in a multi-element hydrogen storage form, efficient utilization of energy of the unmanned aerial vehicle is realized, the operation cost is reduced, the service life of a fuel cell is prolonged, and autonomous intelligent energy management is realized.

Description

Technical Field

[0001] This invention relates to the field of energy management technology for hybrid unmanned aerial vehicles (UAVs) with multiple hydrogen storage systems and load fluctuations, specifically to an optimization method and system for energy management of hydrogen-powered UAVs with multiple hydrogen storage systems. Background Technology

[0002] Currently, consumer and industrial drones primarily rely on lithium-ion batteries as their power source, which generally suffer from low energy density and short flight time, severely restricting their application in scenarios such as long-distance inspection and large-scale logistics delivery. Against the backdrop of the ongoing "dual-carbon" strategy, hydrogen energy, as a clean and efficient energy carrier, shows broad prospects in the field of aviation propulsion. Hydrogen fuel cell drones, possessing both renewable energy characteristics and zero-emission advantages, are expected to become one of the important pathways to promote the green transformation of aviation operations.

[0003] However, relying solely on a single form of hydrogen storage faces several limitations: while high-pressure gaseous hydrogen storage is a mature technology, its hydrogen storage density is limited. To achieve longer endurance, the storage pressure often needs to be further increased (e.g., 70 MPa), which not only places more stringent demands on tank materials and processes but also significantly increases the overall weight and system cost. Cryogenic liquid hydrogen storage, although possessing the highest energy density, relies on complex insulation systems and suffers from continuous "boil-off gas" problems, especially for drones operating intermittently, where fuel loss cannot be avoided even in standby mode. Chemical hydride / metal hydrogen storage technologies typically involve reactors, catalysts, or heating devices during hydrogen release, resulting in slow system startup and difficulty adapting to high-frequency transient operating conditions.

[0004] At present, most related research focuses on optimizing the performance of a specific hydrogen storage method or achieving power complementarity through a simple hybrid architecture (such as fuel cells paired with lithium batteries). There is a lack of effective integration of two or more hydrogen storage methods, and no intelligent system that can autonomously allocate energy according to real-time flight missions has been seen.

[0005] The core technical problem this invention aims to solve is that existing energy management methods for hydrogen-powered drones are mostly designed around a single hydrogen storage form or a simple hybrid architecture, failing to fully consider the distinct dynamic response characteristics and energy transfer efficiency differences among various hydrogen storage devices in multi-mode hydrogen storage systems (such as high-pressure gaseous, cryogenic liquid, and metal hydrides). Especially when facing severe load fluctuations caused by gusts and maneuvers during flight, the system struggles to achieve efficient synergy and dynamic complementarity among different hydrogen storage media, resulting in low overall energy utilization and accelerated fuel cell lifespan degradation. Traditional static optimization strategies, when dealing with this type of multi-hydrogen storage device synergy problem, cannot effectively resolve the energy coupling and state interference between different hydrogen storage media, nor can they synergistically optimize system energy efficiency and fuel cell durability. Summary of the Invention

[0006] To address the shortcomings of existing technologies that fail to adequately consider the diverse dynamic response characteristics and energy transfer efficiencies of various hydrogen storage devices in multi-element hydrogen storage systems (such as high-pressure gaseous, cryogenic liquid, and metal hydride systems), this invention provides an energy management optimization method and system for hydrogen-powered drones with multi-element hydrogen storage. The method includes: constructing physical characteristic models for three hydrogen storage forms, fuel cells, and lithium batteries, as well as a state-of-charge model for lithium batteries; constructing an objective function with the goal of minimizing hydrogen refueling costs and fuel cell lifespan loss costs, and establishing safety operation constraints and energy balance constraints; constructing a sequential decision-making model including a state space, action space, and reward function; predicting future drone power demand using a constructed power prediction model; calculating the reward function value under the selected action based on the predicted drone power demand and the chosen action; optimizing the sequential decision-making model; and solving for the action selection strategy that maximizes the reward value. This invention achieves efficient energy utilization, reduced operating costs, extended fuel cell lifespan, and autonomous intelligent energy management for drones through unified modeling and intelligent optimization scheduling of multi-element hydrogen storage.

[0007] The present invention adopts the following technical solution: The first aspect of the present invention provides a method for optimizing energy management of hydrogen-powered unmanned aerial vehicles (UAVs) with multiple hydrogen storage forms, comprising: S1. Construct physical characteristic models for high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, metal hydride hydrogen storage, fuel cells, and lithium batteries, as well as a state of charge model for lithium batteries. S2. To minimize the hydrogen charging cost of high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, and metal hydride hydrogen storage, as well as the fuel cell life loss cost, an objective function is constructed. Based on the constructed physical characteristic model and state of charge model, safe operation constraints and energy balance constraints are constructed as constraints on the objective function. S3. Based on the power demand of the drone, the charging and discharging power and state of charge of the lithium battery, and the hydrogen storage quality and supply quality of the three hydrogen storage forms, a sequential decision model is constructed. The sequential decision model includes a state space, an action space, and a reward function. The reward function is obtained by weighting the calculated objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage quality penalty. S4. Predict the future power demand of the UAV by constructing a power prediction model; calculate the reward function value under the selected action based on the predicted power demand of the UAV and the selected action; optimize the sequential decision model through TD3 and dynamic dual-zone experience playback mechanism to solve the action selection strategy that maximizes the reward value.

[0008] Preferably, the process of constructing the objective function in S2 is as follows: The hydrogen charging costs for high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, and metal hydride hydrogen storage are calculated separately, and the total hydrogen charging cost is obtained by weighted summation. The fuel cell life loss under all load conditions is calculated and multiplied by the corresponding cost to obtain the fuel cell life loss cost. The total hydrogen charging cost and the fuel cell life loss cost are added together as the objective function.

[0009] Preferably, the process of calculating the hydrogen charging cost for different hydrogen storage methods in S2 is as follows: For any form of hydrogen storage, the electricity cost is multiplied by the energy consumption coefficient of the corresponding hydrogen storage form to obtain the electricity cost per unit of hydrogen supply for that form. The depreciation cost of hydrogen charging for different hydrogen storage forms is divided by the lifespan of the corresponding hydrogen storage material and then multiplied by the dispatch cycle to obtain the depreciation cost of the corresponding hydrogen storage form. The hydrogen price, the electricity cost of the corresponding hydrogen storage form, and the depreciation cost are added together and then multiplied by the hydrogen supply mass of the corresponding hydrogen storage form to obtain the hydrogen charging cost of the corresponding hydrogen storage form.

[0010] Preferably, the process for calculating the fuel cell lifetime loss under all load conditions in S2 is as follows: ; In the formula, This is due to the loss of fuel cell lifespan. The power output of the fuel cell; , as well as The fuel cell life loss is calculated under heavy load conditions, low load conditions, and fuel cell operating conditions, respectively. This refers to the rated power output of the fuel cell; This represents the maximum power change rate. This marks the starting point of the scheduling time. This is the scheduling time interval.

[0011] Preferably, the process of constructing safe operation constraints and energy balance constraints in S2 is as follows: Based on the physical characteristic models of three different hydrogen storage forms, the hydrogen storage mass of each form is obtained; based on the state of charge model of lithium batteries, the corresponding state of charge is obtained; upper and lower limits are set for the hydrogen storage mass, the state of charge at different times, the power generation of the fuel cell, and the power change, respectively, to obtain safe operation constraints. Based on the physical characteristic model of the fuel cell, the charging and discharging power of the lithium battery is obtained; for any given time, the power generation of the fuel cell plus the charging and discharging power of the lithium battery is always equal to the power demand of the UAV at the same time, thus obtaining the energy balance constraint.

[0012] Preferably, in S3, a state space is established based on the current time, the power demand of the drone at the current time, the state of charge of the lithium battery, and the hydrogen storage mass of the three hydrogen storage forms; Based on the power generation of fuel cells, the charging and discharging power of lithium batteries, and the hydrogen supply quality of the three hydrogen storage methods, an action space is established. Based on the state space at different times, the portion of the sum of the power generation of the fuel cell and the charging and discharging power of the lithium battery that exceeds the power demand of the drone is regarded as the excess output power, and the portion of the sum of the power generation of the fuel cell and the charging and discharging power of the lithium battery that cannot meet the power demand of the drone is regarded as the insufficient output power. The hydrogen storage mass penalty is obtained by summing the hydrogen storage mass exceeding the upper and lower limits of the three different energy storage forms. The reward function is obtained by weighting the objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage mass penalty at the current time.

[0013] Preferably, the process of optimizing the sequential decision model in S4 is as follows: Initialize the actor network and its target network, and simultaneously establish two critic networks and their corresponding target networks. During policy exploration, for the current state, the actor network outputs an action and injects exploration noise to form the final execution action. After applying this action to the UAV simulation environment, the corresponding next state and reward function value can be obtained. Based on the predicted power demand of the UAV, the corresponding future reward is calculated, averaged, and added to the reward function value to obtain the enhanced reward value under the action. Based on the enhanced reward value under the action, the TD error is calculated and stored in the corresponding high TD error experience zone or low TD error experience zone according to the error value. When the data accumulation in the experience replay pool reaches a preset threshold, the network parameter update process is started: each time, 80% of the samples are extracted from the low TD error zone and the remaining 20% ​​of the samples are extracted from the high TD error zone to form a training batch. By calculating the noisy clipped actions, the minimum value output by the dual-target critic network is selected as the action value, and then the critic network is updated. The actor network adopts a delayed update mechanism and is optimized by maximizing the policy gradient of the expected action value.

[0014] A second aspect of the present invention provides an energy management optimization system for hydrogen-powered unmanned aerial vehicles (UAVs) with multi-element hydrogen storage, and uses a method for energy management optimization of hydrogen-powered UAVs with multi-element hydrogen storage, comprising: The physical model building module constructs physical characteristic models for high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, metal hydride hydrogen storage, fuel cells, and lithium batteries, as well as a state of charge model for lithium batteries. The optimization objective establishment module aims to minimize the hydrogen charging cost of high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, and metal hydride hydrogen storage, as well as the fuel cell life loss cost, and constructs an objective function. Based on the constructed physical characteristic model and state of charge model, safe operation constraints and energy balance constraints are constructed as constraints on the objective function. The decision model construction module constructs a sequential decision model based on the power demand of the drone, the charging and discharging power and state of charge of the lithium battery, and the hydrogen storage quality and supply quality of the three hydrogen storage forms. The sequential decision model includes a state space, an action space, and a reward function. The reward function is obtained by weighting the calculated objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage quality penalty. The energy management optimization module predicts the future power demand of drones through a constructed power prediction model; based on the predicted drone power demand and the selected action, it calculates the reward function value under the selected action; and through TD3 and dynamic dual-zone experience playback mechanism, it optimizes the sequential decision model to solve the action selection strategy that maximizes the reward value.

[0015] A third aspect of the present invention provides a terminal, including a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of an energy management optimization method for a hydrogen-powered unmanned aerial vehicle with a multi-element hydrogen storage system.

[0016] A fourth aspect of the invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an energy management optimization method for a hydrogen-powered unmanned aerial vehicle with multiple hydrogen storage forms.

[0017] Compared with the prior art, the beneficial effects of the present invention include at least the following: 1. This invention establishes physical characteristic models for high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, metal hydride hydrogen storage, fuel cells, and lithium batteries, as well as a state-of-charge model for lithium batteries, enabling unified modeling and scheduling of various hydrogen storage forms. By combining safety operation constraints and energy balance constraints, the invention allows UAVs to rationally allocate hydrogen and electrical energy under different load conditions, improving overall energy utilization and extending flight endurance.

[0018] 2. In energy management optimization, this invention incorporates the hydrogen refueling costs of high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, and metal hydride hydrogen storage, along with fuel cell lifespan loss costs, into the objective function, and optimizes the scheduling strategy through weighted summation. By allocating the hydrogen supply for various hydrogen storage forms and the charging and discharging power of lithium batteries, it can effectively reduce hydrogen refueling costs and fuel cell lifespan losses, thereby improving the economics of drones.

[0019] 3. This invention optimizes the sequential decision-making model based on the TD3 reinforcement learning algorithm and a dynamic dual-zone experience replay mechanism. Combined with future power prediction, it can dynamically generate the optimal action strategy. The reward function comprehensively considers the objective function, output power deviation, lithium battery charging and discharging power, and hydrogen storage quality penalty, enabling the UAV to achieve autonomous and intelligent energy management in complex flight environments, thereby improving system reliability and adaptability. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of a multi-element hydrogen storage form of unmanned aerial vehicle hybrid power system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the energy management optimization process for hydrogen-powered drones with multiple hydrogen storage methods, as described in an embodiment of the present invention. Figure 3 This is a schematic diagram of the multi-step timing prediction mechanism and the TD error-driven dynamic dual-zone experience playback mechanism in an embodiment of the present invention. Figure 4 This is a training curve diagram of the time series prediction algorithm according to an embodiment of the present invention; Figure 5 The training reward curves of the TD3 algorithm and the improved TD-EDDZER-MP-TD3 algorithm in this embodiment of the invention are shown. Figure 6 The diagram shows the output results of each device in this embodiment of the invention. Figure 7 This is a diagram showing the state of energy changes of a lithium battery according to an embodiment of the present invention. Figure 8 The diagram shows the hydrogen release results for various hydrogen storage methods according to embodiments of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0022] Example 1 Embodiment 1 of the present invention provides an energy management optimization method for hydrogen-powered unmanned aerial vehicles with multiple hydrogen storage forms, see reference. Figure 1 This includes the following steps: S1. Construct physical characteristic models for high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, metal hydride hydrogen storage, fuel cells, and lithium batteries, as well as a state of charge model for lithium batteries.

[0023] See Figure 2The hydrogen-powered drone with multiple hydrogen storage forms includes: a hydrogen supply system and a hybrid power system; the hydrogen supply system includes: high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, metal hydride hydrogen storage, heater, evaporator, hydrogen release valve and pressure reducing valve; the hybrid power system includes: fuel cell, lithium battery, EMS controller, bidirectional DC / DC, unidirectional DC / DC and propulsion device; the key parameters of each component are detailed in Table 1.

[0024] Table 1. Basic parameters of multi-element hydrogen storage hydrogen-powered hybrid drones

[0025] High-pressure gaseous hydrogen storage involves mechanically compressing hydrogen gas into a high-pressure resistant container, thereby reducing the intermolecular distance of hydrogen gas through physical means and increasing the hydrogen storage capacity per unit volume. Its physical characteristics are represented by the following model: ; in, The hydrogen storage mass is directly proportional to the hydrogen storage pressure and the volume of the hydrogen storage tank, and inversely proportional to the absolute temperature of the hydrogen storage tank. For hydrogen storage pressure; This refers to the volume of the hydrogen storage tank. This refers to the absolute temperature of the hydrogen storage tank. The compressibility factor is used to correct for non-ideal gases under high pressure. It is the gas constant; denoted as , where is the molar mass of hydrogen gas.

[0026] Cryogenic liquid hydrogen storage involves cooling hydrogen gas below its boiling point, causing it to condense from a gaseous state into a liquid state, thus significantly increasing the hydrogen storage density through phase change. This method boasts the highest energy density, approximately 10 times that of lithium batteries, making it suitable for high-end applications such as long-endurance drones. Its physical characteristics are represented by the following model: ; in, The hydrogen storage mass of liquid hydrogen storage can be expressed as the product of the density of liquid hydrogen and the actual liquid volume. The actual liquid volume is determined by the percentage of liquid level height, that is, the product of the hydrogen storage level and the container volume. The density of liquid hydrogen; Hydrogen storage level; The volume of the container; Although this method offers the highest energy density, it suffers from evaporation losses. Even when not used, liquid hydrogen will slowly evaporate due to incomplete adiabatic treatment, resulting in some loss. Its physical characteristics are as follows: ; in, The evaporation rate is proportional to the ratio of external heat leakage to the latent heat of vaporization of hydrogen. This is due to heat leakage from the outside. The latent heat of vaporization of hydrogen; The thermal conversion coefficient depends on the insulation performance of the hydrogen storage container and the ambient temperature difference.

[0027] Metal hydride hydrogen storage utilizes the reversible chemical reaction between a metal / alloy and hydrogen to fix hydrogen in the metal lattice through chemical bonds. This embodiment uses a lanthanum-nickel alloy (LaNi5). This method offers the highest safety (i.e., no risk of high-pressure explosion or cryogenic leakage) and has a high bulk density (approximately 50-150 g / L). Its physical properties are as follows: ; in, The equilibrium pressure is determined by both temperature and the thermodynamic properties of the material; For temperature; and These correspond to the enthalpy change and entropy change, respectively. When the system pressure is greater than the equilibrium pressure, the hydrogen storage rate increases; when the system pressure is less than the equilibrium pressure, the hydrogen storage rate decreases. The actual hydrogen storage capacity can be expressed as: ; in, This indicates the mass of hydrogen stored, and the hydrogen storage capacity. and hydrogen storage rate The product of these factors is used to change the equilibrium pressure through heating / cooling, thereby controlling the direction of hydrogen absorption and desorption.

[0028] The fuel cell uses a proton exchange membrane fuel cell (PEMFC), which outputs electrical energy through an electrochemical reaction that consumes hydrogen. The power generation efficiency and output power exhibit a non-linear relationship, and its physical characteristic model is expressed as follows: ; in, The power output of the fuel cell; For the power generation efficiency of fuel cells; These are the fitting coefficients; The hydrogen consumption model for fuel cells is expressed as follows: ; in, This represents the equivalent hydrogen consumption of a fuel cell. This is the lower heating value of hydrogen. This marks the starting point of the scheduling time. This is the scheduling time interval.

[0029] The lithium battery is modeled as a lithium-ion phosphate battery, serving as a power buffer unit. Its main function is to smooth dynamic load fluctuations during UAV flight and alleviate the instantaneous power regulation pressure on the fuel cell. Its physical characteristic model is as follows: ; in, This represents the total power output of the battery pack. Power per battery; and These represent the number of battery rows and columns, respectively. In this physical model, based on the short-time characteristics of the UAV during flight, the open-loop voltage of the battery is... and monomer internal resistance Approximating a constant value, the current expression for a single battery cell can then be derived: ; The state of charge (SOC) of a battery cell can be expressed as: ; SOC represents the state of charge of the battery. k (0≤ k ≤T) represents the scheduling time; The scheduling period; This refers to the battery cell capacity; It represents current; charging is negative, and discharging is positive.

[0030] S2. To minimize the hydrogen charging costs of high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, and metal hydride hydrogen storage, as well as the fuel cell lifespan loss costs, an objective function is constructed. Based on the constructed physical characteristic model and state of charge model, safe operation constraints and energy balance constraints are constructed as constraints on the objective function.

[0031] As a preferred implementation, the hydrogen charging costs for high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, and metal hydride hydrogen storage are calculated separately, and the total hydrogen charging cost is obtained by weighted summation. The fuel cell lifespan loss under all load conditions is calculated and multiplied by the corresponding cost to obtain the fuel cell lifespan loss cost. The total hydrogen charging cost and the fuel cell lifespan loss cost are added together as the objective function; the specific formula is as follows:

[0032] in, The objective function is... Total cost of hydrogen refueling; Costs related to fuel cell lifespan loss; , and The hydrogen refueling costs are for three types of hydrogen storage: high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, and metal hydride hydrogen storage. This is due to the loss of fuel cell lifespan. The power output of the fuel cell; Cost per unit of generated power; This refers to the rated power output of the fuel cell; , , and These are all set parameters, corresponding to the hydrogen charging cost coefficients and fuel cell life decay coefficients for high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, and metal hydride hydrogen storage, respectively.

[0033] As a preferred implementation method, the process for calculating the hydrogen charging cost of different hydrogen storage methods is as follows: For any form of hydrogen storage, the electricity cost is multiplied by the energy consumption coefficient of the corresponding hydrogen storage form to obtain the electricity cost per unit of hydrogen supply. The depreciation cost of hydrogen refilling for different hydrogen storage forms is divided by the lifespan of the corresponding hydrogen storage material and then multiplied by the dispatch cycle to obtain the depreciation cost of the corresponding hydrogen storage form. The hydrogen price, the electricity cost of the corresponding hydrogen storage form, and the depreciation cost are added together and multiplied by the hydrogen supply capacity of the corresponding hydrogen storage form to obtain the hydrogen refilling cost of the corresponding hydrogen storage form. The specific formula is as follows: ; In the formula, The price of hydrogen; For electricity prices; , and These are the compression energy consumption coefficient, liquefaction energy consumption coefficient, and material activation energy consumption coefficient, respectively. , and The depreciation costs for hydrogen refilling in three different hydrogen storage methods are as follows: , and These are the service life cycles of three different hydrogen storage materials; , and The hydrogen supply mass represents the hydrogen supply for three different hydrogen storage methods.

[0034] As a preferred implementation, the process for calculating the fuel cell lifetime loss under all load conditions is as follows: ; In the formula, the life loss cost of the fuel cell is constructed based on fuel cell aging test data. , as well as The fuel cell life loss is calculated under heavy load conditions, low load conditions, and fuel cell operating conditions, respectively. This represents the maximum power change rate. , , These are the loss coefficients set for different load conditions.

[0035] As a preferred implementation method, the process of constructing safety operation constraints and energy balance constraints is as follows: Based on the physical characteristic models of three different hydrogen storage methods, the hydrogen storage mass of each method is obtained; based on the state of charge model of lithium batteries, the corresponding state of charge is obtained; upper and lower limits are set for the hydrogen storage mass, the state of charge at different times, the power generation of the fuel cell, and the power change, respectively, to obtain safe operation constraints.

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] in, , and These represent the lower limits of hydrogen storage capacity for the three hydrogen storage methods. , and These represent the upper limits of hydrogen storage capacity for the three hydrogen storage methods; and These are the lower and upper limits of the state of charge (SOC) of lithium batteries, respectively. The power output of the fuel cell; and These are the upper and lower limits of the fuel cell's power generation capacity, respectively. This represents the change in power of the fuel cell; and These represent the upper and lower limits of the power variation of the fuel cell; Based on the physical characteristic model of the fuel cell, the charging and discharging power of the lithium battery is obtained; for any given time, the power generation of the fuel cell plus the charging and discharging power of the lithium battery is always equal to the power demand of the UAV at that time, thus obtaining the energy balance constraint:

[0043] in, and They are respectively The power generation capacity of the fuel cell and the charging and discharging power of the lithium battery at any given time. Power required for drones.

[0044] S3. Based on the power demand of the drone, the charging and discharging power and state of charge of the lithium battery, and the hydrogen storage quality and supply quality of the three hydrogen storage forms, a sequential decision-making model is constructed. The sequential decision-making model includes a state space, an action space, and a reward function. The reward function is obtained by weighting the calculated objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage quality penalty.

[0045] As a preferred implementation, considering the dynamic load fluctuations during UAV flight and the dynamic state changes of the multi-element hydrogen storage device, a sequential decision model is constructed based on Markov decision process (MDP), including: Based on the current moment, the current power demand of the drone, the state of charge of the lithium battery, and the hydrogen storage mass of the three hydrogen storage methods, a state space is established: ; in, For time steps; The current power requirements for drones; This represents the current state of charge of the lithium battery. Based on the power generation of fuel cells, the charging and discharging power of lithium batteries, and the hydrogen supply quality of the three hydrogen storage methods, an action space is established: ; in, and These are the power generation capacity of the fuel cell and the charging and discharging capacity of the lithium battery, respectively. Based on the state space at different times, the portion of the sum of the fuel cell's power generation and the lithium battery's charging and discharging power exceeding the drone's power requirements is considered as excess output power, and the portion of the sum of the fuel cell's power generation and the lithium battery's charging and discharging power failing to meet the drone's power requirements is considered as insufficient output power. The hydrogen storage mass exceeding the upper and lower limits for the three different energy storage forms is accumulated to obtain a hydrogen storage mass penalty. The reward function is obtained by weighting the current objective function value, excess output power, insufficient output power, the absolute value of the lithium battery's charging and discharging power, and the hydrogen storage mass penalty. ; in, For the reward function; The excess output power represents the power difference between the total output power (i.e., the sum of the power generation of the fuel cell and the charging and discharging power of the lithium battery) and the power required by the drone. The output power is insufficient, indicating the power difference between the total output power and the power required by the drone. , and These represent the hydrogen mass exceeding the upper and lower limits for three different energy storage methods. , , and These are the excess power coefficient, insufficient power coefficient, lithium battery charge / discharge coefficient, and hydrogen storage state coefficient, respectively.

[0046] In this embodiment, the reward function design must be consistent with the objective function, including two core indicators: the hydrogen refueling cost of the hydrogen storage device and the fuel cell lifespan degradation. To further improve the system's energy utilization efficiency, the following dynamic penalty mechanisms are introduced: ① Insufficient power supply penalty: to prevent the lithium battery from over-discharging and causing the system to shut down; ② Excess power penalty: to avoid ineffective energy output from the fuel cell; ③ Hydrogen storage device energy storage state penalty: to ensure the safety of the hydrogen storage state of each hydrogen storage device. Through these additional penalty terms, the agent can be effectively guided to quickly converge to the optimal strategy during training, achieving fuel cell operating point optimization, dynamic balance of lithium battery SOC, minimization of hydrogen refueling cost, and maximization of energy efficiency.

[0047] In the long-endurance energy management problem of unmanned aerial vehicles (UAVs), traditional reinforcement learning algorithms face the problems of sparse reward signals and delays due to the large optimization time span. Specifically, the long-term rewards for multi-step actions need to be calculated through temporal superposition, resulting in unclear immediate feedback of short-term state-action pairs. Moreover, long-term effects such as fuel cell lifespan degradation are difficult to accurately correlate with specific decision moments. Therefore, this embodiment introduces a multi-step temporal prediction mechanism for multiple hydrogen storage states and load demands, as well as a TD error-driven dynamic dual-zone experience replay mechanism, into the TD3 (Dual-Delay Deep Deterministic Policy Gradient) algorithm framework. (See reference...) Figure 3 ,include: Multi-step temporal prediction mechanism: Within the TD3 algorithm framework, the LSTM algorithm is used to additionally predict N scheduling cycles forward for each training step. This method can improve training efficiency, shorten the reward propagation path, accelerate the convergence of long-term value estimation, and enhance the correlation between single-step decisions and long-term goals.

[0048] The TD error-driven dynamic dual-zone experience replay mechanism uses temporal difference error (TD error) to measure the value of each experience and as the basis for priority allocation. Based on this, the experience replay pool is explicitly divided into two parts: a high TD error experience zone and a low TD error experience zone. Before the total amount of experience in the pool reaches the minimum batch sampling size, newly collected experience is temporarily stored in the low TD error experience zone. Subsequently, when the TD error of a new experience is lower than the maximum TD error value in the current low TD error zone, the experience is moved to the low TD error zone, and the existing experience with the maximum TD error in the low zone is swapped to the high TD error zone, thus achieving hierarchical dynamic management of experience. During sampling updates, a fixed-ratio mixed sampling strategy is adopted: 80% of the batch samples are drawn from the low TD error experience zone, and the other 20% are drawn from the high TD error experience zone, together forming a complete training batch for updating the neural network parameters.

[0049] S4. Predict the future power demand of the UAV by constructing a power prediction model; calculate the reward function value under the selected action based on the predicted power demand of the UAV and the selected action; optimize the sequential decision model through TD3 and dynamic dual-zone experience playback mechanism to solve the action selection strategy that maximizes the reward value.

[0050] As a preferred implementation, long-term sequential decision optimization is performed using the improved TD-EDDZER-MP-TD3 algorithm to achieve coordinated operation of the multi-element hydrogen storage device and the fuel cell lithium battery, including: In the power prediction stage, an LSTM network is initialized and trained using historical actual flight data. The training parameters are listed in Table 2. The network after training convergence will serve as a fixed power prediction model to predict the power demand of the UAV, providing a reliable multi-step time series power prediction basis for subsequent energy management algorithms. Table 2 Training parameters for power prediction model

[0051] In the energy management phase, the actor network and its target network are initialized, and two critic networks and their corresponding target networks are established simultaneously. The corresponding hyperparameter configurations (including noise attenuation rate, target network update frequency, etc.) are listed in Table 3. During the policy exploration process, for the current state, the actor network outputs an action, and exploration noise is injected in conjunction with an OU stochastic process to form the final execution action. After applying this action to the UAV simulation environment, the corresponding next state and reward function value can be obtained. Based on the predicted power demand of the UAV, the corresponding future reward is calculated, averaged, and added to the reward function value to obtain the enhanced reward value under the action. Based on the enhanced reward value under the action, the following calculations are performed. The TD error is calculated and stored in the corresponding high TD error experience zone or low TD error experience zone according to the error value. When the data accumulation in the experience replay pool reaches the preset threshold, the network parameter update process is started: each time, 80% of the samples are extracted from the low TD error zone and the remaining 20% ​​of the samples are extracted from the high TD error zone to form a training batch. The target action with noise is calculated and the minimum value of the output of the bi-objective critic network is selected as the action value, and then the critic network is updated. The actor network adopts a delayed update mechanism and optimizes by maximizing the policy gradient of the expected action value, thereby realizing end-to-end autonomous learning of the entire UAV energy management strategy.

[0052] Table 3. Main training parameters of the energy management algorithm

[0053] The training process of multi-step time series prediction is achieved through Figure 4 Two curves (the upper curve represents the change in root mean square error (RMSE), and the lower curve represents the change in training loss) are presented visually. As can be seen from the graph, the model exhibits a remarkably rapid decline in the initial stage (the first 200 rounds): the RMSE drops rapidly from approximately 8 to below 1, and the training loss also plummets from approximately 35 to near 0. This phenomenon fully demonstrates that the multi-step time-series prediction mechanism employed in this invention can quickly capture the temporal characteristics of dynamic load fluctuations during UAV flight and the energy transfer patterns of the multi-element hydrogen storage system, exhibiting extremely high learning efficiency and effectively solving the technical problem of insufficient adaptability to dynamic operating conditions in traditional methods. After 200 rounds, the curve enters a stable convergence phase: the RMSE remains consistently below 0.5 (fluctuation amplitude less than 0.1), and the training loss approaches 0 with minimal fluctuation (less than 0.5). This indicates that the model has fully converged, parameter updates have entered a fine-tuning state, and there is no overfitting or underfitting phenomenon, demonstrating excellent generalization ability. The extremely low RMSE value indicates that the model's prediction error for load fluctuations is minimal, enabling precise guidance of power allocation between fuel cells, lithium batteries, and multi-element hydrogen storage devices.

[0054] Figure 5The training rewards of the traditional dual-delay deep deterministic policy gradient (TD3) algorithm and the improved algorithm (TD-EDDZER-MP-TD3) proposed in this invention, which integrates TD error-driven dynamic dual-zone experience replay and multi-step time series prediction, are compared. The curves show that the reward values ​​of both algorithms gradually increase and then stabilize with the increase of training rounds, indicating that the system performance continuously optimizes as training progresses. Among them, the improved algorithm of this invention significantly outperforms the traditional TD3 algorithm: In the initial stage (first 40 rounds), the reward value of the TD-EDDZER-MP-TD3 algorithm rapidly climbs from about -1200 to about -200, while the TD3 algorithm only rises to about -400, showing that the improved algorithm has a higher learning efficiency for the temporal characteristics of the dynamic load fluctuation of the UAV and the energy transfer law of the multi-element hydrogen storage system; After entering the stable stage, the average reward of the TD-EDDZER-MP-TD3 algorithm is always maintained between -100 and 0, and the fluctuation range is significantly smaller than that of the TD3 algorithm (the average reward of TD3 is stable between -300 and -100), indicating that the improved algorithm strengthens the utilization of high-value experience through the dynamic dual-zone experience replay mechanism driven by TD error, and achieves the optimal energy allocation for long-endurance missions by combining multi-step temporal prediction, effectively solving the problems of insufficient adaptability and low energy scheduling efficiency of traditional algorithms under dynamic conditions, and significantly improving the system energy utilization efficiency and fuel cell lifespan.

[0055] from Figure 6 As can be seen, the total power demand exhibits significant dynamic fluctuations (e.g., a peak of over 2000 watts appears around 150 seconds, and the overall power changes frequently within the 0-2500 watt range), while the fuel cell output power remains relatively stable (maintaining between 500-1500 watts for most of the time), without significant fluctuations due to the drastic changes in total demand; the lithium battery output power exhibits a dynamic adjustment highly coordinated with total demand—when total demand is higher than fuel cell output (e.g., 100-150 seconds), the lithium battery releases power (positive output) to compensate for the difference; when total demand is lower than fuel cell output (e.g., 50-100 seconds), the lithium battery absorbs excess power (negative output) to recharge.

[0056] Figure 7The battery state of matter (SOC) changes as follows: In the initial stage (0-50 seconds), due to high total demand, the lithium battery continuously discharges, and the SOC rapidly decreases from 0.5 to 0.45. Subsequently (50-150 seconds), as the fuel cell output stabilizes, the lithium battery dynamically adjusts between discharging and charging, maintaining the SOC between 0.42 and 0.45 (only slightly decreasing to 0.42 during the peak period at 150 seconds). After 150 seconds, the fluctuations in total demand level off, and the SOC remains within a reasonable range of 0.43-0.45, without over-discharging (<0.4) or overcharging (>0.5). This indicates that the strategy not only meets the power requirements of the dynamic load but also ensures the health of the lithium battery through precise energy allocation, extending its lifespan.

[0057] Figure 8This study demonstrates the dynamic changes in real-time hydrogen release mass and total hydrogen release mass in a multi-element hydrogen storage hybrid drone, encompassing high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, and metal hydride hydrogen storage. The total hydrogen release mass fluctuates over time, and its fluctuation trend is highly consistent with the fuel cell output power (e.g., a small peak of 0.02 g / s in total hydrogen release mass occurs around 150 seconds, corresponding to the peak of fuel cell output power in the previous power distribution diagram). This indicates that the total hydrogen release mass strictly matches the real-time power demand of the fuel cell, achieving "on-demand hydrogen supply." This feature reflects the effectiveness of the multi-step temporal prediction mechanism in the deep reinforcement learning strategy of this invention—by sensing load fluctuations in advance, accurately controlling the total hydrogen supply, and avoiding the waste of hydrogen resources. Meanwhile, the hydrogen release curves of the three hydrogen storage forms show a clear division of labor: the hydrogen release fluctuation of high-pressure gaseous hydrogen storage is the most obvious (such as multiple small peaks at 50-100 seconds and 150-200 seconds), and the peak value coincides with the peak value of the total hydrogen release mass, indicating that its rapid response characteristics are fully utilized. When the load suddenly increases, high-pressure gaseous hydrogen storage compensates for the response delay of other hydrogen storage forms by rapidly releasing hydrogen, ensuring that the fuel cell does not reduce efficiency due to hydrogen shortage; the hydrogen release curve of cryogenic liquid hydrogen storage is the most stable (maintained between 0.003-0.005 g / s for most of the time). Furthermore, it consistently accounts for a fundamental share of the total hydrogen release mass, demonstrating its high energy density as a "basic hydrogen source." During periods of stable load, it continuously and stably releases hydrogen, reducing frequent starts-ups of high-pressure gaseous hydrogen storage and reaction losses from metal hydride hydrogen storage. Metal hydride hydrogen storage exhibits the smallest fluctuation in its hydrogen release curve, and its share slightly increases during low-load periods (e.g., after 200 seconds), indicating that its safe and long-lasting characteristics serve as a "supplementary hydrogen source." When the load is moderate or the number of starts for high-pressure gaseous hydrogen storage needs to be reduced, it maintains system energy balance through slow hydrogen release, further optimizing energy utilization efficiency. This dynamic and coordinated hydrogen release mode fully integrates the advantages of the three hydrogen storage methods, overcoming the limitations of a single hydrogen storage method, while ensuring that the fuel cell always operates in a stable hydrogen concentration environment, reducing its lifespan degradation. In addition, the "redundant hydrogen supply" design of the three hydrogen storage methods improves the system's reliability in complex mission environments, while the dynamically adjusted hydrogen release strategy reduces over-reliance on a single hydrogen storage method, decreasing the drone's weight and cost, thus supporting commercial applications.

[0058] Example 2 Embodiment 2 of the present invention provides an energy management optimization system for hydrogen-powered drones with multi-element hydrogen storage, and uses a method for energy management optimization of hydrogen-powered drones with multi-element hydrogen storage, including: The physical model building module constructs physical characteristic models for high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, metal hydride hydrogen storage, fuel cells, and lithium batteries, as well as a state of charge model for lithium batteries. The optimization objective establishment module aims to minimize the hydrogen charging cost of high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, and metal hydride hydrogen storage, as well as the fuel cell life loss cost, and constructs an objective function. Based on the constructed physical characteristic model and state of charge model, safe operation constraints and energy balance constraints are constructed as constraints on the objective function. The decision model construction module constructs a sequential decision model based on the power demand of the drone, the charging and discharging power and state of charge of the lithium battery, and the hydrogen storage quality and supply quality of the three hydrogen storage forms. The sequential decision model includes a state space, an action space, and a reward function. The reward function is obtained by weighting the calculated objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage quality penalty. The energy management optimization module predicts the future power demand of drones through a constructed power prediction model; based on the predicted drone power demand and the selected action, it calculates the reward function value under the selected action; and through TD3 and dynamic dual-zone experience playback mechanism, it optimizes the sequential decision model to solve the action selection strategy that maximizes the reward value.

[0059] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0060] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0061] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0062] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for optimizing energy management of hydrogen-powered unmanned aerial vehicles (UAVs) with multiple hydrogen storage forms, characterized in that, include: S1. Construct physical characteristic models for high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, metal hydride hydrogen storage, fuel cells, and lithium batteries, as well as a state of charge model for lithium batteries. S2. To minimize the hydrogen charging cost of high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, and metal hydride hydrogen storage, as well as the fuel cell life loss cost, an objective function is constructed. Based on the constructed physical characteristic model and state of charge model, safe operation constraints and energy balance constraints are constructed as constraints on the objective function. S3. Based on the power demand of the drone, the charging and discharging power and state of charge of the lithium battery, and the hydrogen storage quality and supply quality of the three hydrogen storage forms, a sequential decision model is constructed. The sequential decision model includes a state space, an action space, and a reward function. The reward function is obtained by weighting the calculated objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage quality penalty. S4. Predict the future power demand of the UAV by constructing a power prediction model; calculate the reward function value under the selected action based on the predicted power demand of the UAV and the selected action; optimize the sequential decision model through TD3 and dynamic dual-zone experience playback mechanism to solve the action selection strategy that maximizes the reward value.

2. The energy management optimization method for hydrogen-powered unmanned aerial vehicles with multiple hydrogen storage forms according to claim 1, characterized in that: The process of constructing the objective function in S2 is as follows: The hydrogen charging costs for high-pressure gaseous hydrogen storage, cryogenic liquid hydrogen storage, and metal hydride hydrogen storage are calculated separately, and the total hydrogen charging cost is obtained by weighted summation. The fuel cell life loss under all load conditions is calculated and multiplied by the corresponding cost to obtain the fuel cell life loss cost. The total hydrogen charging cost and the fuel cell life loss cost are added together as the objective function.

3. The energy management optimization method for hydrogen-powered unmanned aerial vehicles with multiple hydrogen storage forms according to claim 1 or 2, characterized in that: The process for calculating the hydrogen charging cost for different hydrogen storage methods in S2 is as follows: For any form of hydrogen storage, the electricity cost is multiplied by the energy consumption coefficient of the corresponding hydrogen storage form to obtain the electricity cost per unit of hydrogen supply for that form. The depreciation cost of hydrogen charging for different hydrogen storage forms is divided by the lifespan of the corresponding hydrogen storage material and then multiplied by the dispatch cycle to obtain the depreciation cost of the corresponding hydrogen storage form. The hydrogen price, the electricity cost of the corresponding hydrogen storage form, and the depreciation cost are added together and then multiplied by the hydrogen supply mass of the corresponding hydrogen storage form to obtain the hydrogen charging cost of the corresponding hydrogen storage form.

4. The energy management optimization method for hydrogen-powered unmanned aerial vehicles with multiple hydrogen storage forms according to claim 1 or 2, characterized in that: The process for calculating fuel cell lifetime loss under all load conditions in S2 is as follows: ； In the formula, This is due to the loss of fuel cell lifespan. The power output of the fuel cell; , as well as The fuel cell life loss is calculated under heavy load conditions, low load conditions, and fuel cell operating conditions, respectively. This refers to the rated power output of the fuel cell; This represents the maximum power change rate; This marks the starting point of the scheduling time. The scheduling time interval.

5. The energy management optimization method for hydrogen-powered unmanned aerial vehicles with multiple hydrogen storage forms according to claim 1, characterized in that: The process of constructing safe operation constraints and energy balance constraints in S2 is as follows: Based on the physical characteristic models of three different hydrogen storage forms, the hydrogen storage mass of each form is obtained; based on the state of charge model of lithium batteries, the corresponding state of charge is obtained; upper and lower limits are set for the hydrogen storage mass, the state of charge at different times, the power generation of the fuel cell, and the power change, respectively, to obtain safe operation constraints. Based on the physical characteristic model of the fuel cell, the charging and discharging power of the lithium battery is obtained; for any given time, the power generation of the fuel cell plus the charging and discharging power of the lithium battery is always equal to the power demand of the UAV at the same time, thus obtaining the energy balance constraint.

6. The energy management optimization method for hydrogen-powered unmanned aerial vehicles with multiple hydrogen storage forms according to claim 1, characterized in that: In S3, a state space is established based on the current time, the power demand of the drone at the current time, the state of charge of the lithium battery, and the hydrogen storage mass of the three hydrogen storage forms. Based on the power generation of fuel cells, the charging and discharging power of lithium batteries, and the hydrogen supply quality of the three hydrogen storage methods, an action space is established. Based on the state space at different times, the portion of the sum of the power generation of the fuel cell and the charging and discharging power of the lithium battery that exceeds the power demand of the drone is regarded as the excess output power, and the portion of the sum of the power generation of the fuel cell and the charging and discharging power of the lithium battery that cannot meet the power demand of the drone is regarded as the insufficient output power. The hydrogen storage mass penalty is obtained by summing the hydrogen storage mass exceeding the upper and lower limits of the three different energy storage forms. The reward function is obtained by weighting the objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage mass penalty at the current time.

7. The energy management optimization method for hydrogen-powered unmanned aerial vehicles with multiple hydrogen storage forms according to claim 1, characterized in that: The optimization process for the sequential decision model in S4 is as follows: Initialize the actor network and its target network, and simultaneously establish two critic networks and their corresponding target networks. During policy exploration, for the current state, the actor network outputs an action and injects exploration noise to form the final execution action. After applying this action to the UAV simulation environment, the corresponding next state and reward function value can be obtained. Based on the predicted power demand of the UAV, the corresponding future reward is calculated, averaged, and added to the reward function value to obtain the enhanced reward value under the action. Based on the enhanced reward value under the action, the TD error is calculated and stored in the corresponding high TD error experience zone or low TD error experience zone according to the error value. When the data accumulation in the experience replay pool reaches a preset threshold, the network parameter update process is started: each time, 80% of the samples are extracted from the low TD error zone and the remaining 20% ​​of the samples are extracted from the high TD error zone to form a training batch. By calculating the noisy clipped actions, the minimum value output by the dual-target critic network is selected as the action value, and then the critic network is updated. The actor network adopts a delayed update mechanism and is optimized by maximizing the policy gradient of the expected action value.

8. A multi-element hydrogen storage form hydrogen-powered unmanned aerial vehicle energy management optimization system, using the method described in any one of claims 1-7, characterized in that, include: The physical model building module constructs physical characteristic models for high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, metal hydride hydrogen storage, fuel cells, and lithium batteries, as well as a state of charge model for lithium batteries. The optimization objective establishment module aims to minimize the hydrogen charging cost of high-pressure gaseous hydrogen storage, low-temperature liquid hydrogen storage, and metal hydride hydrogen storage, as well as the fuel cell life loss cost, and constructs an objective function. Based on the constructed physical characteristic model and state of charge model, safe operation constraints and energy balance constraints are constructed as constraints on the objective function. The decision model construction module constructs a sequential decision model based on the power demand of the drone, the charging and discharging power and state of charge of the lithium battery, and the hydrogen storage quality and supply quality of the three hydrogen storage forms. The sequential decision model includes a state space, an action space, and a reward function. The reward function is obtained by weighting the calculated objective function value, the excess output power, the insufficient output power, the absolute value of the charging and discharging power of the lithium battery, and the hydrogen storage quality penalty. The energy management optimization module predicts the future power demand of drones through a constructed power prediction model; based on the predicted drone power demand and the selected action, it calculates the reward function value under the selected action; and through TD3 and dynamic dual-zone experience playback mechanism, it optimizes the sequential decision model to solve the action selection strategy that maximizes the reward value.

9. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-7.

Images