Energy management optimization method and system for hydrogen-electricity hybrid unmanned aerial vehicle based on TD3 algorithm

Abstract

The invention discloses a hydrogen-electricity hybrid unmanned aerial vehicle energy management optimization method and system based on a TD3 algorithm, and relates to the technical field of hybrid unmanned aerial vehicle energy optimization. The method comprises the following steps: establishing a neural network structure; designing a total reward function based on the battery SOC, the fuel consumption and the battery health state, and calculating a total reward through punishment of different factors; and constructing a TD3 agent, initializing an input variable and an output result of the TD3 agent, calculating a total reward according to a total reward function, and performing performance optimization on the TD3 agent. Based on a reinforcement learning strategy, factors such as multiple targets and multiple constraints, hydrogen consumption, battery SOC balance and capacity fading are considered, a reward function is designed, the state and action information of the system is defined, the network structure is trained through an intelligent agent of a TD3 algorithm, the optimal action is predicted, excessive estimation can be reduced, the stability and convergence speed are improved, the exploration performance is enhanced, and the method is suitable for large-scale popularization and application. And the method is suitable for various continuous action control.

Description

Technical Field

[0001] The present invention relates to the technical field of energy optimization for hybrid unmanned aerial vehicles, and specifically to an energy management optimization method for hydrogen-electric hybrid unmanned aerial vehicles based on the TD3 algorithm. Background Art

[0002] Hydrogen fuel hybrid unmanned aerial vehicles have the characteristics of light weight and high energy density, and are widely used in multiple fields. However, due to existing energy management strategies, harsh operating conditions, and improper thermal management solutions, hydrogen fuel hybrid unmanned aerial vehicles have problems such as low overall energy efficiency, weak operating condition adaptability, low thermal management level, and poor intelligence level in practical applications.

[0003] Currently, the research methods for energy management strategies of hybrid power systems are as follows: (1) Design strategies required for different scenarios or operating conditions according to expert experience; (2) Based on the optimized energy management strategy, a cost function is usually constructed according to the response of the system to be tested to the operating conditions, and a certain goal is optimized under constraint conditions, such as reducing fuel consumption and accelerating power response; The optimized energy management strategy can be divided into a global optimization strategy and a local optimization strategy; (3) The strategy based on the equivalent factor requires a relatively accurate mathematical model, so the design difficulty is relatively large; (4) The energy management strategy based on learning is a relatively popular research direction at present and has good adaptability.

[0004] However, traditional energy management strategies all have problems of poor stability and convergence speed when dealing with continuous action space problems. For this reason, the present invention proposes an energy management optimization method and system for hydrogen-electric hybrid unmanned aerial vehicles based on the TD3 algorithm. Summary of the Invention

[0005] The purpose of the present invention is to provide an energy management optimization method and system for hydrogen-electric hybrid unmanned aerial vehicles based on the TD3 algorithm, which can reduce overestimation, improve stability and convergence speed, enhance exploration performance, and be applicable to various continuous action control problems.

[0006] To achieve the above purpose, the present invention provides the following technical solution: An energy management optimization method for hydrogen-electric hybrid unmanned aerial vehicles based on the TD3 algorithm, including the following steps:

[0007] Build a neural network structure, specifically including an Actor network, a Critic network, a Target Actor network, and a Target Critic network;

[0008] Design a total reward function based on the battery SOC, fuel consumption, and battery health status, and calculate the total reward by punishing different factors;

[0009] Build a TD3 agent, initialize the input variables and output results of the TD3 agent, calculate the total reward according to the total reward function, and optimize the performance of the TD3 agent;

[0010] Train the neural network structure according to the input variables, output results and total reward function of the TD3 agent, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 agent;

[0011] After the network structure training is completed, set the iteration termination condition, terminate the iteration when the condition is met, and record and save the corresponding optimal actions and state variables.

[0012] Furthermore, build a neural network structure, specifically including an Actor network, a Critic network, a Target Actor network, and a Target Critic network, as follows:

[0013] (21) Use the Actor-Critic framework, introduce a double Q network, and at the same time adopt a delayed update strategy to alternately train the Actor and the Critic;

[0014] (22) Randomly sample a certain number of samples from the experience pool, then calculate the gradients of the Actor / Critic, and update the parameters of the Actor / Critic;

[0015] (23) Update the parameters of the Target Actor and the Target Critic using a soft update method.

[0016] Furthermore, design a total reward function based on the battery SOC, fuel consumption, and battery health status, and calculate the total reward by punishing different factors, as follows:

[0017] Penalty for battery SOC:

[0018] Penalty soc_ess =(SOC new -0.6) 2

[0019] If the SOC of the battery deviates from 0.6, it will be punished by squaring;

[0020] Penalty for fuel consumption:

[0021] Penalty cons_fc =out.fc_cons

[0022] The more fuel is consumed, the greater the penalty;

[0023] Penalty for fuel cell health status:

[0024] Penalty dosh_fc = out.fc_dosh

[0025] The greater the health loss of the fuel cell, the greater the penalty;

[0026] Penalty for battery health status:

[0027] Penalty dosh_ess = out.ess_dosh

[0028] The greater the health loss of the battery, the greater the penalty;

[0029] Therefore, the total reward function is designed as follows:

[0030]

[0031] The greater the total reward, the better the action taken by the agent at the current time step; conversely, the smaller the reward, the worse the action quality of the agent.

[0032] Furthermore, the input variables of the TD3 agent include demand power, power parameters, and battery parameters;

[0033] The output results include the optimal fuel cell power trajectory of the UAV, fuel consumption, equivalent hydrogen consumption, and battery SOC.

[0034] Furthermore, the neural network structure is trained according to the input variables, output results, and total reward function of the TD3 agent, the battery SOC and fuel consumption are iteratively updated, and a safe layer is added to limit the actions of the TD3 agent, specifically as follows:

[0035] (1) Initialize the input variables

[0036] Demand power:

[0037] Takeoff power Power_takeoff = 52kW, climb power Power_climb = 40kW,

[0038] Cruise power 1 Power_curise1 = 18kW, cruise power 2 Power_curise2 = 23kW,

[0039] Deceleration power Power_deceleration = 10kW, landing power Power_descent = 10kW;

[0040] Efficiency parameter:

[0041] Propeller efficiency eff_propeller = 0.85, motor and electronic speed control efficiency eff_motor = 0.92,

[0042] DC / DC efficiency eff_dcdc = 0.95;

[0043] Battery parameters:

[0044] Initial battery SOC_init = 0.6;

[0045] (2) Define output variables: optimal fuel cell power trajectory, fuel consumption, equivalent hydrogen consumption, and battery SOC;

[0046] (3) Train the neural network structure according to the TD3 agent combined with the total reward function, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 agent.

[0047] Further, the iteration termination condition is specifically as follows:

[0048] 0.2 < SOC < 0.8

[0049] Judge the final output SOC. If the condition is met, terminate the iterative simulation.

[0050] According to the second aspect of the present invention, the present invention provides a hydrogen - electric hybrid UAV energy management optimization system based on the TD3 algorithm, which is used to implement the above - mentioned hydrogen - electric hybrid UAV energy management optimization method based on the TD3 algorithm, including:

[0051] A building module, which is used to build a neural network structure, specifically including an Actor network, a Critic network, a Target Actor network, and a Target Critic network;

[0052] A total reward function design module, which is used to design a total reward function based on the battery SOC, fuel consumption, and battery health state, and calculate the total reward by punishing different factors;

[0053] An agent construction module, which is used to construct a TD3 agent, initialize the input variables and output results of the TD3 agent, calculate the total reward according to the total reward function, and optimize the performance of the TD3 agent;

[0054] A training module, which is used to train the neural network structure according to the input variables, output results, and total reward function of the TD3 agent, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 agent;

[0055] An iterative output module, which is used to set the iteration termination condition after the network structure training is completed, terminate the iteration when the condition is met, and record and output the corresponding optimal actions and state variables.

[0056] According to the third aspect of the present invention, the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores a computer program capable of running on the processor. When the processor loads and executes the computer program, the above-mentioned energy management optimization method for hydrogen-electric hybrid drones based on the TD3 algorithm is adopted.

[0057] According to the fourth aspect of the present invention, the present invention provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the above-mentioned energy management optimization method for hydrogen-electric hybrid drones based on the TD3 algorithm when executed by a computer processor.

[0058] According to the fifth aspect of the present invention, the present invention provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it is used to load and execute the above-mentioned energy management optimization method for hydrogen-electric hybrid drones based on the TD3 algorithm.

[0059] The present invention at least has the following beneficial effects:

[0060] 1. By introducing a dual Q-network, the present invention uses two independent Q-networks to evaluate the value of each action respectively, which can more accurately estimate the true value of the action, thereby reducing overestimation.

[0061] 2. The present invention uses the TD3 algorithm to adopt a delayed update strategy, enabling the actor to more robustly search for the optimal strategy, improving stability and convergence speed.

[0062] 3. By adding noise to the output of the target actor network, the present invention enables the algorithm to more widely try different actions during the exploration process, thereby increasing the possibility of finding a better solution.

[0063] 4. The present invention is applicable to a variety of continuous action control problems. The significant improvement of this algorithm in terms of stability and convergence speed, as well as its enhanced exploration performance, is an effective algorithm applicable to a variety of continuous action control problems.

[0064] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic flow chart of the optimization method described in the present invention;

[0066] Figure 2 It is a topology diagram of the hybrid power system of the hydrogen fuel hybrid drone of the present invention;

[0067] Figure 3This is the schematic diagram of the energy management optimization control based on the TD3 algorithm of the present invention.

[0068] Reference numerals: Specific implementation manners

[0069] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0070] Embodiment 1:

[0071] Please refer to Figure 1 , the present invention provides a technical solution: a hydrogen-electric hybrid UAV energy management optimization method based on the TD3 algorithm, including the following steps:

[0072] S1. Build a neural network structure, use the Actor-Critic framework, introduce a dual Q network, and at the same time adopt a delayed update strategy to alternately train the Actor and the Critic. Randomly sample a certain number of samples from the experience pool, then calculate the gradients of the Actor / Critic, and update the parameters of the Actor / Critic; then, use the soft update method to update the parameters of the Target Actor and the Target Critic;

[0073] It should be noted that when building the neural network structure, the Actor is responsible for generating output actions according to the state, and its goal is to learn an optimal policy π(a|s); the Critic is used to evaluate the score of the output action of the Actor, that is, to evaluate the quality of the action generated by the Actor, and its goal is to learn an optimal Q-value function, that is, to maximize the long-term cumulative reward of executing the action At in the state St; the Target Actor and the Target Critic are respectively parts introduced to reduce the variance during the update of the Actor and the Critic, and their functions are to make the outputs of the Actor and the Critic smoother; the Target network has the same neural network structure as the Actor and the Critic networks, but their parameters are obtained through the soft update method;

[0074] S2. Design a reward function, and its design mainly considers the battery SOC new , fuel consumption out.fc_cons, and battery health states out.fc_dosh, out.fc_ess_dosh:

[0075] Penalty for battery SOC:

[0076] Penalty soc_ess =(SOC new -0.6) 2

[0077] Its purpose is to maintain the SOC of the battery near 0.6, and the SOC deviating from 0.6 will be penalized quadratically;

[0078] Fuel consumption penalty:

[0079] Penalty cons_fc =out.fc_cons

[0080] Its purpose is to reduce fuel consumption. The more fuel is consumed, the greater the penalty;

[0081] Fuel cell state of health penalty:

[0082] Penalty dosh_fc =out.fc_dosh

[0083] Its purpose is to protect the state of health of the fuel cell. The greater the health loss, the greater the penalty;

[0084] Battery state of health penalty:

[0085] Penalty dosh_ess =out.ess_dosh

[0086] The purpose is to protect the state of health of the battery. The greater the health loss, the greater the penalty;

[0087] The total reward function is designed as follows:

[0088]

[0089] The greater the reward, the better the action taken by the agent at the current time step; conversely, the smaller the reward, the worse the quality of the agent's action. In this way, the agent will gradually learn how to take the optimal action in different states during the training process to maximize its long-term cumulative reward;

[0090] S3. Construct a TD3 agent, initialize the input variables and output results of the TD3 agent, calculate the total reward according to the total reward function, and optimize the performance of the TD3 agent:

[0091] Initialize the input data: required power, power parameters, and battery parameters, and obtain the output results: the optimal fuel cell power trajectory of the UAV, fuel consumption, equivalent hydrogen consumption, and battery SOC. Calculate the total reward through the above total reward function, and continuously adjust the parameters to train the TD3 agent;

[0092] S4. Train the neural network structure based on the input variables, output results, and total reward function of the TD3 agent, iteratively update the battery SOC and fuel consumption, add a safe layer to limit the actions of the TD3 agent, improve safety and stability, set hyperparameters, and save the model;

[0093] (S41) Initialize the input variables

[0094] Demand power:

[0095] Takeoff power Power_takeoff = 52kW, climb power Power_climb = 40kW,

[0096] Cruise power 1Power_curise1 = 18kW, cruise power 2Power_curise2 = 23kW,

[0097] Deceleration power Power_deceleration = 10kW, landing power Power_descent = 10kW;

[0098] Efficiency parameters:

[0099] Propeller efficiency eff_propeller = 0.85, motor and electronic speed control efficiency eff_motor = 0.92,

[0100] DC / DC efficiency eff_dcdc = 0.95;

[0101] Battery parameters:

[0102] Initial battery SOC_init = 0.6;

[0103] (S42) Define the output variables optimal fuel cell power trajectory, fuel consumption, equivalent hydrogen consumption, and battery SOC;

[0104] (S43) Train the neural network structure according to the TD3 agent combined with the total reward function, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 agent;

[0105] S5. After the network structure training is completed, set the termination conditions:

[0106] 0.2 < SOC < 0.8

[0107] Judge the final output SOC. If the condition is met, terminate the simulation and record the optimal actions and state variables.

[0108] In summary, based on the reinforcement learning strategy, this embodiment considers multiple objectives and multiple constraints, such as hydrogen consumption, battery SOC balance, and capacity attenuation, designs a reward function, defines the state and action information of the system, and predicts the optimal action through the intelligent agent training network structure of the TD3 algorithm. It can reduce overestimation, improve stability and convergence speed, enhance exploration performance, and is applicable to various continuous action controls.

[0109] Embodiment 2:

[0110] This embodiment provides a hydrogen-electric hybrid UAV energy management optimization system based on the TD3 algorithm, which is used to implement the above-mentioned hydrogen-electric hybrid UAV energy management optimization method based on the TD3 algorithm, including:

[0111] A building module, which is used to build a neural network structure, specifically including an Actor network, a Critic network, a Target Actor network, and a Target Critic network;

[0112] A total reward function design module, which is used to design a total reward function based on the battery SOC, fuel consumption, and battery health status, and calculate the total reward through the punishment of different factors;

[0113] An intelligent agent construction module, which is used to construct a TD3 intelligent agent, initialize the input variables and output results of the TD3 intelligent agent, calculate the total reward according to the total reward function, and optimize the performance of the TD3 intelligent agent;

[0114] A training module, which is used to train the neural network structure according to the input variables, output results, and total reward function of the TD3 intelligent agent, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 intelligent agent;

[0115] An iterative output module, which is used to set iterative termination conditions after the network structure training is completed, terminate the iteration when the conditions are met, and record and output the corresponding optimal actions and state variables.

[0116] Specifically, the above-mentioned building module, total reward function design module, intelligent agent construction module, training module, and iterative output module can be embedded in a computer processing system. The computer calls the above-mentioned modules to complete the task of optimal energy management for the hydrogen-electric hybrid UAV according to the above-mentioned hydrogen-electric hybrid UAV energy management optimization method based on the TD3 algorithm; the above-mentioned building module, total reward function design module, intelligent agent construction module, training module, and iterative output module can perform operations according to the specific steps given by the above-mentioned hydrogen-electric hybrid UAV energy management optimization method based on the TD3 algorithm.

[0117] It should be noted that the division of each module of the above system is only a division of logical functions. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. Moreover, these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the shared remote driving system construction module can be a separately established processing element, or can be integrated in a certain chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and the function of the above signal processing module can be called and executed by a certain processing element of the above device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or can be independently implemented. The processing element mentioned here can be an integrated circuit with the ability to process signals. In the implementation process, each step of the above method or each of the above modules can be completed through the integrated logic circuit of the hardware in the processor element or the instruction in the form of software.

[0118] For example, the above modules can be one or more integrated circuits configured to implement the above method. For example: one or more Application Specific Integrated Circuits (ASICs), or, one or more Digital Signal Processors (DSPs), or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, when a certain module above is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processors that can call program code. Again, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0119] Embodiment 3:

[0120] The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores a computer program capable of running on the processor. When the processor loads and executes the computer program, the above-mentioned hydrogen-electric hybrid UAV energy management optimization method based on the TD3 algorithm is adopted.

[0121] It should be noted that the terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server, and the terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may further include input / output devices, network access devices, and a bus, etc.

[0122] Furthermore, the processor can adopt a central processing unit (CPU). Of course, according to the actual usage situation, other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc. This application does not make any restrictions in this regard.

[0123] Embodiment 4:

[0124] The present invention provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the above-mentioned energy management optimization method for hydrogen-electric hybrid drones based on the TD3 algorithm when executed by a computer processor.

[0125] Among them, the computer program can be stored in a computer-readable medium. The computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some middleware form, etc. The computer-readable medium includes any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.

[0126] Embodiment 5:

[0127] The present invention provides a computer program product, and the computer program product includes a computer program. When the computer program is executed by a processor, it is used to load and execute the above-mentioned energy management optimization method for hydrogen-electric hybrid drones based on the TD3 algorithm.

[0128] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device.

[0129] For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. When an element is referred to as being "assembled on", "mounted on", "fixed to" or "disposed on" another element, it can be directly on the other element or there may also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used herein are for illustrative purposes only and do not represent the only implementation.

[0130] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

[0131] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

Claims

1. An energy management optimization method for a hydrogen-electric hybrid unmanned aerial vehicle based on the TD3 algorithm, characterized in that It includes the following steps: Build a neural network structure, specifically including an Actor network, a Critic network, a Target Actor network, and a Target Critic network; Design a total reward function based on the battery state of charge (SOC), fuel consumption, and battery health status, and calculate the total reward by penalizing different factors; Construct a TD3 agent, initialize the input variables and output results of the TD3 agent, calculate the total reward according to the total reward function, and optimize the performance of the TD3 agent; Train the neural network structure according to the input variables, output results, and total reward function of the TD3 agent, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 agent; After the network structure training is completed, set the iteration termination condition, terminate the iteration when the condition is met, and record and save the corresponding optimal actions and state variables.

2. The energy management optimization method for a hydrogen-electric hybrid unmanned aerial vehicle based on the TD3 algorithm according to claim 1, characterized in that, Build a neural network structure, specifically including an Actor network, a Critic network, a Target Actor network, and a Target Critic network, as follows: (21) Use the Actor-Critic framework, introduce a double Q-network, and at the same time adopt a delayed update strategy to alternately train the Actor and the Critic; (22) Randomly sample a certain number of samples from the experience pool, then calculate the gradients of the Actor / Critic, and update the parameters of the Actor / Critic; (23) Update the parameters of the Target Actor and Target Critic using a soft update method.

3. The energy management optimization method for a hydrogen-electric hybrid UAV based on the TD3 algorithm according to claim 2, characterized in that, Design a total reward function based on the battery SOC, fuel consumption, and battery health status, and calculate the total reward by penalizing different factors, as follows: Penalty for battery SOC: Penalty soc_ess =(SOC new -0.6) 2 If the SOC of the battery deviates from 0.6, it will be penalized quadratically; Penalty for fuel consumption: Penalty cons_fc = out.fc_cons The more fuel is consumed, the greater the penalty; Penalty for fuel cell health status: Penalty dosh_fc = out.fc_dosh The greater the health loss of the fuel cell, the greater the penalty; Penalty for battery health status: Penalty dosh_ess = out.ess_dosh The greater the health loss of the battery, the greater the penalty; Therefore, the total reward function is designed as follows: The greater the total reward, the better the action taken by the agent at the current time step; conversely, the smaller the reward, the worse the quality of the agent's action.

4. The energy management optimization method for hydrogen-electric hybrid UAVs based on the TD3 algorithm according to claim 3, wherein: The input variables of the TD3 agent include the required power, power parameters, and battery parameters; The output results include the optimal fuel cell power trajectory of the drone, fuel consumption, equivalent hydrogen consumption, and battery SOC.

5. The energy management optimization method for a hydrogen-electric hybrid UAV based on the TD3 algorithm according to claim 4, wherein Train the neural network structure according to the input variables, output results, and total reward function of the TD3 agent, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 agent, as follows: (1) Initialize the input variables Required power: Takeoff power Power_takeoff = 52 kW, climb power Power_climb = 40 kW, Cruise power 1 Power_curise1 = 18 kW, cruise power 2 Power_curise2 = 23 kW, Deceleration power Power_deceleration = 10 kW, descent power Power_descent = 10 kW; Efficiency parameters: Propeller efficiency eff_propeller = 0.85, motor and electronic speed control efficiency eff_motor = 0.92, DC / DC efficiency eff_dcdc = 0.95; Battery parameters: Initial battery SOC_init = 0.6; (2) Define the optimal fuel cell power trajectory, fuel consumption, equivalent hydrogen consumption, and battery SOC of the output variables; (3) Train the neural network structure according to the TD3 agent combined with the total reward function, iteratively update the battery SOC and fuel consumption, and add a safe layer to limit the actions of the TD3 agent.

6. The energy management optimization method for hydrogen-electric hybrid UAVs based on the TD3 algorithm according to claim 5, characterized in that: The specific iteration termination conditions are as follows: 0.2 < SOC < 0.8 Judge the final output SOC. If the condition is met, terminate the iterative simulation.

7. A hydrogen-electric hybrid UAV energy management optimization system based on the TD3 algorithm, which is used to implement the hydrogen-electric hybrid UAV energy management optimization method according to any one of claims 1 to 6, and is characterized in that: Including: A building module for building a neural network structure, specifically including an Actor network, a Critic network, a Target Actor network, and a Target Critic network; A total reward function design module for designing a total reward function based on the battery SOC, fuel consumption, and battery health status, and calculating the total reward through penalties for different factors; An agent construction module for constructing a TD3 agent, initializing the input variables and output results of the TD3 agent, calculating the total reward according to the total reward function, and optimizing the performance of the TD3 agent; A training module for training the neural network structure according to the input variables, output results, and total reward function of the TD3 agent, iteratively updating the battery SOC and fuel consumption, and adding a safe layer to limit the actions of the TD3 agent; An iterative output module for setting the iteration termination condition after the network structure training is completed, terminating the iteration when the condition is met, and recording and outputting the corresponding optimal actions and state variables.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, the energy management optimization method for a hydrogen-electric hybrid UAV based on the TD3 algorithm described in any one of claims 1 to 6 is adopted.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the energy management optimization method for a hydrogen-electric hybrid UAV based on the TD3 algorithm described in any one of claims 1 to 6 when executed by a computer processor.

10. A computer program product, characterized in that, The computer program product includes a computer program. When the computer program is executed by a processor, it is used to load and execute the energy management optimization method for a hydrogen-electric hybrid UAV based on the TD3 algorithm described in any one of claims 1 to 6.

Images