Multi-task PEM unmanned aerial vehicle energy scheduling method and system based on MPC and DDPG

Abstract

The invention discloses a multi-task PEM unmanned aerial vehicle energy scheduling method and system based on MPC and DDPG, and the method comprises the steps: building a stress model and a power model in each flight mode according to the flight mode of a hydrogen fuel cell unmanned aerial vehicle; establishing a mathematical model, an objective function and operation constraints of the hydrogen fuel cell unmanned aerial vehicle energy system; constructing an MPC model by combining the stress model and the power model of the unmanned aerial vehicle in each flight mode, collecting state parameters of the unmanned aerial vehicle, and obtaining electric power predicted by the MPC model; and constructing a DDPG model based on the mathematical model, the objective function and the operation constraint of the hydrogen fuel cell unmanned aerial vehicle energy system, and taking the predicted electric power as the input of the trained DDPG model to obtain an energy scheduling result. According to the invention, energy utilization can be optimized in a complex task environment, and the task completion efficiency and cruising ability of the unmanned aerial vehicle are improved.

Description

Technical Field

[0001] The present invention relates to the fields of unmanned aerial vehicle intelligent control engineering and energy system engineering, and more specifically, to a multi-task PEM unmanned aerial vehicle energy scheduling method and system based on MPC and DDPG. Background Art

[0002] With the rapid development of drone technology, drones are increasingly used in logistics, environmental monitoring, agriculture, national defense and other fields. Traditional drones mainly rely on lithium batteries for power supply, but the energy density of lithium batteries is limited, resulting in limited flight time and payload capacity. To solve this problem, hydrogen fuel cells are gradually becoming a new power source for drones.

[0003] Hydrogen fuel cells have the advantages of high energy density, long endurance, and environmental friendliness, but their energy management issues still need to be optimized. In multi-flight mission scenarios, the energy consumption of drones is uncertain and complex. Therefore, a strategy that can intelligently schedule energy is needed to improve the endurance and mission completion efficiency of drones. Model predictive control (MPC) has become an effective tool for energy management due to its advantages in dealing with system dynamic constraints and short-term optimization. However, MPC relies on accurate system models and may perform poorly in complex or highly uncertain mission environments.

[0004] For hydrogen fuel cell hybrid UAVs, under multi-tasking, the uncertainty of UAV energy consumption is greater than that of hybrid vehicles, and the traditional MPC energy management method cannot meet the energy scheduling of UAVs. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the present invention provides a multi-task PEM UAV energy scheduling based on MPC and DDPG.

[0006] The present invention adopts the following technical solution.

[0007] A multi-task PEM UAV energy scheduling method based on MPC and DDPG includes the following steps:

[0008] According to the flight mode of hydrogen fuel cell drone, the force model and power model under each flight mode are established;

[0009] Establish the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system;

[0010] The MPC model is constructed by combining the force model and power model of each UAV flight mode, and the UAV state parameters are collected and the electric power predicted by the MPC model is obtained;

[0011] The DDPG model is constructed based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system. The predicted electric power is used as the input of the trained DDPG model to obtain the energy scheduling result.

[0012] Preferably, the flight modes of the hydrogen fuel cell drone include take-off, climb, cruise, descent, and landing; wherein climb includes steady straight climb, unsteady straight climb, and unsteady non-straight climb; cruise includes level flight acceleration and constant speed cruise; landing includes leveling deceleration stage, landing stage, and taxiing stage;

[0013] For each flight mode of the hydrogen fuel cell drone, the corresponding force model and power model under different flight modes are established respectively, wherein the force model includes vertical force and horizontal force.

[0014] Preferably, when the flight mode of the drone is in the take-off stage, its force model is as follows:

[0015]

[0016] The power model P is as follows:

[0017] P=P tf =P max

[0018] In the formula, F a，tf is the static acceleration of the UAV when it is gliding on the ground, T a，tf is the thrust available during the operation of the UAV, D tf is the air resistance of the drone, W is the gravity of the drone, L tf is the lift force on the drone, M is the total mass of the drone, a tf is the acceleration of the drone when it takes off, P max is the maximum power of the UAV, P tf is the take-off power of the UAV, and μ represents the friction resistance coefficient.

[0019] Preferably, if the UAV performs a steady straight climb, the forces in the two directions are balanced and the motor of the UAV runs at constant power. At this time, the force model of the UAV is as follows:

[0020]

[0021] Among them, γ is the flight climbing angle of the UAV, W is the gravity of the UAV, and L c Indicates the lift during the climb, T a，c represents the thrust available during the climb, D c Indicates the air resistance during the climb;

[0022] The power model P is as follows:

[0023] P=P c，r

[0024] Among them, P c，r is the rated power of the UAV during climbing;

[0025] If the UAV performs an unsteady straight climb, the force model of the UAV is as follows:

[0026]

[0027] In the formula, F a，c It represents the parallel direction resultant force during the climbing process;

[0028] The power model P is as follows:

[0029] P=P c =T a，c ·V(t)

[0030] Where P c represents the power during the climb, and V(t) represents the speed of the UAV;

[0031] If the UAV performs an unsteady and non-linear climb, the force model of the UAV is as follows:

[0032]

[0033] The power model P is as follows:

[0034] P=P c =T a，c ·V(t)

[0035] In the formula, F a，c，y Indicates the vertical force during the climbing process, F a，c，x Represents the parallel direction resultant force during the climbing process.

[0036] Preferably, when the UAV is in the level flight acceleration stage, the UAV climbs to the cruising altitude and then turns to level flight and continues to accelerate to reach the cruising speed. At this time, the force model of the UAV is as follows:

[0037] The force model is as follows:

[0038]

[0039] Where W is the gravity of the drone, L cr Indicates the lift during cruising, F a，cr It represents the horizontal force during cruising, T a，cr Denotes the available thrust during cruise, D cr Indicates the air resistance during cruising;

[0040] The power model P is as follows:

[0041] P=P cr =T a，cr ·V(t)

[0042] Where P cr Indicates the power during cruising;

[0043] If the level flight acceleration phase is uniform acceleration, the power model P satisfies:

[0044] P=P cr，r (a cr )

[0045] Among them, a cr The acceleration in the cruise acceleration phase is a constant value, and the power is a constant value P cr，r ;

[0046] When the UAV is in the constant speed cruising stage, the force model is as follows:

[0047]

[0048] The power model P is as follows:

[0049]

[0050] Where P cr，t Indicates the power corresponding to the long-distance flight speed, P cr，R Indicates the power corresponding to the long-distance speed. V t，max Indicates the long-distance flight speed, V R，max represents the long-distance speed, and P cr，t <P cr，R ;

[0051] When the UAV is in an unbalanced state but still maintains a cruising altitude, the UAV is in the process of acceleration or deceleration. The force model of the deceleration process is as follows:

[0052]

[0053] The power model P is as follows:

[0054] P=P cr =T a，cr ·V(t)

[0055] Where V(t) represents the speed of the UAV.

[0056] Preferably, when the drone is in the descent process, the descent angle and descent speed remain constant, and the force model of the drone is as follows:

[0057]

[0058] Among them, γ2 is the descent angle of the UAV, W is the gravity of the UAV, and L d Indicates the vertical lift during the descent, T a，d Denotes the thrust available during descent, D d Indicates the air resistance during descent;

[0059] The power model P is as follows:

[0060] P=P d =T a，d ·V d

[0061] Among them, P d Indicates the power during the descent process, V d Indicates the descent speed.

[0062] Preferably, when the UAV is in the leveling deceleration stage, the force model of the UAV is as follows:

[0063]

[0064] Where W is the gravity of the drone, L fd Indicates the lift force during the leveling and deceleration phase, F a，fd Denotes the resultant force of the leveling deceleration section, D fd Indicates the air resistance during the deceleration phase, T a，fd Indicates the thrust available during the leveling-off deceleration phase;

[0065] The power model P is as follows:

[0066] P=P fd =T a，fd ·V(t)

[0067] Where P fd represents the power in the leveling and deceleration phase, and V(t) represents the speed of the UAV;

[0068] When the UAV is in the landing phase, the force model of the UAV is as follows:

[0069]

[0070] Where, L l Indicates the lift during the landing phase, F a，2 represents the resultant force during the landing phase, D l represents the air resistance during the landing phase, T a，l Indicates the thrust available during the landing phase;

[0071] The power model P is as follows:

[0072] P=P l =T a，l·V(t)

[0073] Where P l Indicates the power during the landing phase;

[0074] When the drone is in the taxiing stage, the motor is in a stopped state, and the drone gradually stops due to ground friction and air resistance. At this time, there is no power consumption, and the force model is as follows:

[0075] F a，s =μW+D s

[0076] In the formula, F a，s represents the resultant force during the rolling phase, D s Indicates the air resistance during the rollout phase.

[0077] Preferably, the hydrogen fuel cell drone energy system includes a proton exchange membrane fuel cell and a battery;

[0078] The mathematical model of the proton exchange membrane fuel cell includes:

[0079]

[0080] In the formula, H FC is the hydrogen consumption rate, I FC is the operating current, n FC is the number of batteries, F is the Faraday constant;

[0081] The reaction rate of the proton exchange membrane fuel cell is represented by the current density j:

[0082]

[0083] In the formula, A cell is the reaction area of ​​the battery unit;

[0084] Thermal power output of proton exchange membrane fuel cells It is expressed as:

[0085]

[0086] Where η T is the fuel cell thermal efficiency, η E is the fuel cell electrical efficiency, represents the electrical power output of the fuel cell;

[0087] The battery operates in battery pack mode, and its total power is expressed as:

[0088]

[0089] In the formula, is the total power output of the battery pack, is the battery unit power, n1 and n2 are the number of battery rows and columns respectively;

[0090] The current of the battery pack I b as follows:

[0091]

[0092] Where V oc is the open-loop voltage, R b is the internal resistance of the battery pack;

[0093] The dynamic characteristic SOC of the battery pack can be expressed as:

[0094]

[0095] Among them, SOC t+1 and SOC t They represent the dynamic characteristics SOC of the battery pack at time t+1 and time t, respectively. t is the scheduling time, T is the scheduling period, and 0≤t≤T, Q B is the battery cell capacity, I b，t is the current at time t, when charging, I b，t When I is negative, b，t Is positive.

[0096] Preferably, the objective function is based on the operating cost of the hydrogen fuel cell drone energy system and energy storage device status Construct, the objective function J satisfies:

[0097]

[0098] Among them, C1 and are the equivalent hydrogen consumption and power at time t of the equipment in the hydrogen fuel cell drone energy system, i is 1 or 2, k1 and k2 are weight coefficients, SOC0 and SOC T The dynamic characteristics SOC of the battery pack at the initial and end times of the scheduling cycle, represents the fuel cell power output at time t.

[0099] Preferably, the operation constraints include electrical balance constraints, upper and lower power constraints of proton exchange membrane fuel cells, upper and lower battery state constraints, and battery initialization state constraints:

[0100]

[0101]

[0102] SOC min ≤SOC t ≤SOC max

[0103] SOC0=0.5·(SOC min +SOC max )

[0104] In the formula, They represent the output power of the lithium battery and the power consumption of the drone during flight, SOC min and SOC max They are the minimum and maximum values ​​of the dynamic characteristic SOC of the battery pack respectively.

[0105] Preferably, an MPC model is constructed by combining the force model and power model under each flight mode of the UAV, and the collected UAV state parameters and the electric power predicted by the MPC model are obtained, specifically including:

[0106] Collect UAV state parameters, including current flight state vector and control input vector;

[0107] According to the UAV force model and power model obtained in step 1, the state weight matrix, terminal state matrix, and input weight matrix in the MPC model are set: the state matrix is ​​composed of the flight speed, flight altitude, elevation angle, and engine temperature of the actual UAV during flight at each moment. The state weight matrix is ​​a symmetric positive definite matrix, which is set according to the importance of the state quantity to the optimization target. The terminal state matrix is ​​set as a positive definite matrix, and the input weight matrix is ​​set according to the relationship between the flight state and the control input. The number of prediction steps and the prediction interval need to be debugged multiple times to obtain the set values ​​with better effects; the calculation formula of the output electric power is based on the power calculation formula in the flight mode corresponding to step 1 at time t;

[0108] According to the current flight state vector and control input vector, the predicted flight state vector and control input vector are obtained based on the MPC model;

[0109] Obtaining predicted electrical power consumption based on the predicted flight state vector and the control input vector;

[0110] The function of the flight state vector, control input vector and electric power consumption P(t) is as follows:

[0111] P(t)=f(x(t),u(t))

[0112] Where x(t) is the flight state vector and u(t) is the control input vector.

[0113] Preferably, the mathematical model, objective function and operation constraints of the hydrogen fuel cell drone energy system to construct the DDPG model specifically include: defining the state space, action space and reward function as follows:

[0114] (1) State space S:

[0115]

[0116] Where t is the time step, is the electrical load at time t, i.e., the motor power, SOC t is the battery charge state at time t;

[0117] (2) Action space A:

[0118]

[0119] in, is the electric power of hydrogen fuel cell;

[0120] (3) Reward function R:

[0121]

[0122] in, It is defined as the scheduling action based on the policy network at time t, indicating the excess (greater than 0) or shortage (less than 0) when the electric power balance cannot be met. l1 is the weight adjusted according to training experience and effect.

[0123] Preferably, the training of the DDPG model specifically includes:

[0124] Initialize the DDPG neural network: According to the state space and action space, establish the policy network and evaluation network respectively. The number of layers of the policy network and evaluation network are set to 11 and 10 respectively. The policy network has 4 inputs and 1 output. The activation function of the output layer selects Sigmoid mapping to the interval 0-1; the evaluation network has 5 inputs and 1 output;

[0125] Set the parameters of the DDPG model, including training parameter settings: the learning rate α of the network P and α Q , target network update coefficient τ, scheduling period T, number of training rounds e max , discount factor γ, experience recovery area Memory, training batch size Batchsize;

[0126] OU noise parameter settings: including value regression rate θ, mean μ, volatility σ, maximum volatility σ max , minimum fluctuation σ min , noise attenuation period T Nd ;

[0127] Reward function parameter settings: coefficients α and l1;

[0128] Set up training scenarios: Before each training round, add a normal random fluctuation of no more than 10% to the mean of the load source data within the typical mission scenario and the feasible domain of safe operation of the equipment, and continuously generate new typical mission training scenarios until the end of the training;

[0129] Perform cyclic training: before each training, obtain the initial state S0 of the system; for each time step state s t , the policy network is based on P(s t |θ P ) Output action a t , the system gets reward r after executing the action t , and get the next state s t+1 ; training experience (s t , a t , r t ,s t+1 ) Save to the experience replay pool, update the current state, repeat MDP to the end time, and complete one round of training;

[0130] DDPG model parameter update: When the experience parameters of the experience replay pool meet the batch size, the network parameters will be updated at each time step to update the policy network and evaluation network, thus realizing a soft update of the DDPG model.

[0131] The present invention also proposes a multi-task PEM UAV energy scheduling system based on MPC and DDPG, which is used to implement the multi-task PEM UAV energy scheduling system method, including: a UAV model building module, an energy model building module, an MPC model building module, a DDPG model building module, a prediction module, and a training module;

[0132] The UAV model building module is used to build the force model and power model under each flight mode according to the flight mode of the hydrogen fuel cell UAV;

[0133] Energy model building module, used to establish the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system;

[0134] MPC model building module, which builds the MPC model based on the force model and power model of each flight mode of the UAV;

[0135] DDPG model building module, which builds the DDPG model based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system;

[0136] A prediction module is used to input the flight state parameters of the UAV into the MPC model to obtain the predicted electric power;

[0137] The training module is used to train the DDPG model by taking the predicted electric power as the input of the DDPG model, obtain the cumulative reward of the DDPG model, and obtain the energy scheduling result after the DDPG model is trained.

[0138] The present invention also provides a terminal, comprising a processor and a storage medium;

[0139] The storage medium is used to store instructions;

[0140] The processor is used to operate according to the instructions to execute the steps of the multi-task PEM UAV energy scheduling method based on MPC and DDPG.

[0141] The present invention also proposes a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the multi-task PEM UAV energy scheduling method based on MPC and DDPG are implemented.

[0142] The beneficial effect of the present invention is that, compared with the prior art, the present invention combines deep deterministic policy gradient DDPG with model predictive control MPC, which can fully utilize DDPG's ability to learn optimal strategies in high-dimensional continuous action space, and use MPC for short-term prediction and optimization. The present invention performs energy scheduling for multi-flight mission hydrogen fuel cell drones, uses model predictive control to derive its flight control parameters and optimized power consumption based on the drone's flight state, and then uses deep deterministic policy gradients to train the optimized power for energy scheduling, and obtains reasonable scheduling results. It can optimize energy utilization in complex mission environments, improve the drone's mission completion efficiency and endurance, and has broad application prospects. The beneficial effects of the present invention include at least:

[0143] (1) The multi-task PEM UAV energy scheduling method proposed in the present invention fully considers the operating status of the UAV system and the continuous equipment scheduling, and can fully reflect the dynamic operation process of the UAV;

[0144] (2) The present invention utilizes the MPC algorithm to achieve lower electric power for the multi-task hydrogen fuel cell drone, and utilizes the DDPG algorithm to achieve the lowest cost of electric energy scheduling.

[0145] (3) The present invention takes uncertainty into account for the optimized power. By adding 10% normal random fluctuations in the training scenario to simulate the uncertainty during the UAV flight process, the DDPG algorithm achieves higher real-time scheduling capabilities and system adaptability.

[0146] (4) The present invention uses the electric power optimized by the MPC algorithm to schedule the energy of the multi-task PEM UAV using the DDPG algorithm, thereby achieving optimization of the electric power side and efficient scheduling of electric power. However, the use of the MPC algorithm alone cannot solve the uncertainty of the system, and the use of the DDPG algorithm alone cannot optimize the electric power side. BRIEF DESCRIPTION OF THE DRAWINGS

[0147] Figure 1 It is a flow chart of the multi-task PEM UAV energy scheduling method based on MPC and DDPG in the present invention;

[0148] Figure 2 is a cross-sectional schematic diagram of the flight mode of the UAV in the present invention;

[0149] Figure 3 It is a power diagram of the drone in different flight modes of the present invention;

[0150] Figure 4 It is a flow chart of the UAV energy scheduling strategy based on MPC and DDPG in the present invention;

[0151] Figure 5 It is a schematic diagram of the structure of the DDPG strategy network and the evaluation network in the present invention;

[0152] Figure 6 is an example diagram of the mean and range of random working conditions in the present invention;

[0153] Figure 7 It is a comparison chart of the MPC predicted power and the reference value in the present invention.

[0154] Figure 8 It is a schematic diagram of the energy scheduling DDPG cumulative reward in three time periods in the simulation example of the present invention;

[0155] Fig. 9 It is a schematic diagram of the hydrogen fuel cell and SOC scheduling from 0 to 18 minutes in the simulation example of the present invention;

[0156] Fig.10 It is a schematic diagram of the hydrogen fuel cell and SOC scheduling from 19 to 36 minutes in the simulation example of the present invention;

[0157] Fig.11 It is a schematic diagram of the hydrogen fuel cell and SOC scheduling from 346 to 360 minutes in the simulation example of the present invention;

[0158] Fig.12 It is a structural diagram of the multi-task PEM UAV energy scheduling system based on MPC and DDPG in the present invention. DETAILED DESCRIPTION

[0159] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0160] like Figure 1 As shown, the present invention provides a multi-task PEM UAV energy scheduling method based on MPC and DDPG, the method comprising the following steps:

[0161] Step 1, analyzing the flight mode of the hydrogen fuel cell drone and establishing a force analysis and power model;

[0162] Drone flight modes Figure 2 As shown, the flight modes include take-off, climb, cruise, descent and landing; among them, climb includes steady straight climb, unsteady straight climb and unsteady non-straight climb; cruise includes level flight acceleration and constant speed cruise; landing includes leveling deceleration stage, landing stage and taxiing stage;

[0163] Further, such as Figure 3 As shown, for each flight mode of the hydrogen fuel cell drone, the corresponding force model and power model P under different flight modes are established respectively, wherein the force model includes vertical force and horizontal force, specifically:

[0164] 1. Takeoff

[0165] The take-off phase is when the drone accelerates from a stationary state to the take-off speed on the take-off runway and then leaves the ground, and continues to accelerate in the air to a safe height. At this stage, the drone can be considered to be in uniform acceleration motion. The horizontal force on the drone during the take-off phase is the static acceleration force P when the drone is gliding on the ground. a，tf :

[0166] Fa ， t f =T a，tf -D tf -μ(WL tf )=M·a tf

[0167] Among them, F a，tf is the static acceleration of the UAV when it is gliding on the ground, T a，tf is the thrust available during the operation of the UAV, D tf is the air resistance of the drone, W is the gravity of the drone, L tf is the lift force on the drone, M is the total mass of the drone, a tfis the acceleration, μ is the friction coefficient.

[0168] When the drone reaches the take-off speed, it leaves the ground and can be calculated based on the lift equation and the vertical balance equation:

[0169]

[0170] W=L tf

[0171]

[0172] Where ρ is the air density, V is the speed of the drone, S is the wing area, C L is the lift coefficient, V r is the take-off speed of the drone, that is, the speed off the ground.

[0173] During the take-off phase, the motor of the drone runs at maximum power. At this time, the power P of the drone satisfies:

[0174] P=P tf =P max

[0175] Among them, P max is the maximum power. tf Indicates take-off power;

[0176] Based on the above, the force model of the UAV during take-off is as follows:

[0177]

[0178] The power model P is as follows:

[0179] P=P tf =P max

[0180] 2. Climb

[0181] The climbing phase is the process of the drone rising from the ground to the cruising altitude at a certain climbing angle and climbing rate. During the climbing process, the forces on the drone are simplified into the vertical direction and the parallel direction (the aircraft climbing direction).

[0182] If the drone performs a steady straight climb, the forces in the two directions are balanced and the drone motor runs at constant power. The force model of the drone is as follows:

[0183]

[0184] Among them, γ1 is the flight climbing angle of the UAV.

[0185] L c Indicates lift during climbing.

[0186] T a，c Indicates the thrust available during a climb.

[0187] D c Indicates air resistance during climbing.

[0188] The power model P is:

[0189] P=P c，r

[0190] Where P cr It is the rated power of the UAV during climbing.

[0191] If the UAV performs an unsteady straight climb, the force model of the UAV is as follows:

[0192]

[0193] In the formula, F a，c It represents the resultant force in parallel direction during climbing.

[0194] The power model P is as follows:

[0195] P=P c =T a，c ·V(t)

[0196] Where P c Indicates the power during the climb.

[0197] V(t) represents velocity.

[0198] If the UAV performs an unsteady and non-linear climb, the force model of the UAV is as follows:

[0199]

[0200] The power model P is as follows:

[0201] P=P c =T a，c ·V(t)

[0202] In the formula, F a，c，y It represents the vertical resultant force during the climbing process.

[0203] F a，c，x It represents the resultant force in parallel direction during climbing.

[0204] 3. Cruise

[0205] The cruise phase mainly includes the level flight acceleration phase and the constant speed cruise phase:

[0206] When the UAV is in the level flight acceleration stage, the UAV climbs to the cruising altitude and then turns to level flight and continues to accelerate to reach the cruising speed. At this time, the force model of the UAV is as follows:

[0207]

[0208] Where, L cr Indicates lift during cruising.

[0209] F a，cr It represents the horizontal resultant force during cruising.

[0210] T a，cr Indicates the thrust available during cruise.

[0211] D cr Indicates air resistance during cruising.

[0212] The power model P is as follows:

[0213] P=P cr =T a，cr ·V(t)

[0214] Where P cr Indicates the power during cruising.

[0215] If the level flight acceleration phase is uniform acceleration, the power model P satisfies:

[0216] P=P cr，r (a cr )

[0217] Among them, a cr is the acceleration during the cruise acceleration phase, and a cr is a fixed value, and the corresponding power is a fixed value P cr，r , P cr，r The acceleration is a cr The power at

[0218] During the constant speed cruise stage, the drone reaches the cruising speed. There are two modes: the long-distance speed for the longest cruising distance and the long-distance speed for the longest cruising time. The long-distance speed refers to the flight speed at which the drone consumes the least fuel or energy per unit distance. This speed enables the aircraft to fly the farthest distance with a given amount of fuel or energy. The long-distance speed refers to the flight speed at which the drone consumes the least fuel or energy per unit distance. This speed enables the aircraft to stay in the air the longest time at a given altitude. The speed point corresponding to the maximum lift-to-drag ratio is usually also the long-distance speed. Since both modes are constant speed cruise modes, the forces in both modes are balanced.

[0219] When the UAV is in the constant speed cruising stage, the force model is as follows:

[0220]

[0221] The power model P is as follows:

[0222]

[0223] Among them, P cr，t Indicates the power corresponding to the long-distance flight speed, P cr，R Indicates the power corresponding to the long-distance speed. V t，max Indicates the long-distance flight speed, V R，max represents the long-distance speed and satisfies P cr，t <P cr，R .

[0224] In general, the cruising process does not change speed. However, in special cases, the drone will be in an unbalanced state but still needs to maintain the cruising altitude. At this time, the drone is in the process of acceleration or deceleration. The force model and power model of the acceleration process refer to the force model and power model of the drone in the acceleration stage of level flight.

[0225] The force model of the drone during deceleration is as follows:

[0226]

[0227] The power model P is as follows:

[0228] P=P cr =T a，cr ·V(t)

[0229] 4. Decline

[0230] When the drone is in the process of descending, the motor is in slow-speed mode and runs at the minimum thrust output. Due to gravity, the drone gradually descends. Under standard conditions, the descent angle and descent speed remain constant during the descent process. At this time, the force model of the drone is as follows:

[0231]

[0232] Among them, γ2 is the descent angle of the UAV flight,

[0233] L d Indicates the vertical lift during descent.

[0234] T a，d Indicates the thrust available during descent.

[0235] D d Represents the air resistance during descent.

[0236] The power model P is as follows:

[0237] P=P d =T a，d ·V d

[0238] Among them, P d Indicates the power during the descent process.

[0239] V d Indicates the descent speed.

[0240] 5. Landing

[0241] The landing phase mainly includes the descent phase from the cruising altitude, the leveling and deceleration phase, the landing phase, and the rolling phase. The descent phase has been discussed.

[0242] When the aircraft descends from the cruising altitude to the final altitude (buffer altitude), the UAV enters the leveling and deceleration phase. The force model of the UAV in the leveling and deceleration phase is as follows:

[0243]

[0244] Where, L fd Indicates the lift during the leveling and deceleration phase.

[0245] F a，fd It represents the resultant force of leveling the deceleration section.

[0246] D fd Indicates the air resistance in the leveling deceleration section.

[0247] T a，fd Indicates the thrust available in the leveling deceleration section.

[0248] The power model P is as follows:

[0249] P=P fd =T a，fd ·V(t)

[0250] Where P fd Indicates the power of the leveling deceleration section.

[0251] In the landing phase, the drone decelerates and descends from the buffer height to the ground. Since the buffer height is relatively small, it can be approximately regarded as a horizontal linear motion. When the drone is in the landing phase, the force model of the drone is as follows:

[0252]

[0253] Where, L l Represents the lift during landing phase.

[0254] F a，l Represents the landing phase resultant force.

[0255] D l Indicates the air resistance during landing.

[0256] T a，l Indicates the thrust available during landing phase.

[0257] The power model P is as follows:

[0258] P=P l =T a，l ·V(t)

[0259] Where P l Indicates the landing phase power.

[0260] In the taxiing phase, after the drone lands, the motor is in a shutdown state, and the drone gradually stops due to ground friction and air resistance. At this time, there is no power consumption, and the force model is:

[0261] F a，s =μW+D s

[0262] In the formula, F a，s Indicates the resultant force during the rolling phase.

[0263] D s Indicates the air resistance during the rolling phase.

[0264] Step 2, establish the mathematical model, objective function and constraints of the hydrogen fuel cell UAV energy system;

[0265] The main equipment of the hydrogen fuel cell UAV energy system includes proton exchange membrane fuel cells and batteries. The mathematical model of the hydrogen fuel cell UAV energy system constructed includes:

[0266] The calculation formula for hydrogen consumption of proton exchange membrane fuel cells is:

[0267]

[0268] Among them, H FC is the hydrogen consumption rate, I FC is the operating current, n FC is the number of batteries. F is the Faraday constant.

[0269] The reaction rate of the proton exchange membrane fuel cell is represented by the current density j:

[0270]

[0271] Among them, A cell is the reaction area of ​​the battery cell.

[0272] Thermal power output of proton exchange membrane fuel cells It is expressed as:

[0273]

[0274] Among them, η T is the fuel cell thermal efficiency, η E is the fuel cell electrical efficiency, Represents the electrical power output of the fuel cell.

[0275] The lithium battery in the present invention is a storage battery, which is usually operated in a battery pack mode. The total power of the battery pack can be expressed as:

[0276]

[0277] in, is the total power output of the battery pack, is the battery unit power, n1 and n2 are the number of battery rows and columns respectively.

[0278] The current of the battery cell I b as follows:

[0279]

[0280] Among them, the open-loop voltage V oc and the battery cell internal resistance R b can all be considered as fixed values.

[0281] The dynamic characteristic SOC of the battery pack can be expressed as:

[0282]

[0283] Among them, SOC t+1 and SOC t They represent the dynamic characteristics SOC of the battery pack at time t+1 and time t, respectively, t is the scheduling time, T is the scheduling period, and they satisfy (0≤t≤T), Q B is the battery capacity, I b，t is the current at time t, when charging, I b，t When I is negative, b，t Is positive.

[0284] Furthermore, the objective function J is based on the operating cost of the hydrogen fuel cell UAV energy system and energy storage device status Establish:

[0285]

[0286] Among them, C1 and are the equivalent hydrogen consumption and power at time t of the equipment in the hydrogen fuel cell drone energy system, k1 and k2 are weight coefficients, SOC0 and SOC TThe dynamic characteristics SOC of the battery pack at the initial and end times of the scheduling cycle, represents the fuel cell power output at time t.

[0287] Operation constraints include power balance constraints, upper and lower limits of proton exchange membrane fuel cell power, upper and lower limits of battery status, and battery initialization status constraints:

[0288] Electrical balance constraints:

[0289] The upper and lower limits of proton exchange membrane fuel cell power:

[0290] Battery state upper and lower limit constraints: SOC min ≤SOC t ≤SOC max

[0291] Battery initialization state constraint: SOC0 = 0.5 (SOC min +SOC max )

[0292] In the formula, They represent the output power of the lithium battery and the power consumption (electric load) of the drone during flight, SOC min and SOC max They are the minimum and maximum values ​​of the dynamic characteristic SOC of the battery pack respectively.

[0293] Step 3: Set the parameters of the MPC model based on the force model and power model of each flight mode of the UAV, collect the state parameters of the UAV, and obtain the predicted electric power through the MPC model, perform rolling optimization on the flight state parameters of the UAV and update the predicted electric power;

[0294] like Figure 4 As shown, the work of the MPC model includes model establishment, rolling optimization, and feedback correction. The MPC model includes 4 state vectors, 2 input vectors, and electric power as the output vector. The objective function is the minimum electric power. Specifically, step 3 includes:

[0295] Model Predictive Control (MPC) uses the mathematical model of the system to predict the future system behavior and find the optimal control input through the optimization algorithm, so that the system can achieve the optimal control performance under various constraints. The present invention uses MPC to predict and optimize the power consumption according to the flight state parameters of the drone. Step 3 specifically includes:

[0296] Step 3-1, the collected drone state parameters include the current flight state vector and control input vector;

[0297] The parameters in the flight state vector include: flight altitude h(t), flight speed v(t), pitch angle θ(t), engine temperature T E (t);

[0298] The control input vector includes the thrust command u throttle (t), elevation angle control u pitch (t).

[0299] Step 3-2, setting the parameters in the MPC model according to the UAV force model and power model obtained in step 1;

[0300] Specifically, the state weight matrix, the terminal state matrix, and the input weight matrix are set according to the force model and the power model in each flight mode in step 1.

[0301] The state matrix is ​​composed of the flight speed, flight altitude, elevation angle and engine temperature of the actual UAV flight process at each moment. The state weight matrix is ​​a symmetric positive definite matrix, which is set according to the importance of the state quantity to the optimization target. The terminal state matrix is ​​set as a positive definite matrix. The input weight matrix is ​​set according to the relationship between the flight state and the control input. The prediction step number and prediction interval need to be debugged multiple times to obtain the setting value with better effect. In addition, the calculation of output P(t) is based on the power calculation formula in the flight mode corresponding to step 1 at time t. ″

[0302] Step 3-3, according to the current flight state vector and the control input vector, obtain the predicted flight state vector and the control input vector based on the MPC model;

[0303] Step 3-4, obtaining predicted electric power consumption based on the predicted flight state vector and the control input vector;

[0304] The function of the flight state vector, control input vector and electric power consumption P(t) is as follows:

[0305] P(t)=f(x(t),u(t))

[0306] Where x(t) is the flight state vector and u(t) is the control input vector.

[0307] Step 3-5, constructing an objective function with minimum electric power consumption as the goal, and optimizing the control input according to the objective function to obtain the optimal control input at the current moment;

[0308] Taking the minimum electric power consumption as the objective function and meeting the requirements of the flight mission at the same time, the objective function f(t) is:

[0309]

[0310] Where P(t) is the electric power consumption, J(x(t), u(t)) is the cost function of the control objective function, λ is the trade-off parameter, N represents the number of prediction steps, and k represents the number of iteration steps.

[0311] Step 3-6, the optimal control input at the current moment is used for the actual control of the UAV, and the actual flight status of the UAV is updated, and the actual flight status of the UAV is fed back to the MPC model in real time to update the predicted electric power.

[0312] In MPC-based rolling optimization, only the optimal control input at the current moment is applied, and future control inputs will be recalculated in the next iteration.

[0313] The actual flight status of the aircraft will be fed back to the MPC in real time. The MPC will update the system model and prediction based on the new status information, thereby adjusting the control input at the next moment and calculating the predicted electric power based on the updated optimal control input.

[0314] The predicted electric power is used for DDPG algorithm training.

[0315] Step 4: Construct a DDPG model based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system. Combined with the electric power predicted in step 3, the constructed DDPG model is trained to obtain the cumulative reward of the DDPG model to determine whether the training is completed. The trained DDPG model is obtained, and the predicted electric power P is used as the input of the trained DDPG model to obtain the energy scheduling result.

[0316] Step 4-1, construct the DDPG model based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system;

[0317] The work of the DDPG model includes neural network initialization, algorithm parameter setting, training scenario setting, MDP training loop, and network parameter update. The DDPG model includes state space, action space, and reward function.

[0318] Explore Noise and Markov Decision Processes:

[0319] DDPG is a classic deep reinforcement learning algorithm that can be used to solve the continuous control problem of integrated energy systems. In order to better explore the unknown environment in the decision-making process, the Ornstein-Uhlenbeck (OU) noise N is added to the inertial system. OU And set the noise attenuation to achieve a balance between exploration and utilization. The specific mathematical expression is as follows:

[0320] dN t_oU =θ(μ-N t_OU )dt+σdW t

[0321] a t =P(s t |θ p )+N t_oU

[0322] Where t is the time step, N t_OU is the OU noise value, θ is the mean reversion rate, μ is the mean, σ is the volatility, W t is Brownian motion, a t is the action, P(s t |θ P ) is the actor network in DDPG, θ P is the network parameter, s t The system status.

[0323] The temporal correlation of the OU process enables the system to perform multi-step exploration in the same direction to accumulate training experience, thereby effectively improving the exploration efficiency of the inertial system.

[0324] Reinforcement learning tasks are usually described as Markov decision processes (MDPs), which are defined as five tuples: (S, A, P, R, γ), which are state, action, transition matrix, reward, and discount factor. Since the present invention adopts a model-free DDPG algorithm, there is no transition matrix in the constructed DDPG model.

[0325] The state space, action space and reward function of the DDPG model are constructed based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell drone energy system as follows:

[0326] (1) State space S:

[0327]

[0328] Where t is the time step, is the electrical load at time t, i.e., motor power, SOC t is the battery charge state at time t.

[0329] (2) Action space A:

[0330]

[0331] in, is the fuel cell power output at time t.

[0332] (3) Reward function R:

[0333]

[0334] in, It is defined as the scheduling action based on the policy network at time t, that is, the excess (greater than 0) or shortage (less than 0) when the power balance cannot be met. l1 is the weight adjusted according to training experience and effect.

[0335] Step 4-2, training the DDPG model in combination with the electric power predicted in step 3 to obtain the energy scheduling result after training;

[0336] Taking the predicted electric power P as the input of the DDPG model, the training process of the DDPG model is as follows:

[0337] Step 4-2-1, neural network initialization;

[0338] like Figure 5 As shown, according to the state space and action space, the strategy network (P network) and the evaluation network (Q network) are established respectively, as shown in Figure 4 As shown. Except for the input layer, the number of layers of the policy network and the evaluation network are 11 and 10 respectively. The policy network has 4 inputs (states) and 1 output (action). The activation function of the output layer selects Sigmoid mapping to the interval 0-1 to facilitate the scaling and restoration of actions; the evaluation network has 5 inputs (states + actions) and 1 output (reward).

[0339] Step 4-2-2, model parameter setting;

[0340] Training parameter setting: Set the learning rate α of the network P and α Q , target network update coefficient τ, scheduling period T, number of training rounds e max , discount factor γ, experience recycling area Memory, training batch size Batchsize.

[0341] OU noise parameter settings: value regression rate θ, mean μ, volatility σ, maximum volatility σ max , minimum fluctuation σ min , noise attenuation period T Nd .

[0342] Reward function parameter setting: setting coefficients α and l1

[0343] Step 4-2-3, training scene setting;

[0344] Before the start of each training round, within the typical mission scenario and the feasible domain of safe operation of the equipment, a normal random fluctuation of no more than 10% is added to the mean of the load source data, and new typical mission training scenarios are continuously generated until the end of the training.

[0345] Step 4-2-4, training cycle based on Markov decision process;

[0346] Before each training, the initial state S0 of the system is obtained; for each time step state s t , the policy network is based on P(s t |θ P ) Output action a t , the system gets reward r after executing the action t , and get the next state s t+1 ; training experience (s t , a t , r t ,s t+1 ) is saved to the experience replay pool, the current state is updated, and the MDP is repeated until the end moment to complete one round of training.

[0347] (5) Network parameter update

[0348] When the experience replay pool experience parameters meet the batch size, the network parameters will be updated at each time step. The policy network and evaluation network are updated, and the target network is soft-updated.

[0349] Step 4-3, judging whether the training is completed based on the cumulative reward of the DDPG model, and obtaining the trained DDPG model;

[0350] Specifically, the convergence of the training process is judged based on the cumulative rewards; if it converges within the set number of training rounds, it means that the training of the DDPG model is completed, otherwise the training is repeated.

[0351] Preferably, the number of training rounds is set to 200, and 200 sets of training scenarios need to be randomly generated. The examples of the mean and fluctuation range of electric power are as follows: Figure 6 The solid line is the load mean, and the shadow is the range in which the data can fluctuate.

[0352] Step 4-4, using the predicted electric power P as the input of the trained DDPG model to obtain the energy scheduling result.

[0353] The predicted electric power obtained in step 3 is used as the input of the trained DDPG model to obtain the energy scheduling result output by the DDPG model.

[0354] In order to verify the beneficial effects of the present invention, the following experiments were carried out:

[0355] For the working link of the MPC model, the experimentally measured drone data, including flight altitude, flight speed, elevation angle, and engine temperature, are collected as the flight state vector. Due to the large amount of data, only the range is shown in Table 1.

[0356] Table 1: Flight status range

[0357]

[0358] The above flight state is divided into five stages: take-off, climb, cruise, descent, and landing, and model predictive control is performed separately. The corresponding state matrix, input matrix, state weight matrix, terminal state weight matrix, input weight matrix, prediction step number, prediction interval, etc. are set respectively. After the MPC link, the output is electrical power. Figure 7 As shown, the obtained electric power is substantially lower than the reference value (measured value).

[0359] The power consumption predicted by the MPC model is used as input into the trained DDPG model for energy scheduling.

[0360] The main equipment parameters involved in energy scheduling are:

[0361] Hydrogen fuel cell: rated power 3kW, power range 0~3kW;

[0362] Lithium battery: battery cell capacity 20Ah, battery pack size 10×10, battery state 0.2~0.9, open loop voltage 4.2V;

[0363] The algorithm parameters of the DDPG model are shown in Table 2 below.

[0364] Table 2: Parameters of the DDPG model

[0365]

[0366] Since the system time step is 360, the time span is too long, and the UAV is in the cruising process from 19 to 345 minutes, the energy scheduling situation changes periodically. Therefore, three time periods are taken to explore the energy scheduling situation, namely 0-18 minutes, 19-36 minutes, and 346-360 minutes. Figure 8 , the energy scheduling DDPG cumulative rewards for the three time periods correspond to DDPG-1, DDPG-2, and DDPG-3 respectively. DDPG-M-1, DDPG-M-2, and DDPG-M-3 are the average cumulative rewards for every 5 rounds. The cumulative rewards for the three time periods all converge. Figures 9-11 As shown, the hydrogen fuel cell and SOC scheduling results in the three time periods are all within a reasonable range. It can be seen that the method proposed in the present invention can achieve efficient utilization of electric energy.

[0367] like Fig.12 As shown, the present invention also proposes a multi-task PEM UAV energy scheduling system based on MPC and DDPG, which is used to implement the above-mentioned multi-task PEM UAV energy scheduling system method, and the system includes: a UAV model building module, an energy model building module, an MPC model building module, a DDPG model building module, a prediction module, and a training module;

[0368] The UAV model building module is used to build the force model and power model under each flight mode according to the flight mode of the hydrogen fuel cell UAV;

[0369] Energy model building module, used to establish the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system;

[0370] MPC model building module, which builds the MPC model based on the force model and power model of each flight mode of the UAV;

[0371] DDPG model building module, which builds the DDPG model based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system;

[0372] A prediction module is used to input the flight state parameters of the UAV into the MPC model to obtain the predicted electric power;

[0373] The training module is used to train the DDPG model by taking the predicted electric power as the input of the DDPG model, obtain the cumulative reward of the DDPG model, and obtain the energy scheduling result after the DDPG model is trained.

[0374] The beneficial effect of the present invention is that compared with the prior art, the present invention utilizes model predictive control to derive the flight control parameters of the UAV and the optimized power consumption based on the flight state of the UAV, and then uses the deep deterministic policy gradient to train the energy scheduling of the optimized power to obtain a reasonable scheduling result. It can optimize energy utilization in complex mission environments, improve the mission completion efficiency and endurance of the UAV, and has broad application prospects.

[0375] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0376] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive lists) of computer readable storage medium include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disk read-only memories (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or convex structures in grooves on which instructions are stored, and any suitable combination thereof. Computer readable storage medium used here is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (e.g., light pulses by optical fiber cables), or electrical signals transmitted by wires.

[0377] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0378] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages-such as Smalltalk, C++, etc., and conventional procedural programming languages-such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network-including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0379] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A multi-task PEM UAV energy scheduling method based on MPC and DDPG, characterized in that: The steps include: According to the flight mode of hydrogen fuel cell drone, the force model and power model under each flight mode are established; Establish the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system; The MPC model is constructed by combining the force model and power model of each UAV flight mode, and the UAV state parameters are collected and the electric power predicted by the MPC model is obtained; The DDPG model is constructed based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system. The predicted electric power is used as the input of the trained DDPG model to obtain the energy scheduling result.

2. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 1 is characterized in that: The flight modes of hydrogen fuel cell drones include take-off, climb, cruise, descent, and landing; among them, climb includes steady straight climb, unsteady straight climb, and unsteady non-straight climb; cruise includes level flight acceleration and constant speed cruise; landing includes leveling deceleration stage, landing stage, and taxiing stage; For each flight mode of the hydrogen fuel cell drone, the corresponding force model and power model under different flight modes are established respectively, wherein the force model includes vertical force and horizontal force.

3. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 2 is characterized in that: When the UAV's flight mode is in the take-off stage, its force model is as follows: The power model P is as follows: P=P tf =P max In the formula, F a，tf is the static acceleration of the UAV when it is gliding on the ground, T a，tf is the thrust available during the operation of the UAV, D tf is the air resistance of the drone, W is the gravity of the drone, L tf is the lift force on the drone, M is the total mass of the drone, a tf is the acceleration of the drone when it takes off, P max is the maximum power of the UAV, P tf is the take-off power of the UAV, and μ represents the friction resistance coefficient.

4. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 2 is characterized in that: If the drone performs a steady straight climb, the forces in the two directions are balanced and the drone motor runs at constant power. The force model of the drone is as follows: Among them, γ is the flight climbing angle of the UAV, W is the gravity of the UAV, and L c Indicates the lift during the climb, T a，c represents the thrust available during the climb, D c Indicates the air resistance during the climb; The power model P is as follows: P=P c，r Among them, P c，r is the rated power of the UAV during climbing; If the UAV performs an unsteady straight climb, the force model of the UAV is as follows: In the formula, F a,c It represents the parallel direction resultant force during the climbing process; The power model P is as follows: P=P c =T a，c ·V(t) Where P c represents the power during the climb, and V(t) represents the speed of the UAV; If the UAV performs an unsteady and non-linear climb, the force model of the UAV is as follows: The power model P is as follows: P=P c =T a，c ·V(t) In the formula, F a，c，y Indicates the vertical force during the climbing process, F a,c，x Represents the parallel direction resultant force during the climbing process.

5. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 2 is characterized in that: When the UAV is in the level flight acceleration stage, the UAV climbs to the cruising altitude and then turns to level flight and continues to accelerate to reach the cruising speed. At this time, the force model of the UAV is as follows: The force model is as follows: Where W is the gravity of the drone, L cr Indicates the lift during cruising, F a，cr It represents the horizontal force during cruising, T a，cr Denotes the available thrust during cruise, D cr Indicates the air resistance during cruising; The power model P is as follows: P=P cr =T a，cr ·V(t) Where P cr Indicates the power during cruising; If the level flight acceleration phase is uniform acceleration, the power model P satisfies: P=P cr，r (a cr ) Among them, a cr The acceleration in the cruise acceleration phase is a constant value, and the power is a constant value P cr，r ; When the UAV is in the constant speed cruising stage, the force model is as follows: The power model P is as follows: Where P cr，t Indicates the power corresponding to the long-distance flight speed, P cr，R Indicates the power corresponding to the long-distance speed. V t，max Indicates the long-distance flight speed, V R，max represents the long-distance speed, and P cr，t <P cr，R ; When the UAV is in an unbalanced state but still maintains a cruising altitude, the UAV is in the process of acceleration or deceleration. The force model of the deceleration process is as follows: The power model P is as follows: P=P cr =T a，cr ·V(t) Where V(t) represents the speed of the UAV.

6. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 2 is characterized in that: When the drone is in the process of descent, the descent angle and descent speed remain constant. At this time, the force model of the drone is as follows: Among them, γ2 is the descent angle of the UAV, W is the gravity of the UAV, and L d Indicates the vertical lift during the descent, T a,d Denotes the thrust available during descent, D d Indicates the air resistance during descent; The power model P is as follows: P=P d =T a，d ·V d Among them, P d Indicates the power during the descent process, V d Indicates the descent speed.

7. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 2 is characterized in that: When the UAV is in the leveling and deceleration stage, the force model of the UAV is as follows: Where W is the gravity of the drone, L fd Indicates the lift force during the leveling and deceleration phase, F a，fd Denotes the resultant force of the leveling deceleration section, D fd Indicates the air resistance during the deceleration phase, T a，fd Indicates the thrust available during the leveling-off deceleration phase; The power model P is as follows: P=P fd =T a，fd ·V(t) Where P fd represents the power in the leveling and deceleration phase, and V(t) represents the speed of the UAV; When the UAV is in the landing phase, the force model of the UAV is as follows: Where, L l Indicates the lift during the landing phase, F a，l represents the resultant force during the landing phase, D l represents the air resistance during the landing phase, T a，l Indicates the thrust available during the landing phase; The power model P is as follows: P=P l =T a，l ·V(t) Where P l Indicates the power during the landing phase; When the drone is in the taxiing stage, the motor is in a stopped state, and the drone gradually stops due to ground friction and air resistance. At this time, there is no power consumption, and the force model is as follows: F a，s =μW+D s In the formula, F a,s represents the resultant force during the rolling phase, D s Indicates the air resistance during the rollout phase.

8. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 1 is characterized in that: The hydrogen fuel cell UAV energy system includes proton exchange membrane fuel cells and batteries; The mathematical model of the proton exchange membrane fuel cell includes: In the formula, H FC is the hydrogen consumption rate, I FC is the operating current, n FC is the number of batteries, F is the Faraday constant; The reaction rate of the proton exchange membrane fuel cell is represented by the current density j: In the formula, A cell is the reaction area of ​​the battery unit; Thermal power output of proton exchange membrane fuel cells It is expressed as: Where η T is the thermal efficiency of the fuel cell, η E is the fuel cell electrical efficiency, represents the electrical power output of the fuel cell; The battery operates in battery pack mode, and its total power is expressed as: In the formula, is the total power output of the battery pack, is the battery unit power, n1 and n2 are the number of battery rows and columns respectively; The battery current I b as follows: Where V oc is the open-loop voltage, R b is the internal resistance of the battery pack; The dynamic characteristic SOC of the battery pack can be expressed as: Among them, SOC t+1 and SOC t They represent the dynamic characteristics SOC of the battery pack at time t+1 and time t, respectively. t is the scheduling time, T is the scheduling period, and 0≤t≤T, Q B is the battery cell capacity, I b，t is the current at time t, when charging, I b，t When I is negative, b，t Is positive.

9. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 8 is characterized in that: The objective function is based on the operating cost of the hydrogen fuel cell UAV energy system and energy storage device status Construct, the objective function J satisfies: Among them, C1 and are the equivalent hydrogen consumption and power at time t of the equipment in the hydrogen fuel cell drone energy system, i is 1 or 2, k1 and k2 are weight coefficients, SOC0 and SOC T The dynamic characteristics SOC of the battery pack at the initial and end times of the scheduling cycle, represents the fuel cell power output at time t.

10. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 9 is characterized in that: The operation constraints include power balance constraints, upper and lower limits of proton exchange membrane fuel cell power, upper and lower limits of battery status, and battery initialization status constraints: SOC min ≤SOC t ≤SOC max SOC0=0.5·(SOC min +SOC max ) In the formula, They represent the output power of the lithium battery and the power consumption of the drone during flight, SOC min and SOC max They are the minimum and maximum values ​​of the dynamic characteristic SOC of the battery pack respectively.

11. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 1 is characterized in that: The MPC model is constructed by combining the force model and power model of the UAV in each flight mode. The collected UAV state parameters and the electric power predicted by the MPC model are specifically including: Collect UAV state parameters, including current flight state vector and control input vector; According to the UAV force model and power model obtained in step 1, the state weight matrix, terminal state matrix, and input weight matrix in the MPC model are set: the state matrix is ​​composed of the flight speed, flight altitude, elevation angle, and engine temperature of the actual UAV during flight at each moment. The state weight matrix is ​​a symmetric positive definite matrix, which is set according to the importance of the state quantity to the optimization target. The terminal state matrix is ​​set as a positive definite matrix, and the input weight matrix is ​​set according to the relationship between the flight state and the control input. The number of prediction steps and the prediction interval need to be debugged multiple times to obtain the set values ​​with better effects; the calculation formula of the output electric power is based on the power calculation formula in the flight mode corresponding to step 1 at time t; According to the current flight state vector and control input vector, the predicted flight state vector and control input vector are obtained based on the MPC model; Obtaining predicted electrical power consumption based on the predicted flight state vector and the control input vector; The function of the flight state vector, control input vector and electric power consumption P(t) is as follows: P(t)=f(x(t),u(t)) Where x(t) is the flight state vector and u(t) is the control input vector.

12. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 1 is characterized in that: The mathematical model, objective function and operation constraints of the hydrogen fuel cell drone energy system to build the DDPG model specifically include: defining the state space, action space and reward function as follows: (1) State space S: Where t is the time step, is the electrical load at time t, i.e., the motor power, SOC t is the battery charge state at time t; (2) Action space A: in, is the electric power of hydrogen fuel cell; (3) Reward function R: in, It is defined as the scheduling action based on the policy network at time t, indicating the excess (greater than 0) or shortage (less than 0) when the electric power balance cannot be met. t1 is the weight adjusted according to training experience and effect.

13. The multi-task PEM UAV energy scheduling method based on MPC and DDPG according to claim 12 is characterized in that: The training of the DDPG model specifically includes: Initialize the DDPG neural network: According to the state space and action space, establish the policy network and evaluation network respectively. The number of layers of the policy network and evaluation network are set to 11 and 10 respectively. The policy network has 4 inputs and 1 output. The activation function of the output layer selects Sigmoid mapping to the interval 0-1; the evaluation network has 5 inputs and 1 output; Set the parameters of the DDPG model, including training parameter settings: the learning rate α of the network P and α Q , target network update coefficient τ, scheduling period T, number of training rounds e max , discount factor γ, experience recovery area Memory, training batch size Batchsize; OU noise parameter settings: including value regression rate θ, mean μ, volatility σ, maximum volatility σ max , minimum fluctuation σ min , noise attenuation period T Nd ; Reward function parameter settings: coefficients α and l1; Set up training scenarios: Before each training round, add a normal random fluctuation of no more than 10% to the mean of the load source data within the typical mission scenario and the feasible domain of safe operation of the equipment, and continuously generate new typical mission training scenarios until the end of the training; Perform cyclic training: before each training, obtain the initial state S0 of the system; for each time step state s t , the policy network is based on P(s t |θ P ) Output action a t , the system gets reward r after executing the action t , and get the next state s t+1 ; training experience (s t , a t , r t ,s t+1 ) Save to the experience replay pool, update the current state, repeat MDP to the end time, and complete one round of training; DDPG model parameter update: When the experience parameters of the experience replay pool meet the batch size, the network parameters will be updated at each time step to update the policy network and evaluation network, thus realizing a soft update of the DDPG model.

14. A multi-task PEM UAV energy scheduling system based on MPC and DDPG, used to implement the multi-task PEM UAV energy scheduling system method described in any one of claims 1 to 13, characterized in that: include: UAV model building module, energy model building module, MPC model building module, DDPG model building module, prediction module, training module; The UAV model building module is used to build the force model and power model under each flight mode according to the flight mode of the hydrogen fuel cell UAV; Energy model building module, used to establish the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system; MPC model building module, which builds the MPC model based on the force model and power model of each flight mode of the UAV; DDPG model building module, which builds the DDPG model based on the mathematical model, objective function and operation constraints of the hydrogen fuel cell UAV energy system; A prediction module is used to input the flight state parameters of the UAV into the MPC model to obtain the predicted electric power; The training module is used to train the DDPG model by taking the predicted electric power as the input of the DDPG model, obtain the cumulative reward of the DDPG model, and obtain the energy scheduling result after the DDPG model is trained.

15. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images