A method and system for energy management of a multi-electric aircraft power supply system under pulse loads

Abstract

The present application belongs to the technical field of energy management of multi-electric aircraft power supply system, and relates to a kind of multi-electric aircraft power supply system energy management method and system under pulse load;The method comprises establishing power supply system dynamics equation and state constraint model;Energy management problem is modeled as Markov decision process, impulse characteristics such as high-power impact load power and load change rate are introduced in state space, and multi-objective cooperative reward function is designed;Adopt double-delay deep deterministic policy gradient algorithm, combine system historical operation data offline training energy management intelligent agent;Finally, the converged intelligent agent is deployed in online controller, and generator power command and battery charging and discharging power are output in real time.The present application can adapt to the random disturbance of pulse load, satisfy multiple operating constraints, realize real-time power optimization distribution under full task profile, and is suitable for energy management of multi-electric aircraft, hybrid power unmanned aerial vehicle and all-electric aviation platform.

Description

Technical Field

[0001] This invention belongs to the field of energy management technology for power supply systems of multi-electric aircraft, and specifically relates to an energy management method and system for power supply systems of multi-electric aircraft under pulsed load. Background Technology

[0002] Multi-electric aircraft power supply systems replace traditional hydraulic, pneumatic, and mechanical systems, providing unified power for flight control, environmental control, avionics, and high-power impact loads. Under complex missions, the system faces a dual challenge: pulse loads such as electromechanical actuators and impact loads exhibit instantaneous high power, peak-to-average power ratio, and strong randomness, leading to drastic power fluctuations; at the same time, it must meet multiple physical constraints such as generator limiting, battery charging and discharging constraints, state-of-charge boundaries, and power change rates, increasing the difficulty of energy management.

[0003] Existing methods fall into three categories: rule-based methods, such as state machines and fuzzy logic, offer fast responses but rely on expert experience, making them ill-suited to random disturbances and task switching, and lacking in global optimization; optimization-based methods, such as dynamic programming and model predictive control, can achieve theoretical optimality but rely on accurate models and prior information, resulting in high computational burden and limited real-time performance; and learning-based methods, such as deep reinforcement learning, often employ stable loads during training, failing to systematically incorporate impulse randomness, resulting in insufficient response to shocks, and their reward functions are biased towards a single objective, lacking a coordinated trade-off between multiple constraints, making it difficult to balance efficient operation under normal conditions with rapid impulse response.

[0004] In view of this, it is very necessary to provide an energy management method and system for a multi-electric aircraft power supply system under pulsed load to solve the above-mentioned defects in the prior art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies, such as difficulty in adapting to random disturbances under pulsed loads and the fact that reward functions often focus on a single objective and lack a coordinated trade-off for multiple operational constraints of the power supply system. This invention provides an energy management method and system for a multi-electric aircraft power supply system under pulsed loads to solve the aforementioned technical problems.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for energy management of a multi-electric aircraft power supply system under pulsed load includes the following steps: Step S1: Construct the dynamic equations and state constraint model of the power supply system for multi-electric aircraft; Step S2: The energy management problem of the power supply system of the multi-electric aircraft is modeled as a Markov decision process. A reinforcement learning simulation environment is constructed based on the dynamic equations and state constraint model of the power supply system of the multi-electric aircraft, and the state space, action space and reward function are defined. The state space is introduced with pulse load characteristics, and the reward function is a multi-objective collaborative optimization function. Step S3: Combining the historical operating data of the power supply system of the multi-electric aircraft, the energy management agent is trained using the dual-delay deep deterministic strategy gradient algorithm, so that the energy management agent learns the mapping from the power supply system state space to the energy management decision action space. Step S4: Deploy the trained and converged energy management agent on the online controller. In the actual flight application environment, collect the system status in real time. The energy management agent outputs the total output power of the generator and calculates the charging and discharging power of the battery to realize the power distribution under normal load and pulse load.

[0007] Preferably, the dynamic equations and state constraint models of the multi-electric aircraft power supply system in step S1 specifically include: The dynamic equations include: the dynamic equations of the state of charge of the battery; The state constraint model includes: generator model, load model, battery model, and power balance operation constraints; The generator model includes power output constraints, power change rate constraints, and optimal operating point constraints; the battery model includes charge / discharge power constraints and state-of-charge safety operation constraints; the load model includes conventional loads and pulse loads, wherein conventional loads include daily loads, navigation loads, communication loads, and radar loads, and pulse loads include electromechanical actuators and high-power impulse loads, wherein the launch time, power amplitude, and launch interval of the high-power impulse loads are random; the power balance operation constraint is specifically: during steady-state operation of the multi-electric aircraft power supply system, the sum of the generator output power and the battery charge / discharge power must be equal to the total load of the multi-electric aircraft at any given time.

[0008] This step can achieve the following technical effects: By constructing a state-constraint model that includes generators, batteries, and loads, and incorporating operational constraints such as power balance and rate of change, a physical boundary is provided for energy management strategies. This model can characterize the transient power response characteristics under pulsed load impacts, ensuring that subsequent reinforcement learning training is conducted within a realistic and feasible action space, avoiding invalid decisions that violate physical constraints, and forming the foundation for improving the practicality and safety of the strategy.

[0009] Preferably, in step S2, the energy management problem of the multi-electric aircraft power supply system is modeled as a Markov decision process, specifically including: S21, define the state space S, which represents the operating status of the power supply system at the decision moment; S22, define the action space A, which represents the decision variables output by the energy management agent; S23, Define the state transition probability P: , representing the state at the current time t Next action Then transition to the next state The probability of the state transition is calculated by recursively deriving the battery state of charge at the next moment based on the dynamic equation of the multi-electric aircraft power supply system, and the state constraint model performs legality verification and boundary constraints on the transition result. In the state constraint model, the power balance operation constraint is used to ensure that the algebraic sum of the generator output power and the battery charging and discharging power at any time must be equal to the total load of the multi-electric aircraft, so as to ensure the real-time power conservation of the system and avoid power mismatch leading to bus voltage collapse. The power output constraint in the generator model is used to limit the active power output of the generator to be between the lower limit and the upper limit of its rated capacity, in order to prevent the generator from burning out due to long-term overload or unstable operation due to low output. The power change rate constraint in the generator model is used to limit the increase or decrease of the generator's output per unit time to not exceed its maximum allowable ramp rate, in order to avoid the power sudden change caused by pulse load switching to cause mechanical shock and frequency fluctuation to the generator set. The charging and discharging power constraint of the battery is used to verify whether the calculated battery power is within the allowable range. If it exceeds the range, the transfer is rejected or a large negative reward is given to prevent power surges from damaging the energy storage equipment. The safe operation constraint of the battery's state of charge is used to verify whether the state of charge at the next moment after the transfer exceeds the overcharge or over-discharge threshold, to ensure the healthy state of the battery during long-term cyclic operation. If it exceeds the limit, the state transfer will be forcibly cut off. The optimal operating point constraint in the generator model is used to further restrict the optimal domain of the action space. Under the premise of satisfying all the above hard constraints, the generator output power is guided to the high-efficiency fuel economy range, so that the state transition process evolves in a more energy-efficient direction. S24, Define the reward function R: Evaluate the agent's state Next action The immediate advantages and disadvantages; S25, to maximize expected cumulative discount reward To optimize the objective, the energy management problem is transformed into a Markov decision process; among which, is the discount factor; T is the termination time of the interaction between the energy management agent and the environment.

[0010] This step can achieve the following technical effects: This approach transforms the energy management problem into a Markov decision process, utilizing state transition probabilities and reward signals to describe the temporal dynamics, enabling the agent to learn optimal strategies for multi-step decision-making. This modeling method adapts to the randomness of pulsed loads and the time-varying nature of task profiles, without relying on future prior information, providing a theoretical framework for real-time online control and enhancing the system's adaptability to uncertainty.

[0011] Preferably, the state space mentioned in step S21 specifically includes: task process information, system operating status, load characteristics, rate of change of state of charge, and rate of change of generator output power. The task progress information includes: the current time t and the remaining time. , used to characterize the impact of flight phases on energy distribution; The system operating status includes: battery state of charge. The current total output power of the generator and flight altitude This reflects the real-time adjustment capability of the power supply system; The load characteristics include: total load of the multi-electric aircraft. Load change rate The power values ​​of each sub-load; the power values ​​of the sub-loads include: electromechanical actuator loads. Power values, daily load Power values, navigation / communication / radar load Power value, high power impact load The power value; The rate of change of state of charge , used to characterize the charging and discharging trends of energy storage systems; The generator output power change rate , is used to characterize the rate of adjustment of the generator's power command.

[0012] This step can achieve the following technical effects: By incorporating pulse characteristics such as high-power impact load power and load change rate into the state space, and introducing the rate of change of state of charge and generator power change rate, the intelligent agent can perceive the amplitude, rate, and trend of power impacts in real time, and predict sudden changes in energy demand in advance. The design of the 13-dimensional continuous variable state space takes into account the task stage, system adjustment capability, and load composition, improving the agent's sensitivity to pulse disturbances and response accuracy, and making up for the shortcomings of traditional state settings in capturing dynamic information.

[0013] Preferably, the action space mentioned in step S22 specifically refers to: the total output power of the generator. The upper limit of the action space is determined by the power output constraint of the generator model, and the lower limit is 0.

[0014] This step can achieve the following technical effects: Defining the action space as a continuous scalar of the generator's total output power, with the action boundaries directly determined by the generator's power limit, reduces the decision-making dimensionality and simplifies the learning complexity. This design allows the agent's output commands to be directly mapped to actual controllable execution quantities. Action pruning ensures that commands always fall within a safe range, while also facilitating the calculation of battery charging and discharging power in conjunction with the power balance equation, achieving efficient integration between control commands and the physical system.

[0015] Preferably, the reward function in step S24 adopts a weighted penalty negative sum form, and its specific expression is as follows: , in, The reward function is represented by the following terms: the first term is the battery power penalty, the second term is the generator efficiency penalty, the third term is the SOC over-limit penalty, and the fourth term is the generator power change rate penalty. All penalty terms are normalized. Battery power; This is the maximum power of the battery; This represents the total power of the generator; This represents the maximum total power of the generator; This is the optimal total power point for the generator; The squared penalty function for out-of-bounds charging states; This represents the change in the total power output of the generator. This represents the maximum permissible rate of change of the generator's total power. These are the weighting coefficients. The penalty is set according to the priority of each penalty item: the higher the priority, the larger the corresponding weight coefficient, so as to guide the agent to prioritize the protection of key operational constraints. The charge state out-of-bounds squared penalty function The specific calculation method is as follows: , in, This is the minimum allowable state of charge for the safe operation of the battery pack; This represents the maximum permissible state of charge for the safe operation of the battery pack. The state of charge of the battery is obtained by recursively deriving the dynamic equation of the state of charge at each time step.

[0016] This step can achieve the following technical effects: The multi-objective collaborative reward function comprehensively balances battery power, generator efficiency, state-of-charge safety, and power change rate. It employs weighted penalties and a squared-out-of-bounds function to guide efficient operation under normal conditions while suppressing over-discharge risks and power fluctuations under pulse impacts. Each penalty is normalized, and its weight can be flexibly adjusted according to priority, enabling the agent to autonomously balance multiple constraint conflicts at different task stages. This achieves collaborative optimization that mitigates energy storage losses, improves fuel economy, enhances dynamic response capabilities, and extends equipment lifespan.

[0017] Preferably, step S3 specifically includes: S31, a network architecture for an energy management agent is constructed based on a dual-delay deep deterministic policy gradient algorithm; the network architecture includes: a set of action networks and two sets of evaluation networks; wherein, the set of action networks includes an online action network and a target action network, and the two sets of evaluation networks include two online evaluation networks and two target evaluation networks; An experience replay pool is constructed to store experience samples from each moment in the historical operation data of the power supply system. These experience samples include: states. ,action ,award and the state at the next moment ; S32, the energy management agent undergoes iterative training, and the current state... The input is to an online action network, which outputs a generator power command. Gaussian noise is added to the command and then clipped to a power limit range to obtain the total generator output power. Based on the total output power of the generator The battery charging and discharging power is calculated based on the power balance operation constraints, and the reward at the current moment is calculated according to the reward function. And obtain the state at the next moment. , to use empirical samples Store in the experience replay pool; S33, randomly sample experience samples in batches from the experience replay pool, and update the online evaluation network and online action network using the gradient descent method to minimize the temporal difference error; S34, a delayed update mechanism is used to update the online action network, making the update frequency of the action network lower than that of the evaluation network, so as to reduce the estimation error of the evaluation network before the policy is updated; the delayed update mechanism is specifically as follows: the online evaluation network is updated once every step, and the online action network is updated every... Update once per step; S35, the parameters of the target action network and the two target evaluation networks are updated using a soft update method to smooth the changes in the target value of the target network; S36. Repeat steps S32 to S35 until the cumulative reward of the energy management agent converges, and obtain a converged energy management agent model.

[0018] This step can achieve the following technical effects: To address the transient impact and strong randomness of pulsed loads, the dual-delay deep deterministic policy gradient algorithm effectively suppresses Q-value overestimation through its dual-evaluation network structure. This prevents the agent from outputting overly aggressive generator commands due to erroneous overestimation of the impulse response gains, ensuring decision-making safety. The delayed policy update mechanism optimizes the policy only after the evaluation error has been sufficiently reduced, suppressing oscillations and abrupt changes in power commands under impact transients. Combined with the smoothing of the target policy, the action output is more stable. This algorithm is particularly suitable for the stringent requirements of high reliability and low volatility in multi-electric aircraft power supply systems, ensuring rapid training convergence and policy robustness.

[0019] Preferably, step S4 specifically includes: S41, load and deploy the network structure and trained network parameters of the converged energy management agent to the airborne online controller; S42, During the actual flight of the aircraft, the online controller collects the operating data of the power supply system at the current moment in real time at a preset sampling period and inputs it into the energy management intelligent agent; S43, the energy management agent outputs the total output power of the generator. ; S44 sends the total generator output power to the generator controller for execution; the battery charging and discharging power is calculated based on power balance operation constraints. , in, This represents the charging and discharging power of the battery; a positive value indicates discharging, and a negative value indicates charging. Total load for multi-electric aircraft; For action Represents the total output power of the generator; The battery is driven to charge and discharge in order to meet the total load demand at the current moment. S45, repeat steps S42 to S44 in the next sampling period until the flight mission ends, to achieve real-time power distribution between conventional load and pulse load under the full mission profile.

[0020] This step can achieve the following technical effects: The lightweight deployment of a single agent model that has achieved convergence during training to the airborne controller requires only real-time acquisition of 13-dimensional state data to compute generator commands, meeting millisecond-level decision-making requirements. During online execution, it automatically adapts to the entire flight profile, maintaining the generator's high-efficiency zone during normal phases and rapidly increasing output and coordinating short-term battery compensation during pulse events, ensuring a consistently safe state of charge and limiting the rate of power change. This achieves plug-and-play real-time power optimization allocation across the entire mission profile.

[0021] Furthermore, the present invention also provides an energy management system for a multi-electric aircraft power supply system under pulsed load, comprising: The system modeling module contains: Construct the dynamic equations and state constraint model of the power supply system for multi-electric aircraft; The Markov decision process modeling module contains: The energy management problem of the power supply system of a multi-electric aircraft is modeled as a Markov decision process. A reinforcement learning simulation environment is constructed based on the dynamic equations and state constraint model of the power supply system of the multi-electric aircraft, and the state space, action space and reward function are defined. The state space is introduced with pulse load characteristics, and the reward function is a multi-objective collaborative optimization function. The agent training module contains: By combining historical operating data of the power supply system of multi-electric aircraft, a dual-delay deep deterministic strategy gradient algorithm is used to train the energy management agent, so that the energy management agent learns the mapping from the power supply system state space to the energy management decision action space. The online decision-making module includes: The trained and converged energy management agent is deployed on the online controller to collect system status in real time in actual flight application environments. The energy management agent outputs the total output power of the generator and calculates the charging and discharging power of the battery, realizing power distribution under normal load and pulse load.

[0022] The beneficial effects of this invention are as follows: by directly incorporating features such as pulse load power and load change rate into the reinforcement learning state space, the agent can perceive the arrival of power surges in real time and make smooth adjustments in advance, overcoming the problem of delayed response to pulse randomness in traditional methods; through the designed multi-objective collaborative reward function, energy storage loss, fuel economy, safety boundary and power stability are comprehensively balanced, and multiple constraints are coordinated in the form of weighted penalty and squared out-of-bounds function, eliminating the defects of safety and efficiency imbalance caused by the single reward in the prior art; in addition, the dual-delay deep deterministic policy gradient algorithm, combined with experience replay and delayed policy update, after being fully trained on offline historical data, only requires lightweight forward computation to make online decisions, which avoids the dependence of optimization methods on accurate prediction models and breaks through the limitation of traditional methods that are difficult to optimize globally, realizing adaptive energy management that takes into account fast response, multi-constraint satisfaction and real-time control under the full task profile.

[0023] Therefore, it is evident that the present invention has outstanding substantive features and significant progress compared with the prior art, and the beneficial effects of its implementation are also obvious. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the multi-electric aircraft power supply system architecture provided by the present invention.

[0026] Figure 2 This is a flowchart of an energy management method for a multi-electric aircraft power supply system under pulsed load, provided by the present invention.

[0027] Figure 3 This is a typical flight altitude profile and load diagram provided by the present invention.

[0028] Figure 4 This is a schematic diagram of the training architecture of the energy management agent based on the dual-delay deep deterministic policy gradient algorithm provided by the present invention.

[0029] Figure 5 This is a timing diagram of the interaction between the power supply system environment and the energy management intelligent agent provided by the present invention.

[0030] Figure 6 This is a schematic diagram of the energy management system for a multi-electric aircraft power supply system under pulsed load provided by the present invention.

[0031] Among them, 1-system modeling module, 2-Markov decision process modeling module, 3-agent training module, and 4-online decision-making module. Detailed Implementation

[0032] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following implementation methods.

[0033] Example 1: This embodiment provides, as follows: Figure 1 The power supply system for the multi-electric aircraft shown is specifically configured as follows: two generators, each with a rated power of 30kW, operate in parallel to form the main power generation unit; a battery pack with a rated power of 30kW and a capacity of 35 kWh serves as an energy storage unit for peak shaving, valley filling, and instantaneous power compensation; the generators and batteries are coupled through a DC bus to jointly supply power to the onboard loads.

[0034] like Figure 2 As shown in the figure, this embodiment provides an energy management method for a multi-electric aircraft power supply system under pulsed load, which includes the following steps: Step S1: Construct the dynamic equations and state constraint model of the power supply system for multi-electric aircraft; The dynamic equations and state constraint models of the multi-electric aircraft power supply system in step S1 specifically include: The dynamic equations include: the dynamic equations of the state of charge of the battery; The state constraint model includes: generator model, load model, battery model, and power balance operation constraints; The generator model includes: power output constraints. Power change rate constraint Optimal operating point constraint ;in, The output power of generator i, This represents the maximum output power of generator i; For adjacent time intervals, The maximum power change rate of generator i; Let be the optimal operating power of generator i. The optimal power coefficient is determined by the generator output power; when the generator output power deviates from the optimal operating point, the fuel consumption rate of the aircraft engine will increase. The battery model includes: charging and discharging power constraints. Constraints on safe operation under state of charge , ;in, The charging and discharging power of the battery pack. In this embodiment, the maximum charging and discharging power of the battery pack is set to 30kW; when When the battery pack is charging; when At that time, the battery pack discharges; The total capacity of the battery pack is 35kWh in this embodiment; and These represent the charging efficiency and discharging efficiency of the battery pack, respectively. In this embodiment, the difference between charging and discharging efficiencies is ignored, and we take... The initial value of SOC is set to... ; and In this embodiment, the minimum and maximum allowable State of Charge (SOC) for safe operation of the battery pack are taken as... , .

[0035] The load model includes conventional loads and pulsed loads. Conventional loads include daily operating loads, navigation loads, communication loads, and radar loads. Pulsed loads include electromechanical actuators and high-power impulse loads. The launch time, power amplitude, and launch interval of the high-power impulse loads are random. The load characteristics and fluctuation patterns of the power supply system of the multi-electric aircraft differ at different flight altitude profiles, such as... Figure 3 As shown; electromechanical actuator The power requirements of actuators used to drive flight control mechanisms such as control surfaces and landing gear are closely related to the intensity of flight maneuvers. During the cruise phase, the actuator power requirements are low and relatively stable; however, during climb, descent, and aerial maneuvers, the actuators operate frequently, and the power exhibits transient fluctuations. In this embodiment, the power range is set to 2-10kW, and the power variation has a certain degree of randomness; daily load This includes basic electrical equipment such as environmental control systems, lighting systems, and anti-icing systems. The power demand of this type of load is relatively stable, but is affected by flight altitude and ambient temperature. When the aircraft is cruising at high altitude, the demand from the environmental control system increases, leading to a rise in daily load power. Conversely, at low altitude or on the ground, the power decreases accordingly. In this embodiment, the power range for daily load is set at 5–10 kW. (Navigation / communication / radar loads) Depending on the intermittent operation of the flight mission mode, such as during cruise or aerial maneuvering, the radar power will increase significantly. In this embodiment, the power range for this type of load is set to 8–20 kW; high-power impact load. This type of load, which releases extremely high power in a short period of time, only occurs during in-flight maneuvers, and the launch timing, interval, and power magnitude are all random; this embodiment sets its power range to 25–40 kW; total load of the multi-electric aircraft. The sum of the power of the above four types of loads: ; The power balance operation constraint is specifically defined as follows: During steady-state operation of the multi-electric aircraft power supply system, the sum of the generator output power and the battery charging / discharging power must equal the total load of the multi-electric aircraft at any given time. ,in, The output power of the first generator at time t is expressed in kW, with positive values ​​indicating power supply to the bus. The output power of the second generator at time t is expressed in kW, with positive values ​​indicating power supply to the bus. This represents the charging and discharging power of the battery at time t, in kW. A positive value indicates discharging (supplying power to the bus), and a negative value indicates charging (absorbing power from the bus).

[0036] This step can achieve the following technical effects: By constructing a refined state constraint model that includes generators, batteries, and loads, and incorporating operational constraints such as power balance and rate of change, a physical boundary is provided for energy management strategies. This model can characterize the transient power response characteristics under pulsed load impacts, ensuring that subsequent reinforcement learning training is conducted within a realistic and feasible action space, avoiding invalid decisions that violate physical constraints, and forming the foundation for improving the practicality and safety of the strategy.

[0037] Step S2: The energy management problem of the power supply system of the multi-electric aircraft is modeled as a Markov decision process. A reinforcement learning simulation environment is constructed based on the dynamic equations and state constraint model of the power supply system of the multi-electric aircraft, and the state space, action space and reward function are defined. The state space is introduced with pulse load characteristics, and the reward function is a multi-objective collaborative optimization function. In step S2, the energy management problem of the multi-electric aircraft power supply system is modeled as a Markov decision process, specifically including: S21, define the state space S, which represents the operating status of the power supply system at the decision moment; S22, define the action space A, which represents the decision variables output by the energy management agent; S23, Define the state transition probability P: , representing the state at the current time t Next action Then transition to the next state The probability of the state transition is calculated by recursively deriving the battery state of charge at the next moment based on the dynamic equation of the multi-electric aircraft power supply system, and the state constraint model performs legality verification and boundary constraints on the transition result. S24, Define the reward function R: Evaluate the agent's state Next action The immediate advantages and disadvantages; S25, to maximize expected cumulative discount reward To optimize the objective, the energy management problem is transformed into a Markov decision process; among which, is the discount factor; T is the termination time of the interaction between the energy management agent and the environment.

[0038] This step can achieve the following technical effects: This approach transforms the energy management problem into a Markov decision process, utilizing state transition probabilities and reward signals to describe the temporal dynamics, enabling the agent to learn optimal strategies for multi-step decision-making. This modeling method adapts to the randomness of pulsed loads and the time-varying nature of task profiles, without relying on future prior information, providing a theoretical framework for real-time online control and enhancing the system's adaptability to uncertainty.

[0039] The state space mentioned in step S21 specifically includes: task process information, system operating status, load characteristics, rate of change of state of charge, and rate of change of generator output power. The task progress information includes: the current time t and the remaining time. , used to characterize the impact of flight phases on energy distribution; The system operating status includes: battery state of charge. The current total output power of the generator and flight altitude This reflects the real-time adjustment capability of the power supply system; The load characteristics include: total load of the multi-electric aircraft. Load change rate The power values ​​of each sub-load; the power values ​​of the sub-loads include: electromechanical actuator loads. Power values, daily load Power values, navigation / communication / radar load Power value, high power impact load The power value; The rate of change of state of charge , used to characterize the charging and discharging trends of energy storage systems; The generator output power change rate , used to characterize the adjustment rate of the generator's power command; The current state Composed of the above variables, the specific expression is as follows: .

[0040] This step can achieve the following technical effects: By incorporating pulse characteristics such as high-power impact load power and load change rate into the state space, and introducing the rate of change of state of charge and generator power change rate, the intelligent agent can perceive the amplitude, rate, and trend of power impacts in real time, and predict sudden changes in energy demand in advance. The 13-dimensional continuous state design takes into account the task stage, system adjustment capability, and load composition, improving the agent's sensitivity to pulse disturbances and response accuracy, and making up for the shortcomings of traditional state settings in capturing dynamic information.

[0041] In this embodiment, the action space mentioned in step S22 specifically refers to the total output power of the two generators. The operating space is a continuous interval [0, 60] kW, and the output power of each generator is distributed proportionally, i.e. .

[0042] This step can achieve the following technical effects: Defining the action space as a continuous scalar of the generator's total output power, with the action boundaries directly determined by the generator's power limit, reduces the decision-making dimensionality and simplifies the learning complexity. This design allows the agent's output commands to be directly mapped to actual controllable execution quantities. Action pruning ensures that commands always fall within a safe range, while also facilitating the calculation of battery charging and discharging power in conjunction with the power balance equation, achieving efficient integration between control commands and the physical system.

[0043] The reward function described in step S24 adopts a weighted penalty negative sum form, and its specific expression is as follows: , in, The reward function is represented by the following terms: the first term is the battery power penalty, the second term is the generator efficiency penalty, the third term is the SOC over-limit penalty, and the fourth term is the generator power change rate penalty. All penalty terms are normalized. Battery power; This is the maximum power of the battery; This refers to the total power of the generator; This represents the maximum total power of the generator; This is the optimal total power point for the generator; The squared penalty function for out-of-bounds charging states; This represents the change in the total power output of the generator. This represents the maximum permissible rate of change of the generator's total power. These are the weighting coefficients. The penalty is set according to the priority of each penalty item: the higher the priority, the larger the corresponding weight coefficient, so as to guide the agent to prioritize the protection of key operational constraints. The charge state out-of-bounds squared penalty function The specific calculation method is as follows: , in, This is the minimum allowable state of charge for the safe operation of the battery pack; This represents the maximum permissible state of charge for the safe operation of the battery pack. The state of charge of the battery is obtained by recursively deriving the dynamic equation of the state of charge at each time step.

[0044] This step can achieve the following technical effects: The multi-objective collaborative reward function comprehensively weighs battery power, generator efficiency, state-of-charge (SOC) safety, and power change rate. It employs a weighted penalty and a squared-out-of-bounds function to guide efficient operation under normal conditions while suppressing over-discharge risks and power fluctuations under pulse loads. The battery power penalty term guides the agent to prioritize generator power supply and reduce battery output under normal conditions, achieving peak shaving and valley filling. When a pulse load suddenly connects, this penalty term appropriately relaxes the restrictions on short-term high-power discharge, while controlling its cumulative effect through subsequent SOC penalties, thus achieving a dynamic balance between rapid power response and energy storage system protection. The SOC out-of-bounds penalty term uses a squared function instead of a conventional linear function, causing the penalty intensity to increase exponentially as the SOC approaches the safety boundary, thereby suppressing the over-discharge risk under repeated pulse load impacts. The squared penalty is zero within the safety range, preserving sufficient freedom for the agent to respond to instantaneous power demands. After normalization, the weights of various penalties can be flexibly adjusted according to priority, enabling the agent to autonomously balance multiple constraint conflicts at different task stages. This achieves synergistic optimization that reduces energy storage losses, improves fuel economy, enhances dynamic response capabilities, and extends equipment life.

[0045] Step S3: Combining historical operational data from the power supply system of the multi-electric aircraft, a dual-delay deep deterministic strategy gradient algorithm is used to train the energy management agent. The training architecture diagram of the energy management agent is shown below. Figure 4 As shown, the energy management agent learns the mapping from the power supply system state space to the energy management decision action space; In this embodiment, the historical operating data of the power supply system covers typical scenarios where conventional loads and pulse loads overlap. These typical scenarios include five flight phases: takeoff and climb, conventional cruise, aerial maneuvering, cruise return, and descent and landing. The randomness of the pulse load in the historical operating data is mainly reflected in three aspects: the randomness of the high-power impact load launch time, the randomness of the load power, and the randomness of electromechanical actuator fluctuations. Conventional loads also exhibit slight random fluctuations. In this embodiment, the total load peak value for all scenarios is within the preset power limit, and the peak demand can be covered by the combined power supply of the generator and battery. Step S3 specifically includes: S31, a network architecture for an energy management agent is constructed based on a dual-delay deep deterministic policy gradient algorithm; the network architecture includes: a set of action networks and two sets of evaluation networks; wherein, the set of action networks includes a parameter of Online action network and a parameter of The target action network, the two sets of evaluation networks include two online evaluation networks. and and two target evaluation networks and ; An experience replay pool is constructed to store experience samples from each moment in the historical operation data of the power supply system. These experience samples include: states. ,action ,award and the state at the next moment ; S32, the energy management agent undergoes iterative training, and the current state is... The input is to an online action network, which outputs a generator power command. Gaussian noise is added to the command and then clipped to within the generator power limit range to obtain the total generator output power. : , in, The deterministic action output by the policy network is the generator power command. The noise is Gaussian, with a larger amplitude at the beginning of training to encourage exploration, and gradually decreases as training progresses. and The clipping operation ensures that the action always remains within the legal range by setting the minimum and maximum values ​​for the action output. Based on the total output power of the generator The battery charging and discharging power is calculated based on the power balance operation constraints, and the reward at the current moment is calculated according to the reward function. And obtain the state at the next moment. , to use empirical samples Store in the experience replay pool; S33, Randomly sample experience samples in batches from the experience replay pool, and update the online evaluation network using the gradient descent method. and and parameters are Online action network; The online action network updates its parameters via gradient descent along the gradient direction of the action's Q-value from the first online evaluation network. Its loss function and parameter update formulas are as follows: , , in, Let the loss function be that of the online action network. This is for gradient calculation, where M is the number of samples in the batch. This represents the state value of the m-th sample taken from the experience replay pool. The first online evaluation network output Q-value for action gradient, For online action networks in state The deterministic action of the output, Output the gradient of the online action network with respect to its own parameters; The learning rate of the online action network is used to control the extent to which the model is updated based on the error function each time. Online evaluation networks optimize network parameters by minimizing a loss function. The loss function and parameter update formula for online evaluation networks are as follows: , , , , in, The loss function for online evaluation of the network; , The m-th sample action value and reward value are sampled from the experience replay pool; To evaluate the network's "label value" for the target; Discount factor; Evaluate network parameters for the target; For the target action network parameters; Gaussian noise is added to the output of the target action network to smooth the target policy. To evaluate the learning rate of the network; To perform the action The action value function corresponding to the "state-action pair" is output later; S34, a delayed update mechanism is used to update the online action network, making the update frequency of the action network lower than that of the evaluation network, so as to reduce the estimation error of the evaluation network before the policy is updated; the delayed update mechanism is as follows: the online evaluation network is updated once every step, and the online action network is updated every... Update once per step, usually taking =2; S35, the parameters of the target action network and the two target evaluation networks are updated using a soft update method to smooth changes in the target network's target value; the parameter update expressions for the target action network and the target evaluation network are as follows: , in, This is the soft update coefficient, typically ranging from 0.005 to 0.01. S36. Repeat steps S32 to S35 until the cumulative reward of the energy management agent converges, and obtain a converged energy management agent model.

[0046] This step can achieve the following technical effects: To address the transient impact and strong randomness of pulsed loads, the dual-delay deep deterministic policy gradient algorithm effectively suppresses Q-value overestimation through its dual-evaluation network structure. This prevents the agent from outputting overly aggressive generator commands due to erroneous overestimation of the impulse response gains, ensuring decision-making safety. The delayed policy update mechanism optimizes the policy only after the evaluation error has been sufficiently reduced, significantly suppressing oscillations and abrupt changes in power commands under impact transients. Combined with the smoothing of the target policy, the action output is more stable. This algorithm is particularly suitable for the stringent requirements of high reliability and low volatility in multi-electric aircraft power supply systems, ensuring rapid training convergence and policy robustness.

[0047] Step S4: Deploy the trained and converged energy management agent on the online controller. In the actual flight application environment, collect the system status in real time. The energy management agent outputs the total output power of the generator and calculates the charging and discharging power of the battery to realize the power distribution under normal load and pulse load. Step S4 specifically includes: S41, load and deploy the network structure and trained network parameters of the converged energy management agent to the airborne online controller; S42, as Figure 5 As shown, during the actual flight of the aircraft, the online controller collects the operating data of the power supply system at the current moment in real time at a preset sampling period and inputs it into the energy management intelligent agent; S43, the energy management agent outputs the total output power of the generator. ; S44 sends the total generator output power to the generator controller for execution; the battery charging and discharging power is calculated based on power balance operation constraints. , in, This represents the charging and discharging power of the battery; a positive value indicates discharging, and a negative value indicates charging. Total load for multi-electric aircraft; For action Represents the total output power of the generator; The battery is driven to charge and discharge in order to meet the total load demand at the current moment. S45, repeat steps S42 to S44 in the next sampling period until the flight mission ends, to achieve real-time power distribution between conventional load and pulse load under the full mission profile.

[0048] This step can achieve the following technical effects: The lightweight deployment of a single agent model that has achieved convergence during training to the airborne controller requires only real-time acquisition of 13-dimensional state data to compute generator commands, meeting millisecond-level decision-making requirements. During online execution, it automatically adapts to the entire flight profile, maintaining the generator's high-efficiency zone during normal phases and rapidly increasing output and coordinating short-term battery compensation during pulse events, ensuring a consistently safe state of charge and limiting the rate of power change. This achieves plug-and-play real-time power optimization allocation across the entire mission profile.

[0049] Example 2: like Figure 6 As shown in this embodiment, an energy management system for a multi-electric aircraft power supply system under pulsed load includes: System modeling module 1, in which: Construct the dynamic equations and state constraint model of the power supply system for multi-electric aircraft; Markov decision process modeling module 2, in which: The energy management problem of the power supply system of a multi-electric aircraft is modeled as a Markov decision process. A reinforcement learning simulation environment is constructed based on the dynamic equations and state constraint model of the power supply system of the multi-electric aircraft, and a state space, action space and reward function are constructed. The state space is introduced with pulse load characteristics, and the reward function is a multi-objective collaborative optimization function. Agent training module 3, in which: By combining historical operating data of the power supply system of multi-electric aircraft, a dual-delay deep deterministic strategy gradient algorithm is used to train the energy management agent, so that the energy management agent learns the mapping from the power supply system state space to the energy management decision action space. Online decision-making module 4, in which: The trained and converged energy management agent is deployed on the online controller to collect system status in real time in actual flight application environments. The energy management agent outputs the total output power of the generator and calculates the charging and discharging power of the battery, realizing power distribution under normal load and pulse load.

[0050] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The methods disclosed in the embodiments are described simply because they correspond to the systems disclosed in the embodiments; relevant details can be found in the method section.

[0051] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0052] In the embodiments provided by this invention, it should be understood that the disclosed systems, methods, and approaches can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0053] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0054] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit.

[0055] Similarly, in the various embodiments of the present invention, each processing unit can be integrated into a functional module, or each processing unit can exist physically, or two or more processing units can be integrated into a functional module.

[0056] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0057] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] The above-disclosed embodiments are merely preferred embodiments of the present invention, but the present invention is not limited thereto. Any non-creative variations that can be conceived by those skilled in the art, as well as any improvements and modifications made without departing from the principles of the present invention, should fall within the protection scope of the present invention.

Claims

1. An energy management method for a multi-electric aircraft power supply system under pulsed load, characterized in that, Includes the following steps: Step S1: Construct the dynamic equations and state constraint model of the power supply system for multi-electric aircraft; Step S2: The energy management problem of the power supply system of the multi-electric aircraft is modeled as a Markov decision process. A reinforcement learning simulation environment is constructed based on the dynamic equations and state constraint model of the power supply system of the multi-electric aircraft, and the state space, action space and reward function are defined. The state space is introduced with pulse load characteristics, and the reward function is a multi-objective collaborative optimization function. Step S3: Combine the historical operating data of the power supply system of the multi-electric aircraft and train the energy management agent using the dual-delay deep deterministic strategy gradient algorithm. Step S4: Deploy the trained and converged energy management agent on the online controller. In the actual flight application environment, collect the system status in real time. The energy management agent outputs the total output power of the generator and calculates the charging and discharging power of the battery to complete the power allocation between the normal load and the pulse load.

2. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 1, characterized in that, The dynamic equations and state constraint models of the multi-electric aircraft power supply system in step S1 specifically include: The dynamic equations include: the dynamic equations of the state of charge of the battery; The state constraint model includes: generator model, load model, battery model, and power balance operation constraints; The generator model includes power output constraints, power change rate constraints, and optimal operating point constraints. The load model includes conventional loads and pulsed loads. Conventional loads include daily loads, navigation loads, communication loads, and radar loads. Pulsed loads include electromechanical actuators and high-power impulse loads. The launch time, power amplitude, and launch interval of the high-power impulse loads are all random. The battery model includes charging and discharging power constraints and state-of-charge safety operation constraints. The power balance operation constraint specifically states that during steady-state operation of the multi-electric aircraft power supply system, the sum of the generator output power and the battery charging and discharging power must equal the total load of the multi-electric aircraft at any given time.

3. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 1, characterized in that, In step S2, the energy management problem of the multi-electric aircraft power supply system is modeled as a Markov decision process, specifically including: S21, define the state space S, which represents the operating status of the power supply system at the decision moment; S22, define the action space A, which represents the decision variables output by the energy management agent; S23, Define the state transition probability P: , representing the state at the current time t Next action Then transition to the next state The probability of the state transition is calculated by recursively deriving the battery state of charge at the next moment based on the dynamic equation of the multi-electric aircraft power supply system, and the state constraint model performs legality verification and boundary constraints on the transition result. S24, Define the reward function R: Evaluate the agent's state Next action The immediate advantages and disadvantages; S25, to maximize expected cumulative discount reward To optimize the objective, the energy management problem is transformed into a Markov decision process; among which, is the discount factor; T is the termination time of the interaction between the energy management agent and the environment.

4. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 3, characterized in that, The state space mentioned in step S21 specifically includes: task process information, system operating status, load characteristics, rate of change of state of charge, and rate of change of generator output power. The task progress information includes: the current time t and the remaining time. This characterizes the impact of the flight phase on energy distribution; The system operating status includes: battery state of charge. The current total output power of the generator and flight altitude This reflects the real-time adjustment capability of the power supply system; The load characteristics include: total load of the multi-electric aircraft. Load change rate The power values ​​of each sub-load; the power values ​​of the sub-loads include: electromechanical actuator loads. Power values, daily load Power values, navigation / communication / radar load Power value, high power impact load The power value; The rate of change of state of charge Characterizes the charging and discharging trend of the energy storage system; The generator output power change rate This characterizes the rate at which the generator's power command is adjusted.

5. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 3, characterized in that, The action space mentioned in step S22 specifically refers to: the total output power of the generator. The upper limit of the action space is determined by the power output constraint of the generator model, and the lower limit is 0.

6. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 3, characterized in that, The reward function described in step S24 adopts a weighted penalty negative sum form, and its specific expression is as follows: ， in, The reward function is represented by the following terms: the first term is the battery power penalty, the second term is the generator efficiency penalty, the third term is the SOC over-limit penalty, and the fourth term is the generator power change rate penalty. All penalty terms are normalized. Battery power; This is the maximum power of the battery; This represents the total power of the generator; This represents the maximum total power of the generator; This is the optimal total power point for the generator; The squared penalty function for out-of-bounds charging states; This represents the change in the total power output of the generator. This represents the maximum permissible rate of change of the generator's total power. These are the weighting coefficients. The penalty is set according to the priority of each penalty item: the higher the priority, the greater the corresponding weight coefficient.

7. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 6, characterized in that, The reward function includes the squared penalty function for out-of-bounds charge state. The specific calculation method is as follows: ， in, This is the minimum allowable state of charge for the safe operation of the battery pack; This represents the maximum permissible state of charge for the safe operation of the battery pack. The state of charge of the battery is obtained by recursively deriving the dynamic equation of the state of charge at each time step.

8. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 2, characterized in that, Step S3 specifically includes: S31, a network architecture for an energy management agent is constructed based on a dual-delay deep deterministic policy gradient algorithm; the network architecture includes: a set of action networks and two sets of evaluation networks; wherein, the set of action networks includes an online action network and a target action network, and the two sets of evaluation networks include two online evaluation networks and two target evaluation networks; An experience replay pool is constructed to store experience samples from each moment in the historical operation data of the power supply system. These experience samples include: states. ,action ,award and the state at the next moment ; S32, the energy management agent undergoes iterative training, and the current state... The input is to an online action network, which outputs a generator power command. Gaussian noise is added to the command and then clipped to a power limit range to obtain the total generator output power. Based on the total output power of the generator The battery charging and discharging power is calculated based on the power balance operation constraints, and the reward at the current moment is calculated according to the reward function. And obtain the state at the next moment. , to use empirical samples Store in the experience replay pool; S33, randomly sample experience samples in batches from the experience replay pool, and update the online evaluation network and online action network using the gradient descent method; S34, a delayed update mechanism is used to update the online action network, making the update frequency of the action network lower than that of the evaluation network; the delayed update mechanism specifically means that the online evaluation network is updated once per step, while the online action network is updated every [missing information - likely a time interval]. Update once per step; S35, the parameters of the target action network and the two target evaluation networks are updated using a soft update method; S36. Repeat steps S32 to S35 until the cumulative reward of the energy management agent converges, and obtain a converged energy management agent model.

9. The energy management method for a multi-electric aircraft power supply system under pulsed load according to claim 2, characterized in that, Step S4 specifically includes: S41, load and deploy the network structure and trained network parameters of the converged energy management agent to the airborne online controller; S42, During the actual flight of the aircraft, the online controller collects the operating data of the power supply system at the current moment in real time at a preset sampling period and inputs it into the energy management intelligent agent; S43, the energy management agent outputs the total output power of the generator. ; S44 sends the total output power of the generator to the generator controller for execution; calculates the battery charging and discharging power according to the power balance operation constraints, and drives the battery to charge and discharge. S45, repeat steps S42 to S44 in the next sampling cycle until the flight mission ends.

10. An energy management system for a multi-electric aircraft power supply system under pulsed load, characterized in that, include: The system modeling module contains: Construct the dynamic equations and state constraint model of the power supply system for multi-electric aircraft; The Markov decision process modeling module contains: The energy management problem of the power supply system of a multi-electric aircraft is modeled as a Markov decision process. A reinforcement learning simulation environment is constructed based on the dynamic equations and state constraint model of the power supply system of the multi-electric aircraft, and the state space, action space and reward function are defined. The state space is introduced with pulse load characteristics, and the reward function is a multi-objective collaborative optimization function. The agent training module contains: Based on historical operational data of the power supply system of multi-electric aircraft, an energy management agent is trained using a dual-delay deep deterministic strategy gradient algorithm. The energy management agent learns the mapping from the power supply system state space to the energy management decision action space. The online decision-making module includes: The trained and converged energy management agent is deployed on the online controller to collect system status in real time in actual flight application environments. The energy management agent outputs the total output power of the generator and calculates the charging and discharging power of the battery, completing the power allocation between normal load and pulse load.

Images