A multi-electric aircraft second-level energy scheduling method based on hybrid learning optimization

Abstract

The application discloses a kind of multi-electric aircraft second-level energy scheduling methods based on hybrid learning optimization, comprising: constructing the MIQP problem of multi-electric aircraft energy scheduling;The solving process of MIQP problem is decomposed into two stages: the first stage is converted into Markov decision process to the integer variable optimization problem;The feasible solution of integer variable is obtained by using deep reinforcement learning model to solve Markov decision process;The feasible solution of integer variable obtained by using in the second stage simplifies the MIQP problem constructed into QP problem;The optimal solution of continuous variable is obtained by using commercial solver to solve the QP problem;Based on the feasible solution of integer variable and the optimal solution of continuous variable solved, the energy scheduling result of multi-electric aircraft second-level is obtained.The application can significantly improve optimization speed and accuracy.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of electric power systems for multi-electric aircraft, deep learning, and reinforcement learning. Specifically, it provides a second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization. This method combines the real-time advantages of deep reinforcement learning with the optimality of solutions from modern commercial solvers. While ensuring the optimality of the solution, this method achieves an order-of-magnitude improvement in computation time. Background Technology

[0002] During the operation of multi-electric aircraft, the power demand from the load side, the output power of the generator, and the charging and discharging power of the energy storage system maintain the power balance of the electrical system. Therefore, energy scheduling and optimization of multi-electric aircraft during operation are crucial, as reasonable energy scheduling can significantly reduce energy consumption and lower aircraft operating costs. To this end, some researchers have proposed designing the multi-electric aircraft energy scheduling problem as a mixed-integer quadratic programming (MIQP) problem and using commercial solvers to obtain the system's power allocation. However, the increased complexity of the multi-electric aircraft electrical system, along with the increased safety constraints and objective requirements, makes modeling and computation more difficult during decision-making and control, introducing more uncertainty into real-time energy scheduling.

[0003] With the rapid development of deep learning, researchers have leveraged the powerful nonlinear representation capabilities of deep neural networks to integrate deep learning and reinforcement learning, forming the Deep Reinforcement Learning (DRL) algorithm, which outperforms most previous artificial intelligence algorithms and provides a solution for real-time energy scheduling scenarios. While the DRL method can achieve online energy scheduling requirements by using a trained neural network model in a test environment, it suffers from some unavoidable problems: firstly, it's difficult to guarantee that the calculation results for continuous variables in the problem are optimal; secondly, when the problem is large, model training consumes a significant amount of time.

[0004] To address the challenges of computational speed of commercial solvers and optimality and training time of existing learning algorithms in large-scale systems, this invention proposes a novel hybrid learning optimization method for the MIQP energy scheduling problem in multi-electric aircraft, combining the real-time nature of learning algorithms with the optimality of solver solutions. Summary of the Invention

[0005] The purpose of this invention is to provide a second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization, thereby improving optimization speed and accuracy.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization, the method comprising:

[0008] Step 1: Construct the MIQP problem for energy scheduling of multi-electric aircraft;

[0009] Step 2: Decompose the solution process of the MIQP problem into two stages;

[0010] Step 2.1, First Stage: Optimization of Integer Variables in the MIQP Problem;

[0011] Step 2.1.1: Transform the integer variable optimization problem into a Markov decision process;

[0012] Step 2.1.2: Solve the Markov decision process using a deep reinforcement learning model to obtain feasible solutions for integer variables;

[0013] Step 2.2, Second Stage: Optimization of continuous variables in the MIQP problem;

[0014] Step 2.2.1: Use the feasible solutions of integer variables obtained from the solution to simplify the constructed MIQP problem into a QP problem;

[0015] Step 2.2.2: Solve the QP problem using a commercial solver to obtain the optimal solution for continuous variables;

[0016] Step 3: Based on the feasible solutions for integer variables and the optimal solutions for continuous variables, the energy scheduling results for the multi-electric aircraft are obtained at the second level.

[0017] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0018] Preferably, the MIQP problem for constructing energy scheduling for multi-electric aircraft includes:

[0019] Step 1.1: Establish constraints:

[0020] 1.1) Busbar and load priority constraints;

[0021] First, there are load connection priority constraints, with non-removable loads having higher connection priority than removable loads; second, there are connection priority constraints between each generator and the main bus, and between the main bus and the secondary bus.

[0022] 1.2) Power balance constraints;

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029] In the formula, Let k be the output power of generator k at time t; Let q and i be the power capacities of the main bus and secondary bus, respectively, at time t. These represent the power transmitted between generator k and main bus q, and between main bus q and secondary bus i at time t, respectively. These are the transmission efficiencies between generator k and main bus q, and between main bus q and secondary bus i, respectively. These represent the charging and discharging power and total load demand of the energy storage system connected to the secondary bus i at time t, respectively. These are the load-removable and non-load-removable requirements connected to the secondary bus i, respectively. Let j1 and j2 be the connection relationships of the detachable loads j1 and j2 connected to the secondary bus i at time t, respectively. These are collections of generators, main busbars, and secondary busbars, respectively. These represent the sets of detachable and non-detachable loads connected to the secondary bus i, respectively.

[0030] 1.3) Busbar connection and generator state constraints;

[0031]

[0032]

[0033]

[0034] In the formula, ξ kq (t)∈{0,1} represents the connection state between generator k and main bus q at time t; The connection state between the main bus q and the secondary bus i at time t; Let t represent the operating state of generator k.

[0035] 1.4) Generator and bus power capacity constraints;

[0036]

[0037]

[0038]

[0039]

[0040]

[0041] In the formula, For the upper and lower limits of the output power of generator k; These are the upper limits of the power capacity of the main bus q and the secondary bus i, respectively. These are the upper limits of power transmission between generator k and main bus q, and between main bus q and secondary bus i, respectively.

[0042] 1.5) Generator optimal operating range constraints;

[0043]

[0044] In the formula, Let k be the optimal operating point of generator k at time t. The upper and lower limits of the optimal generator power range determined at time t based on the engine status;

[0045] 1.6) Constraints of energy storage systems;

[0046] At time t, the charging and discharging power of the energy storage system connected to the secondary bus i It is in charging state. The system is in a discharging state, but the charging and discharging processes cannot occur simultaneously. The change in SoC, ΔSoC, reflects the charging and discharging process of the energy storage system, and the relationship is as follows:

[0047]

[0048]

[0049] In the formula, This represents the change in SoC at time t. The total capacity of the energy storage system; τ represents the time interval; σ c σ disc As the efficiency factor for the charging and discharging process, the dynamic process of the SoC can be described as follows:

[0050]

[0051]

[0052] Equation (19) represents the SoC recovering to its initial value at the last moment during the optimization process, where T is the total time for MEA energy scheduling, and t init t is the start time of MEA energy dispatch.end This marks the end of the MEA energy dispatch process.

[0053] To avoid overcharging and discharging of the energy storage system, It needs to be limited to predetermined upper and lower limits, that is:

[0054]

[0055]

[0056] In the formula, These are the upper and lower limits of the charging and discharging power of the energy storage system connected to the secondary bus i. These are the upper and lower limits of the SoC value in the energy storage system.

[0057] Step 1.2: Establish the objective function:

[0058] 2.1) The cost function F1(t) that keeps the generator operating within its optimal range is expressed as:

[0059]

[0060] 2.2) The cost function F2(t) for ensuring that the generator, main bus, and secondary bus execute the pre-defined rules is as follows:

[0061]

[0062] In the formula, ω kq ω is the penalty factor for the connection between generator k and main bus q. qi The penalty factor for connecting the main bus q and the secondary bus i;

[0063] 2.3) The cost function F3(t) considering the lifespan of the battery in the energy storage system is expressed as:

[0064]

[0065] In the formula, The maximum rechargeable and rechargeable power of the energy storage system;

[0066] Step 1.3: Establish a multi-electric aircraft energy scheduling model:

[0067]

[0068] Equations (1) to (21)

[0069] In the formula, α, β, and γ are weighting coefficients.

[0070] Preferably, the transformation of the integer variable optimization problem into a Markov decision process includes:

[0071] Using the objective function F2(t) with integer variables and the established bus connection and generator state constraints, the action space a, state space s, and reward function r in the Markov decision process are obtained. The action space a is set as {connection strategy between generator and main bus, connection strategy between main bus and secondary bus, connection strategy between load shedding and secondary bus}, and the state space s is set as {current operating state of generator, current time}. The designed reward function r is as follows:

[0072] r(s t ,a t )=-(F2(t)+ρ(t)) (26)

[0073] In the formula, r(s) t ,a t ) represents the state s based on time t. t and action a t The reward value obtained is ρ(t)≥0, which is a penalty term. ρ(t)=0 when the constraint condition is met, otherwise ρ(t>0.

[0074] Preferably, the step of simplifying the constructed MIQP problem into a QP problem using feasible solutions of integer variables obtained from the solution includes:

[0075] Substituting the feasible solution of integer variables obtained from the solution into the energy scheduling model formula (25) for the multi-electric aircraft, the integer variable ξ in the constraint formulas (7) to (10) and formulas (13) and (14) is eliminated. kq (t), ζ qi (t), U G k(t) is used to simplify the MIQP problem into a QP problem.

[0076] This invention proposes a second-level energy scheduling method for multi-electric aircraft (MEA) based on hybrid learning optimization, addressing the MIQP problem. The MIQP problem is simplified to a QP problem using integer feasible solutions. For optimization of continuous variables, the conventional commercial solver Gurobi is employed. In complex MEA power system environments, the proposed method achieves orders-of-magnitude speedup compared to a single conventional commercial solver. Compared to the DRL algorithm, this method significantly reduces training time by not requiring training on continuous variables, and also achieves higher accuracy in solving the objective function. Attached Figure Description

[0077] Figure 1 This is a flowchart of the second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization according to the present invention;

[0078] Figure 2 This is a schematic diagram of the electric power system model of the multi-electric aircraft in the experiment of this invention;

[0079] Figure 3 This is a schematic diagram of the hybrid learning method framework used in the experiments of this invention;

[0080] Figure 4 This is a diagram of the Dueling network structure used in the experiments of this invention.

[0081] Figure 5 This is a schematic diagram of the D3QN algorithm training process in the experiment of this invention;

[0082] Figure 6 This is a schematic diagram showing the comparison results between the proposed method and Gurobi under different working conditions in the experiments of this invention. Detailed Implementation

[0083] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0084] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0085] To address the problem of optimizing energy scheduling for multi-electric aircraft in complex environments, thereby improving not only the solution speed but also ensuring the finding of an optimal solution, this embodiment provides a second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization.

[0086] like Figure 1 As shown in this embodiment, the second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization includes the following steps:

[0087] Step 1: Construct the MIQP problem for energy scheduling of multi-electric aircraft.

[0088] Using constraints such as bus and load priority constraints, power balance constraints, bus connection and generator state constraints, generator and bus power capacity constraints, generator optimal operating range constraints, and energy storage system constraints as conditions, and taking maximizing generator stability within the optimal operating range, executing predefined connection priority rules between generators and buses, and optimizing battery life in the energy storage system as cost functions, the energy scheduling problem for multi-electric aircraft is described as a multi-objective optimization MIQP problem.

[0089] Step 1.1: Establish constraints:

[0090] 1.1) Busbar and load priority constraints;

[0091] First, there are load connection priority constraints: non-removable loads always have a higher connection priority than removable loads. During flight, non-removable loads should not detach under any emergency, while the detachment of removable loads is related to whether the corresponding main power generator is malfunctioning. Second, there are connection priority constraints between each generator and the main bus, and between the main bus and the secondary bus.

[0092] 1.2) Power balance constraints;

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099] In the formula, Let k be the output power of generator k at time t; Let q be the power capacity of the main bus and i be the power capacity of the secondary bus at time t, respectively. These represent the power transmitted between generator k and main bus q at time t, and the power transmitted between main bus q and secondary bus i, respectively. These are the transmission efficiency between generator k and main bus q, and the transmission efficiency between main bus q and secondary bus i, respectively. These represent the charging and discharging power and total load demand of the energy storage system connected to the secondary bus i at time t, respectively. These are the load-removable and non-load-removable requirements connected to the secondary bus i, respectively. Let j1 and j2 be the connection relationships of the detachable loads j1 and j2 connected to the secondary bus i at time t, respectively. These are the generator set, the main bus set, and the secondary bus set, respectively. Let i represent the set of unloadable and non-unloadable loads connected to the secondary bus i, respectively.

[0100] 1.3) Busbar connection and generator state constraints;

[0101]

[0102]

[0103]

[0104] In the formula, ξ kq (t)∈{0,1} represents the connection state between generator k and main bus q at time t; The connection state between the main bus q and the secondary bus i at time t; Let t represent the operating state of generator k at time t.

[0105] 1.4) Generator and bus power capacity constraints;

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] In the formula, For the upper and lower limits of the output power of generator k; These are the upper limits of the power capacity of the main bus q and the secondary bus i, respectively. These represent the upper limit of power transmission between generator k and main bus q, and the upper limit of power transmission between main bus q and secondary bus i, respectively.

[0112] 1.5) Generator optimal operating range constraints;

[0113]

[0114] In the formula, Let k be the optimal operating point of generator k at time t. Let be the upper and lower limits of the optimal generator power range determined at time t based on the engine status.

[0115] 1.6) Constraints of energy storage systems;

[0116] At time t, the charging and discharging power of the energy storage system connected to the secondary bus i It is in charging state. The system is in a discharging state, but the charging and discharging processes cannot occur simultaneously. The change in SoC, ΔSoC, reflects the charging and discharging process of the energy storage system, and the relationship is as follows:

[0117]

[0118]

[0119] In the formula, This represents the change in SoC at time t. The total capacity of the energy storage system; τ represents the time interval; σ c σ disc As the efficiency factor for the charging and discharging process, the dynamic process of the SoC can be described as follows:

[0120]

[0121]

[0122] Equation (19) represents the SoC recovering to its initial value at the last moment during the optimization process, where T is the total time for MEA energy scheduling, and t init t is the start time of MEA energy dispatch. end This marks the end of MEA energy dispatch. The SoC value at time t+1 Let be the SoC value at time t. This represents the SoC value at the start of MEA energy dispatch. This represents the SoC value at the end of MEA energy scheduling.

[0123] To avoid overcharging and discharging of the energy storage system, It needs to be limited to predetermined upper and lower limits, that is:

[0124]

[0125]

[0126] In the formula, These are the upper and lower limits of the charging and discharging power of the energy storage system connected to the secondary bus i. These represent the upper and lower limits of the SoC value in the energy storage system.

[0127] Step 1.2: Establish the objective function:

[0128] 2.1) The cost function F1(t) that keeps the generator operating within its optimal range is expressed as:

[0129]

[0130] 2.2) The cost function F2(t) for ensuring that the generator, main bus, and secondary bus execute the pre-defined rules is as follows:

[0131]

[0132] In the formula, ω kq ω is the penalty factor for the connection between generator k and main bus q. qiThe penalty factor is the connection between the main bus q and the secondary bus i. The connection priority increases as the penalty factor decreases.

[0133] 2.3) The cost function F3(t) considering the lifespan of the battery in the energy storage system is expressed as:

[0134]

[0135] In the formula, The maximum rechargeable and rechargeable power of the energy storage system;

[0136] Step 1.3: Establish a multi-electric aircraft energy scheduling model:

[0137]

[0138] Equations (1) to (21)

[0139] In the formula, α, β, and γ are weighting coefficients.

[0140] Step 2: Decompose the solution process of the MIQP problem into two stages.

[0141] Step 2.1, First Stage: Optimization of Integer Variables in the MIQP Problem.

[0142] Step 2.1.1: Transform the integer variable optimization problem into a Markov decision process.

[0143] In the first stage, based on the properties of deep reinforcement learning, the optimization of integer variables is transformed into solving a Markov decision process, as follows:

[0144] Using the objective function F2(t) with integer variables and the established bus connection and generator state constraints, the action space a, state space s, and reward function r in the Markov decision process are obtained. The action space a is set as {connection strategy between generator and main bus, connection strategy between main bus and secondary bus, connection strategy between load shedding and secondary bus}, and the state space s is set as {current operating state of generator, current time}. The reward function r is related to the objective function F2(t) with integer variables. In order to ensure that the solutions with integer variables all satisfy the constraints, a penalty term ρ(t) is added to the reward function. The designed reward function r is as follows:

[0145] r(s t ,a t )=-(F2(t)+ρ(t)) (26)

[0146] In the formula, r(s) t ,a t Let s be the state space based on time t. t and action space at The reward value obtained is ρ(t)≥0, which is a penalty term. ρ(t)=0 when the constraint condition is met, otherwise ρ(t>0.

[0147] The first part of the reward function is a function of the connection relationship between the generator, main bus, and secondary bus. The second part, ρ(t)≥0, is a penalty term. When the constraint condition of integer variables is met, ρ(t)=0; otherwise, ρ(t) will be assigned a sufficiently large constant.

[0148] Step 2.1.2: Solve the Markov decision process using a deep reinforcement learning model to obtain feasible solutions for integer variables.

[0149] This embodiment employs the Dueling DQN framework, constructing a competitive double deep Q-network (D3QN) algorithm on top of Double DQN, and using this method to obtain feasible solutions for all integer variables. In one embodiment, the specific solution process is as follows:

[0150] (1) Initialize the estimated network parameters θ and the target network parameters θ', and assign the estimated network parameters to the target network. Set the exploration rate, learning rate, discount factor, experience replay pool size, number of samples for each random sampling, target network parameter update frequency, and number of training rounds. Here, the estimated network and the target network adopt the same neural network structure, where the number of neurons in the input layer of the neural network is equal to the dimension of the state space, there are 3 hidden layers in the middle, and then the hidden layers of the parallel state value function network V and the hidden layers of the advantage function network A are connected to the output layer. The number of neurons in the output layer is equal to the dimension of the action space.

[0151] (2) Observe the state s of the multi-electric aircraft system at the current time t. t The input is fed into the estimation network, the Q-value for each action is calculated, and the current state s is selected using the ε-greedy method. t The corresponding action a t The ε-greedy approach is used here to balance the exploration and exploitation of actions in DRL.

[0152] (3) Execute action a t In the multi-electric aircraft system, control the connection status of the generator, bus, and removable load, according to action a. t Obtain new state s t+1 and reward r t That is, r(s) t ,a t ).

[0153] (4) Transfer the data {s} t ,a t ,r t ,s t+1Store it in the experience replay pool and update the current state of the multi-electric aircraft system.

[0154] (5) Draw a small batch of samples {s} from the experience replay pool. t ,a t ,r t ,s t+1 Train the neural network, where i = 1, 2, ..., M.

[0155] (6) Calculate the estimated network state value function V(s) i The network's output and action advantage function A(s) i ,a i The network outputs are then coupled to the Q-value function for each action, where the action advantage function network output is the current action advantage function value minus the mean of all action advantage functions, i.e.:

[0156]

[0157] Here, A represents the set containing all executable actions, |A| is the number of all actions, and a′ t For state s i The actions that may be taken at that time.

[0158] (7) Calculate the Q value of the target network:

[0159]

[0160] Here, μ is the discount factor. To select action a in the estimation network computation i , state s i+1 The maximum Q value at that time;

[0161] (8) Using the mean squared loss function Calculate the loss and backpropagate to update the estimated network parameters θ;

[0162] (9) Every so often, the estimated network parameters θ are copied to update the target network parameters θ';

[0163] (10) If training is not complete, repeat steps (2)-(9) above. Save the estimated network parameters until offline training is completed. During online learning, use the trained model to determine feasible solutions for all integer variables in the multi-electric aircraft energy scheduling problem.

[0164] Step 2.2, Second Stage: Optimization of continuous variables in the MIQP problem.

[0165] Step 2.2.1: Use the feasible solutions of the integer variables obtained from the solution to simplify the constructed MIQP problem into a QP problem.

[0166] Based on all feasible integer solutions provided by the D3QN algorithm in the first stage, the cost function and constraints containing integer variables in the energy scheduling problem of multi-electric aircraft can be eliminated, simplifying the energy scheduling problem of MIQP into a typical quadratic programming (QP) problem. This problem can be solved quickly using modern commercial solvers for continuous variables. Specifically, the feasible solutions with integer variables obtained by solving are substituted into the multi-electric aircraft energy scheduling model formula (25), and the integer variable ξ in the constraint formulas (7) to (10) and formulas (13) and (14) is eliminated. kq (t), This simplifies the MIQP problem into a QP problem (quadratic programming problem).

[0167] Step 2.2.2: Solve the QP problem using a commercial solver to obtain the optimal solution for the continuous variables. In this embodiment, an existing commercial solver, such as Gurobi, is used.

[0168] Step 3: Based on the feasible solutions for integer variables and the optimal solutions for continuous variables, the energy scheduling results for the multi-electric aircraft are obtained at the second level.

[0169] To further demonstrate the advantages of the accelerated optimization method proposed in this application, a specific experiment is provided below.

[0170] like Figure 2 As shown, the experimental multi-electric aircraft system includes two main generators and one APU, two AC main buses, and three secondary buses, including two DC buses and one AC bus. Each secondary bus has five offloadable loads and five non-offloadable loads, and each DC bus has an energy storage system installed. Table 1 summarizes the system parameter settings in detail. The penalty factor for the connection between the generators, main buses, and secondary buses is:

[0171]

[0172] Table 1 System Parameters

[0173]

[0174] Table The optimal operating points for generators 1, 2, and 3 are determined.

[0175] The connection priority between the generator and the bus is shown in Table 2, and the priority for unloading loads is shown in Table 3.

[0176] Table 2. Connection Priority between Generator and Busbar

[0177]

[0178] Table 3 Priority for Removable Loads

[0179]

[0180] This experiment decomposes the energy scheduling problem of multi-electric aircraft involving continuous and integer variables into two stages for solution, such as... Figure 3 This forms the basic framework for hybrid learning methods.

[0181] Settings such as Figure 4 The Dueling DQN framework shown builds a competitive dual deep Q-network (D3QN) algorithm on top of Double DQN and uses this method to obtain feasible solutions for all integer variables. The estimated network parameters θ and the target network parameters θ' are initialized, and the estimated network parameters are assigned to the target network. The initial exploration constant is set to 1 and decreases to 0.05 with iterations. The learning rate is 0.01, the discount factor is 0.9, the experience replay pool capacity is 500, the random sampling batch size is 32, the target network parameter update frequency is 300, the training epochs are 600, and each epoch has 120 time steps. The estimated and target networks use the same neural network structure, where the number of neurons in the input layer is equal to the dimension of the state space, there are three hidden layers connected to the Dueling network structure to the output layer, with the hidden layers containing 200, 100, and 50 neurons respectively, and the number of neurons in the output layer equal to the dimension of the action space.

[0182] The training process of the D3QN algorithm is as follows: Figure 5 As shown, the algorithm begins to converge gradually when the number of training rounds reaches 218, and the final reward will stabilize at -7.23. Figure 6 (a, b, c) show the comparison results between the proposed method and Gurobi under different operating conditions. It is clear that the energy scheduling results obtained by the proposed method are completely consistent with those of Gurobi, indicating that the proposed model not only has good adaptability in different environments, but also can obtain optimal solutions for both integer and continuous variables. The performance comparison of the proposed algorithms is shown in Table 4.

[0183] Table 4 Comparison of Algorithm Performance

[0184] D3QN 34842.57s 0.07s 34.72 Gurobi — 127.63s 21.35 D3QN+Gurobi 465.16s 0.12s 21.35

[0185] As can be seen from Table 4, the D3QN+Gurobi method proposed in this application can achieve an order-of-magnitude improvement in computation speed while ensuring the optimal solution compared to the existing D3QN and Gurobi methods.

[0186] This embodiment utilizes the Dueling DQN framework for integer variable optimization, constructing a competitive double-depth Q-network (D3QN) algorithm on top of Double DQN. It combines the features of Double DQN and Dueling DQN, solving the Q-value overestimation problem while obtaining a more accurate Q-value. For the design of the reward function, the cost function of the connection relationship between the generator, main bus, and secondary bus is considered as part of it. Another part introduces a penalty term, which is indirectly related to the integer variable constraints (7)-(9). If the obtained action space a t If the above constraints are not met, and to prevent the agent from tolerating behaviors that do not meet the constraints, the penalty term is set to a sufficiently large constant. Conversely, if all action spaces a t If all constraints are met, the penalty term is set to zero.

[0187] Due to the unique reward function design of this invention, the method can obtain feasible solutions for all integer variables during model testing under different main generator operating conditions. The obtained integer feasible solutions simplify the MIQP problem into a QP problem. Then, for optimization of continuous variables, the conventional commercial solver Gurobi is used. In the complex MEA power system environment, the proposed method achieves an order-of-magnitude speedup compared to a single conventional commercial solver. Compared to the DRL algorithm, this method significantly reduces training time because it does not require training continuous variables, and the accuracy of the solved objective function value is higher.

[0188] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0189] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization, characterized in that, The second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization includes: Step 1: Construct the MIQP problem for energy scheduling of multi-electric aircraft; Step 2: Decompose the solution process of the MIQP problem into two stages; Step 2.1, First Stage: Optimization of Integer Variables in the MIQP Problem; Step 2.1.1: Transform the integer variable optimization problem into a Markov decision process; Step 2.1.2: Use the D3QN deep reinforcement learning model to solve the Markov decision process and obtain feasible solutions for integer variables; Step 2.2, Second Stage: Optimization of continuous variables in the MIQP problem; Step 2.2.1: Use the feasible solutions of integer variables obtained from the solution to simplify the constructed MIQP problem into a QP problem; Step 2.2.2: Solve the QP problem using a commercial solver to obtain the optimal solution for continuous variables; Step 3: Based on the feasible solutions for integer variables and the optimal solutions for continuous variables, the energy scheduling results for the multi-electric aircraft are obtained at the second level.

2. The second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization as described in claim 1, characterized in that, The MIQP problem for constructing energy scheduling for multi-electric aircraft includes: Step 1.1: Establish constraints: 1.1) Bus and load priority constraints; First, there are load connection priority constraints, with non-removable loads having higher connection priority than removable loads; second, there are connection priority constraints between each generator and the main bus, and between the main bus and the secondary bus. 1.2) Power balance constraints; （1） （2） （3） （4） （5） （6） In the formula, for Time generator ; output power; , They are respectively Main bus and secondary bus Power capacity; , They are respectively Time generator With the main bus Main busbar With secondary bus Transmission power between; , Generators With the main bus Main busbar With secondary bus Transmission efficiency between them; , They are respectively Timing and Secondary Bus The charging and discharging power and total load requirements of the connected energy storage system; , They are respectively with the secondary bus Connected offloadable and non-offloadable load requirements; , They are respectively Timing and Secondary Bus Connected detachable load and non-removable load The connection relationship, These are collections of generators, main busbars, and secondary busbars, respectively. They represent the secondary busbar respectively. A collection of connected offloadable and non-offloadable loads; 1.3) Busbar connection and generator state constraints; （7） （8） （9） In the formula, for Time generator With the main bus The connection status; for Main bus With secondary bus The connection status; for Time generator The running status; 1.4) Generator and bus power capacity constraints; （10） （11） （12） （13） （14） In the formula, , For generator The upper and lower limits of the output power; , Main busbar and secondary bus The upper limit of power capacity; , Generators With the main bus Main busbar With secondary bus The upper limit of transmission power between them; 1.5) Generator optimal operating range constraints; （15） In the formula, For a moment dynamo The optimal operating point; , In order to be in The upper and lower limits of the optimal generator power range determined at all times based on the engine status; 1.6) Constraints of energy storage systems; exist At that time, with the secondary bus The charging and discharging power of the connected energy storage system It is in charging state. It is in a discharging state, but the charging and discharging processes cannot occur simultaneously. Change This reflects the charging and discharging process of the energy storage system, and the relationships are as follows: （16） （17） In the formula, express time The change This refers to the total capacity of the energy storage system; Indicates a time interval; , This is the efficiency factor for the charging and discharging process. The dynamic process can be described as follows: （18） （19） Equation (19) represents the final time in the optimization process. Restore to the initial value. The total time for MEA energy scheduling. This marks the start of MEA energy dispatch. This marks the end of the MEA energy dispatch process. To avoid overcharging and discharging of the energy storage system, , It needs to be limited to predetermined upper and lower limits, that is: （20） （21） In the formula, , For connection with secondary bus The upper and lower limits of the charging and discharging power of the connected energy storage system. , For energy storage systems Upper and lower limits of the value; Step 1.2: Establish the objective function: 2.1) Cost function for keeping the generator operating within its optimal range Represented as: （22） 2.2) Ensure that the generator, main bus, and secondary bus execute the pre-defined cost function. As shown below: （23） In the formula, For generator With the main bus The penalty factors connected between them Main bus With secondary bus The penalty factors connected between them; 2.3) Cost function considering the lifespan of batteries in the energy storage system Represented as: （24） In the formula, The maximum rechargeable and rechargeable power of the energy storage system; Step 1.3: Establish a multi-electric aircraft energy scheduling model: （25） In the formula, This represents the weighting ratio.

3. The second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization as described in claim 2, characterized in that, The process of transforming the integer variable optimization problem into a Markov decision process includes: Objective function using integer variables Based on the established bus connection and generator state constraints, the action space a, state space s, and reward function r in the Markov decision process are obtained. The action space a is defined as {connection strategy between generator and main bus, connection strategy between main bus and secondary bus, connection strategy between offloadable load and secondary bus}, and the state space s is defined as {current operating state of generator, current time}. The designed reward function r is as follows: （26） In the formula, Based on state of time and actions The reward value obtained, As a penalty term, when the constraint conditions are met. ,otherwise .

4. The second-level energy scheduling method for multi-electric aircraft based on hybrid learning optimization as described in claim 2, characterized in that, The process of simplifying the constructed MIQP problem into a QP problem using feasible solutions of integer variables obtained from the solution includes: Substituting the feasible solution with integer variables obtained from the solution into the energy scheduling model formula (25) for the multi-electric aircraft, the integer variables in the constraint formulas (7) to (10) and formulas (13) and (14) are eliminated. , , This simplifies the MIQP problem into a QP problem.

Images