A method and system for planning four-dimensional climb trajectory of an aircraft

By using a reinforcement learning algorithm with a single-step timing difference update evaluation strategy in aircraft four-dimensional climbing track planning, combining environment and intelligent model, the reward value is calculated to solve the optimal action and fit the track, the traditional method of aerial track planning is solved The complexity problem has achieved efficient four-dimensional climbing track planning.

CN114115304BActive Publication Date: 2024-12-06NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111248553.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-26
Publication Date
2024-12-06
Estimated Expiration
2041-10-26

AI Technical Summary

Technical Problem

The traditional four-dimensional track planning method has complex aircraft timing differential processing processes, making it difficult to effectively solve the problems in aircraft four-dimensional climb track planning.

Method used

Using a reinforcement learning algorithm that uses a single-step timing differential update evaluation strategy, the aircraft can be achieved by obtaining optional actions when climbing, combining the environment and intelligent models, and computing the optimal action, and fitting the climbing track to achieve four-dimensional climbing of the aircraft Trail planning.

Benefits of technology

An efficient and refined aircraft four-dimensional climbing track planning is achieved, reducing the complexity of track planning and improving the optimization effect of tracks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114115304B_ABST
    Figure CN114115304B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for planning a four-dimensional climb trajectory of an aircraft. The present invention obtains an optional action space under a flight state according to aircraft performance, adopts a reinforcement learning algorithm of a single-step temporal difference update evaluation strategy, solves the optimal action in aircraft climbing, fits the climb trajectory obtained based on the optimal action, obtains the four-dimensional climb trajectory of the aircraft, and realizes efficient and refined four-dimensional climb trajectory planning of the aircraft.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method and system for planning a four-dimensional climbing trajectory of an aircraft, and belongs to the field of trajectory planning. Background Art

[0002] Since the concept of trajectory based operation (TBO) was proposed, four-dimensional trajectory planning has developed rapidly as one of its key technologies. Traditional path planning methods such as A* algorithm, ant colony algorithm, genetic algorithm, etc. have complex processing procedures on aircraft time difference, which makes it difficult for them to play their advantages in four-dimensional climb trajectory planning. Therefore, a new aircraft four-dimensional climb trajectory planning method is urgently needed. Summary of the invention

[0003] The present invention provides a method and system for planning a four-dimensional climb trajectory of an aircraft, which solves the problems disclosed in the background technology.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A four-dimensional climb trajectory planning method for an aircraft, comprising:

[0006] Obtain the optional actions in each flight state when the aircraft climbs in the simulated flight experiment;

[0007] According to the optional actions under the flight state, with the goal of reaching the cruise waypoint with the minimum fuel consumption and the minimum running time, a reinforcement learning algorithm with a single-step temporal difference update evaluation strategy is used to solve the optimal action of the aircraft in climbing;

[0008] The climb trajectory obtained based on the optimal action is fitted to obtain the four-dimensional climb trajectory of the aircraft.

[0009] The flight status includes horizontal position, altitude position, speed and pitch angle, and the optional actions include acceleration and pitch angle velocity.

[0010] The interacting models in reinforcement learning algorithms include environment models and agent models;

[0011] Environmental model: Based on the parameters of the aircraft's environment, calculate the aircraft's fuel consumption and position change per unit time under the current flight status; based on the fuel consumption and position change, calculate the reward value given by the environment, and use the reward value as a parameter to evaluate the aircraft's current flight status;

[0012] Agent Model: Select the best action from the available actions.

[0013] The reward value calculation formula is:

[0014]

[0015] Among them, r(s t ,a t ) is the aircraft at time t, flight state s t Next select action a t After that, the reward value given by the environment;

[0016] r t 1 The positive reward value obtained when the aircraft approaches the cruise waypoint;

[0017] r t 2 is the negative reward value obtained when the aircraft passes through the time sequence t;

[0018] r t 3 Do action for aircraft t Consume fuel and get negative rewards.

[0019] The climb trajectory obtained based on the optimal action is fitted to obtain the aircraft's four-dimensional climb trajectory, including:

[0020] Determine the aircraft's transition altitude and level flight distance during step climb based on the aircraft's performance;

[0021] According to the conversion altitude and level flight distance, the climb trajectory obtained based on the optimal action is fitted to obtain the four-dimensional climb trajectory of the aircraft.

[0022] A four-dimensional climb trajectory planning system for an aircraft, comprising:

[0023] Acquisition module: Acquisition of optional actions in various flight states when the aircraft climbs in the simulated flight experiment;

[0024] Reinforcement learning module: Based on the optional actions under the flight state, with the goal of reaching the cruise waypoint with the minimum fuel consumption and the minimum running time, a reinforcement learning algorithm with a single-step temporal difference update evaluation strategy is used to solve the optimal action of the aircraft in climbing;

[0025] Track fitting module: fits the climb track obtained based on the optimal action to obtain the four-dimensional climb track of the aircraft.

[0026] The flight status includes horizontal position, altitude position, speed and pitch angle, and the optional actions include acceleration and pitch angle velocity.

[0027] The interacting models in reinforcement learning algorithms include environment models and agent models;

[0028] Environmental model: Based on the parameters of the aircraft's environment, calculate the aircraft's fuel consumption and position change per unit time under the current flight status; based on the fuel consumption and position change, calculate the reward value given by the environment, and use the reward value as a parameter to evaluate the aircraft's current flight status;

[0029] Agent Model: Select the best action from the available actions.

[0030] The reward value calculation formula is:

[0031] r(s t ,a t )=r t 1 +r t 2 +r t 3

[0032] Among them, r(s t ,a t ) is the aircraft at time t, flight state s t Next select action a t After that, the reward value given by the environment;

[0033] r t 1 The positive reward value obtained when the aircraft approaches the cruise waypoint;

[0034] r t 2 is the negative reward value obtained when the aircraft passes through the time sequence t;

[0035] r t 3 Do action for aircraft t Consume fuel and get negative rewards.

[0036] The track fitting module includes:

[0037] The module for determining the altitude change and level flight distance is used to determine the altitude change and level flight distance of the aircraft during step climb according to the aircraft performance.

[0038] Fitting module: According to the conversion altitude and level flight distance, the climb trajectory obtained based on the optimal action is fitted to obtain the four-dimensional climb trajectory of the aircraft.

[0039] The beneficial effects achieved by the present invention are as follows: the present invention obtains the optional action space under the flight state according to the aircraft performance, adopts the reinforcement learning algorithm of the single-step temporal difference update evaluation strategy to solve the optimal action in the aircraft climb, fits the climb trajectory obtained based on the optimal action, obtains the four-dimensional climb trajectory of the aircraft, and realizes efficient and refined four-dimensional climb trajectory planning of the aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of the method of the present invention;

[0041] Figure 2 Schematic diagram of one-hot encoding discretization of aircraft state-action space;

[0042] Figure 3 A line chart showing the average total round reward value every 100 times during the 500,000 learning cycles of the agent;

[0043] Figure 4 A line chart showing the average total round reward value every 1,000 times during the agent's 500,000 learning cycles;

[0044] Figure 5 This is a schematic diagram of the track fitting effect. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0046] like Figure 1 As shown, a four-dimensional climb trajectory planning method for an aircraft includes the following steps:

[0047] Step 1, obtaining optional actions in various flight states when the aircraft climbs in a simulated flight experiment; the simulated flight experiment is a simulated flight experiment based on the actual performance of the aircraft;

[0048] Step 2: Based on the optional actions under the flight state, with the goal of reaching the cruise waypoint with the minimum fuel consumption and the minimum running time, a reinforcement learning algorithm with a single-step temporal difference update evaluation strategy is used to solve the optimal action of the aircraft in climbing;

[0049] Step 3: Fit the climb trajectory obtained based on the optimal action to obtain the four-dimensional climb trajectory of the aircraft.

[0050] The above method obtains the optional action space under the flight state according to the aircraft performance, adopts the reinforcement learning algorithm of the single-step temporal difference update evaluation strategy to solve the optimal action in the aircraft climb, and fits the climb trajectory obtained based on the optimal action to obtain the four-dimensional climb trajectory of the aircraft, thereby realizing efficient and refined four-dimensional climb trajectory planning of the aircraft.

[0051] The flight status of an aircraft is generally obtained by the flight management system (FMS) and sent to the pilot's instrument panel and the controller's secondary radar or ADS-B equipment, so it is not difficult to obtain the flight attitude during track planning. In a simulated flight experiment, the pilot's operation can be simulated according to the actual performance of the aircraft to obtain the corresponding flight status of the aircraft.

[0052] Aircraft position, speed, etc. are all continuous variables. To derive a limited number of features from the continuous space, one-hot coding can be used to process the continuous flight state of the aircraft into discrete flight states, that is, s(x i -δ≤x i ≤x i +δ)=s i , δ is the encoding granularity.

[0053] The flight status includes horizontal position, altitude position, speed and pitch angle. The optional actions include acceleration and pitch angle velocity, which can be expressed as s i =(x i ,y i ,v i ,θ i ), a i =(acc i ,w i ), s i ∈S′,a i ∈A, where s i is the flight status of aircraft i at the time sequence, x i ,y i 、v i ,θ i Respectively represent the horizontal position, altitude position, speed and pitch angle, a i For i An optional action under acc i 、w i denote acceleration and pitch angular velocity respectively, S′ is the set of all flight states, and A is the set of all optional actions (see Figure 2 )

[0054] Due to the aerodynamic characteristics of fixed-wing aircraft, the horizontal speed of the aircraft must not be 0. In order to facilitate algorithm design and speed up the algorithm iteration convergence speed, the discretized aircraft flight state space should satisfy: the flight state of the aircraft after a unit time changes in the discrete space (dynamic) and still belongs to the discrete space (closed).

[0055] Should avoid i+1 =s i =V(s i |a i )or The situation is that i+1 =V(s i |a i ),s i+1 ≠s i And s i+1 =V(a i |s i ),s i+1 ∈S′, which can be solved by setting the minimum speed, that is, the coding granularity needs to satisfy the requirement that the aircraft will change its horizontal position state even if it runs at the minimum speed for one second. At the same time, the decision should be based on the state of the aircraft's one-hot coding flight state; where V(s i |a i ) means that the aircraft is in flight state s i When selecting action a i Transfer to the next sequential flight state s i+1 The state transition equation.

[0056] There are usually multiple optional actions in each flight state. The flight state and optional actions are used as the input of the reinforcement learning algorithm of the single-step temporal difference update evaluation strategy. After iterative calculation, the optimal action in the flight state can be obtained.

[0057] The reinforcement learning algorithm of the single-step temporal difference update evaluation strategy generally has two interaction models, including the environment model and the agent model.

[0058] Environmental model: Based on the parameters of the aircraft's environment (altitude, atmospheric temperature, etc.), an aircraft performance model and a fuel consumption calculation model are constructed to calculate the aircraft's fuel consumption per unit time and position change under the current flight status. Based on the fuel consumption and position change, the reward value given by the environment is calculated, and the reward value is used as a parameter to evaluate the aircraft's current flight status, that is, to evaluate the quality of the current flight status.

[0059] The reward and penalty system and reward value calculation can be set as follows:

[0060] 1. Reward and punishment system

[0061] The aircraft approaches the cruise waypoint and obtains a certain positive reward value r t 1 (making the decision tend to fly to the destination); the aircraft passes through the time sequence t and obtains a certain negative reward value r t 2 (Reduce flight time); aircraft performs action a t Consume fuel and get a certain negative reward value r t 3 (Reduce fuel consumption).

[0062] 2. Therefore, based on the above reward and punishment system, the formula for calculating the reward value can be as follows:

[0063] r(s t ,a t )=r t 1 +r t 2 +r t 3

[0064] Among them, r(s t ,a t ) is the aircraft at time t, flight state s t Next select action a t Finally, the reward value given by the environment.

[0065] Agent model: The agent acts as the decision maker for aircraft flight action selection, selecting the best action from the optional actions and executing it.

[0066] Based on the aircraft performance, the optional actions of the aircraft in different flight states can be obtained, as follows:

[0067] 1) Determine the speed range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database:

[0068] v min ≤v si +acc ai ≤v max

[0069] v min =C Vmin ×V stall

[0070] v max =V MO / M MO

[0071] Among them, v si is the current aircraft speed at time i, acc ai represents the acceleration decision available in the current aircraft flight state, v max 、v min They represent the maximum and minimum controllable speeds under aviation performance limitations, C Vmin 、V MO 、M MO These are the performance hyperparameters corresponding to specific models in the BADA model, among which V MO represents the maximum controllable corrected airspeed, V stall is the stall speed of the aircraft, M MO Indicates the maximum controllable Mach number.

[0072] 2) Determine the acceleration range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database:

[0073] 0≤acc ai ≤a max

[0074] Thr maxclimb -D-mg=ma max

[0075]

[0076] Thr maxclimb =(Thr maxclimb ) ISA ×(1-C Tc,5 ×(ΔT-C Tc,4 ))

[0077]

[0078] C D =C D0,CR +C D2,CR ×(C L ) 2

[0079]

[0080] Among them, a max is the maximum acceleration of the aircraft, Thr maxclimb C is the maximum thrust available for the aircraft in climbing condition. Tc,1 , C Tc,2 , C Tc,3 , C Tc,4 , C Tc,5 are all hyperparameters set in the BADA model of the corresponding aircraft. They have no specific physical meaning and are only used for fitting. ΔT is the International Standard Atmosphere (ISA) temperature correction value, which is related to the actual local temperature. H p is the pressure altitude of the aircraft, D is the drag on the aircraft, ρ is the air density, v is the aircraft speed (true airspeed), S is the wing area of ​​the aircraft, and C D is the aircraft drag coefficient, C L is the aircraft lift coefficient, C D0,CR , C D2,CR is the parameter in the BADA model of a specific aircraft, m is the mass of the aircraft, and g0 is the acceleration of gravity, which is 9.8 m / s 2 3) Determine the pitch angle (climb gradient) range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database:

[0081] 0≤θ si+w i ≤θ max

[0082]

[0083]

[0084]

[0085] Thr maxclimb =(Thr maxclimb ) ISA ×(1-C Tc,5 ×(ΔT-C Tc,4 ))

[0086]

[0087] Among them, θ si is the climb gradient of aircraft i, w i is the pitch angle action available for aircraft i in sequence, ROC max is the maximum climb rate of the aircraft, θ max is the maximum climb gradient of the aircraft, M is the aircraft Mach number, T0 is the ISA atmospheric temperature, which is taken as 288.15K, f{M} is a piecewise function of the aircraft speed, ΔT is the standard atmospheric temperature correction value, T is the theoretical temperature calculated according to the vertical decrease rate of the temperature at the current aircraft pressure altitude, β T,< is the vertical temperature lapse rate, take -0.0065K / m, R=287.05m 2 / (K·s 2 ), κ=1.4, is the derivative of the vertical velocity of the aircraft with respect to the height, m ​​is the mass of the aircraft, g0 is the acceleration due to gravity, which is taken as 9.8m / s 2 .

[0088] 4) Determine the pitch velocity range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database:

[0089] w min ≤w i ≤w max

[0090] Among them, w max is the maximum pitch angular velocity of the aircraft (rad / s), w min is the maximum nose-down angular velocity of the aircraft.

[0091] Interaction between the two models: The intelligent agent selects and executes the optimal action under the current flight state of the aircraft. The environment evaluates the action of the intelligent agent according to the set reward function, and then updates the optimal action selection strategy of the intelligent agent. Through continuous interaction between the intelligent agent and the environment, the optimal action selection strategy of the aircraft is obtained, and then the optimal climb trajectory is obtained.

[0092] from Figure 3 and 4 It can be seen that after continuous learning, the reward value obtained by the agent for each simulated flight eventually tends to be stable, which also reflects the feasibility of the reinforcement learning algorithm with single-step temporal difference update optimal strategy to solve the four-dimensional trajectory planning problem.

[0093] The specific reinforcement learning algorithm can be as follows:

[0094] S1) Setting the initial flight state of the aircraft and the target cruise waypoint position;

[0095] The initial flight state of the aircraft is generally selected from the altitude, speed, and attitude when the aircraft leaves the runway and is 15 feet above the ground, while the target cruise waypoint is related to the airspace division and flight procedure design.

[0096] S2) after obtaining the current aircraft flight state, generating or updating the available action space under the current aircraft flight state based on the aircraft performance;

[0097] S3) Calculate the value (estimated value) of the state-action pair in the agent; where value is a measure of the total reward value in the future t time (global);

[0098] S4) selecting an aircraft action based on the current flight status of the aircraft;

[0099] The specific method of agent decision-making (action selection) is: to use the probability P (r ≥ ε) to bet on the roulette wheel to determine whether the agent takes the action corresponding to the highest expected return under the current state of the aircraft or takes other actions.

[0100] S5) Obtain the actual state of the aircraft after executing the selected action - the value corresponding to the action (actual value).

[0101] S6) Update the value of the state-action pair in the agent according to the actual value:

[0102] Determine the state s at time t t ∈S′, reward r at time t t ∈R, action a at time t t ∈A, the Markov decision process model is initially obtained, and the function p:S′×R×S′×A→[0,1] is defined as the state transition probability of the Markov decision process:

[0103] p(s'|s,a)=Pr[s t+1 =s',r t+1 =r|s t =s,a t =a],s∈S′,a∈A,s’∈S′

[0104] Therefore, the state transition probability is:

[0105]

[0106] The expected reward for a state-action pair r(s,a):

[0107]

[0108] Strategy π(a|s):

[0109] π(a|s)=Pr(a t =a|s t =s)

[0110] Expected return (estimated value) G t :

[0111]

[0112] Among them, γ∈[0,1] is the time-series value decay parameter, which means that the reward of 1 unit in the next τ steps is equivalent to the γ received now. τ Reward, Pr represents the function of finding probability;

[0113] The action-value function q of the Bellman expectation equation π (s|a) means:

[0114]

[0115] According to the compression mapping principle and the Banach fixed-point theorem, we have q * Compression operator for (s|a) So we only need to iterate the following formula:

[0116]

[0117] The iterative convergence point of the above formula is the fixed point of the Bellman optimal operator, which is the optimal strategy value;

[0118] The true strategy value U after single-step temporal difference t:t+1 (q) :

[0119] U t:t+1 (q) =rt+1 +γq(s t+1 ,a t+1 )=r t+1 +γ n q(s t+n ,a t+n )

[0120] Among them, r t+1 represents the expected value return at time t+1, and q(s,a) represents the reward return for taking action a under the aircraft state s;

[0121] So the iterative formula is (reduce [Uq (s, a)] 2 ):

[0122] q(s t ,a t )←q(s t ,a t )+α[g t -q(s t ,a t )]

[0123] Among them, α is the learning rate, g t The real return of environmental feedback when the aircraft makes decisions, including fuel consumption cost, time cost and target proximity reward.

[0124] S7) determining whether the aircraft has reached the target cruise waypoint, if yes, then ending this step, otherwise returning to S2);

[0125] S8) Determine whether the aircraft has reached the set learning times, if yes, end this step, otherwise return to S1);

[0126] S9) Determine the optimal action for each state of the aircraft based on the iterated optimal state-action-value pair table.

[0127] After using the reinforcement learning algorithm to obtain the optimal actions for each time sequence, assuming that the aircraft is step-climbing, the aircraft's transition altitude and level flight distance during step-climbing are determined based on the aircraft's performance;

[0128] The aircraft's transition altitude can be:

[0129]

[0130]

[0131]

[0132] Where a0 is the ISA atmospheric sound speed, which is 340 m / s, and V CAS is the corrected airspeed of the aircraft, H transis the aircraft's optimal conversion altitude, η trans is the atmospheric temperature ratio at the optimal conversion altitude, δ trans is the atmospheric pressure ratio at the optimal conversion altitude.

[0133] like Figure 3 As shown, the climbing track obtained by the above reinforcement learning has the problem that the aircraft movement changes too frequently. Therefore, it is necessary to fit the climbing track according to the climbing method; among them, the climbing method here adopts step climbing.

[0134] Determine the level flight distance of the aircraft at the transition altitude based on the route or flight procedure characteristics and control requirements.

[0135] According to the conversion altitude and level flight distance, the climb track based on the optimal action is fitted by the least square method to obtain the four-dimensional climb track of the aircraft;

[0136] Use linear least squares method to fit the track:

[0137]

[0138] f(x i )=a1x i +a2

[0139] Where J is the sum of the mean square errors between the fitted track points and the original track points, f(·) represents a univariate linear regression function, a1 and a2 are the parameters to be fitted, and x i is the horizontal position of the track sample point, y i is the altitude position of the track sample point.

[0140] Based on the same technical solution, the present invention also discloses a software system of the above method, a four-dimensional climb trajectory planning system for an aircraft, comprising:

[0141] Acquisition module: acquires optional actions in various flight states when the aircraft climbs in a simulated flight experiment; the flight state includes horizontal position, altitude position, speed and pitch angle, and the optional actions include acceleration and pitch angle velocity.

[0142] Reinforcement learning module: Based on the optional actions under the flight state, with the goal of reaching the cruise waypoint with minimum fuel consumption and minimum running time, a reinforcement learning algorithm with a single-step temporal difference update evaluation strategy is used to solve the optimal action for the aircraft to climb.

[0143] The interacting models in reinforcement learning algorithms include environment models and agent models;

[0144] Environmental model: Based on the parameters of the aircraft's environment, calculate the aircraft's fuel consumption and position change per unit time under the current flight status; based on the fuel consumption and position change, calculate the reward value given by the environment, and use the reward value as a parameter to evaluate the aircraft's current flight status;

[0145] Agent Model: Select the best action from the available actions.

[0146] The reward value calculation formula is:

[0147] r(s t ,a t )=r t 1 +r t 2 +r t 3

[0148] Among them, r(s t ,a t ) is the aircraft at time t, flight state s t Next select action a t After that, the reward value given by the environment;

[0149] r t 1 The positive reward value obtained when the aircraft approaches the cruise waypoint;

[0150] r t 2 is the negative reward value obtained when the aircraft passes through the time sequence t;

[0151] r t 3 Do action for aircraft t Consume fuel and get negative rewards.

[0152] Track fitting module: fits the climb track obtained based on the optimal action to obtain the four-dimensional climb track of the aircraft.

[0153] The track fitting module includes:

[0154] The module for determining the altitude change and level flight distance is used to determine the altitude change and level flight distance of the aircraft during step climb according to the aircraft performance.

[0155] Fitting module: According to the conversion altitude and level flight distance, the climb trajectory obtained based on the optimal action is fitted to obtain the four-dimensional climb trajectory of the aircraft.

[0156] Based on the same technical solution, the present invention also discloses a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes a four-dimensional climb trajectory planning method for an aircraft.

[0157] Based on the same technical solution, the present invention also discloses a computing device, including one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing four-dimensional climb trajectory planning of an aircraft.

[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0159] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0162] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A method for planning a four-dimensional climb trajectory of an aircraft, characterized in that: include: Obtain the optional actions in each flight state when the aircraft climbs in the simulated flight experiment; According to the optional actions under the flight state, with the goal of reaching the cruise waypoint with the minimum fuel consumption and the minimum running time, a reinforcement learning algorithm with a single-step temporal difference update evaluation strategy is used to solve the optimal action of the aircraft in climbing; The interactive models in the reinforcement learning algorithm include the environment model and the agent model; the environment model: calculates the fuel consumption and position change of the aircraft per unit time under the current flight state according to the parameters of the environment in which the aircraft is located; calculates the reward value given by the environment according to the fuel consumption and position change, and uses the reward value as a parameter for evaluating the current flight state of the aircraft; the agent model: selects the best action from the optional actions; The reward value calculation formula is: r(s t ,a t )=r t 1 +r t 2 +r t 3 Among them, r(s t ,a t ) is the aircraft at time t, flight state s t Next select action a t After that, the reward value given by the environment; r t 1 is the positive reward value obtained when the aircraft approaches the cruise waypoint; r t 2 is the negative reward value obtained when the aircraft passes through the time sequence t; r t 3 Do action for aircraft t Consuming fuel will result in negative rewards; The reinforcement learning algorithm includes: S1) setting the initial flight state of the aircraft and the position of the target cruise waypoint; S2) after obtaining the current aircraft flight state, generating or updating the available action space under the current aircraft flight state based on the aircraft performance; S3) calculating the estimated value corresponding to the state-action pair in the intelligent agent; S4) making an aircraft action selection based on the current flight state of the aircraft; S5) obtaining the actual state of the aircraft after executing the selected action-the actual value corresponding to the action; S6) updating the estimated value corresponding to the state-action pair in the intelligent agent according to the actual value; S7) judging whether the aircraft has reached the target cruise waypoint, if so, ending the step, otherwise returning to S2); S8) judging whether the aircraft has reached the set number of learning times, if so, ending the step, otherwise returning to S1); S9) determining the optimal action of the aircraft in each state according to the optimal state-action-value pair table after iteration; The flight state includes horizontal position, altitude position, speed and pitch angle; the flight state space is a discretized flight state space, which shall satisfy the following conditions: the flight state of the aircraft after a unit time changes in the discrete space and still belongs to the discrete space; Based on the aircraft performance, the optional actions of the aircraft in different flight states can be obtained, as follows: 1) Determine the speed range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: v min ≤v si +acc ai ≤v max in min =C Vmin ×V stall in max =V MO / M MO In the formula, v si is the current aircraft speed at time i, acc ai represents the acceleration decision available in the current aircraft flight state, v max 、v min They represent the maximum and minimum controllable speeds under aviation performance limitations, C Vmin 、V MO 、M MO These are the performance hyperparameters corresponding to specific models in the BADA model. MO represents the maximum controllable corrected airspeed, V stall is the stall speed of the aircraft, M MO Indicates the maximum controllable Mach number; 2) Determine the acceleration range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: 0≤acc ai ≤a max Thr maxclimb -D-mg=ma max Thr maxclimb =(Thr maxclimb ) ISA ×(1-C Tc,5 ×(ΔT-C Tc,4 )) C D =C D0,CR +C D2,CR ×(C L ) 2 In the formula, a max is the maximum acceleration of the aircraft, Thr maxclimb C is the maximum thrust available for the aircraft in climbing condition. Tc,1 , C Tc,2 , C Tc,3 , C Tc,4 , C Tc,5 are all hyperparameters set in the corresponding aircraft BADA model. ΔT is the international standard atmospheric temperature correction value, which is related to the actual local temperature. p is the pressure altitude of the aircraft, D is the drag on the aircraft, ρ is the air density, v is the true air speed of the aircraft, S is the wing area of ​​the aircraft, and C D is the aircraft drag coefficient, C L is the aircraft lift coefficient, C D0,CR , C D2,CR are the parameters in the BADA model of a specific aircraft, m is the mass of the aircraft, and g0 is the acceleration due to gravity; 3) Determine the pitch angle range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: 0≤θ si +in i ≤θ max Thr maxclimb =(Thr maxclimb ) ISA ×(1-C Tc,5 ×(ΔT-C Tc,4 )) In the formula, θ si is the climb gradient of aircraft i, w i is the pitch angle action available for aircraft i in sequence, ROC max is the maximum climb rate of the aircraft, θ max is the maximum climb gradient of the aircraft, M is the aircraft Mach number, f{M} is a piecewise function of the aircraft speed, ΔT is the standard atmospheric temperature correction value, T is the theoretical temperature calculated according to the vertical temperature decrease rate of the current aircraft pressure altitude, β T,< is the vertical temperature lapse rate, R = 287.05m 2 / (K·s 2 ), κ=1.4, is the derivative of the aircraft's vertical velocity with respect to altitude; 4) Determine the pitch velocity range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: In min ≤in i ≤in max Among them, w max is the maximum pitch angular velocity of the aircraft (rad / s), w min is the maximum nose-down angular velocity of the aircraft; The climb trajectory obtained based on the optimal action is fitted to obtain the four-dimensional climb trajectory of the aircraft.

2. The method for planning a four-dimensional climb trajectory of an aircraft according to claim 1, characterized in that: The climb trajectory obtained based on the optimal action is fitted to obtain the aircraft's four-dimensional climb trajectory, including: Determine the aircraft's transition altitude and level flight distance during step climb based on the aircraft's performance; According to the conversion altitude and level flight distance, the climb trajectory based on the optimal action is fitted to obtain the four-dimensional climb trajectory of the aircraft.

3. A four-dimensional aircraft climb trajectory planning system, characterized in that: include: Acquisition module: Acquisition of optional actions in various flight states when the aircraft climbs in the simulated flight experiment; Reinforcement learning module: Based on the optional actions under the flight state, with the goal of reaching the cruise waypoint with the minimum fuel consumption and the minimum running time, a reinforcement learning algorithm with a single-step temporal difference update evaluation strategy is used to solve the optimal action of the aircraft in climbing; The interactive models in the reinforcement learning algorithm include the environment model and the agent model; the environment model: calculates the fuel consumption and position change of the aircraft per unit time under the current flight state according to the parameters of the environment in which the aircraft is located; calculates the reward value given by the environment according to the fuel consumption and position change, and uses the reward value as a parameter for evaluating the current flight state of the aircraft; the agent model: selects the best action from the optional actions; The reward value calculation formula is: r(s t ,a t )=r t 1 +r t 2 +r t 3 Among them, r(s t ,a t ) is the aircraft at time t, flight state s t Next select action a t After that, the reward value given by the environment; r t 1 is the positive reward value obtained when the aircraft approaches the cruise waypoint; r t 2 is the negative reward value obtained when the aircraft passes through the time sequence t; r t 3 Do action for aircraft t Consuming fuel will result in negative rewards; The reinforcement learning algorithm includes: S1) setting the initial flight state of the aircraft and the position of the target cruise waypoint; S2) after obtaining the current aircraft flight state, generating or updating the available action space under the current aircraft flight state based on the aircraft performance; S3) calculating the estimated value corresponding to the state-action pair in the intelligent agent; S4) making an aircraft action selection based on the current flight state of the aircraft; S5) obtaining the actual state of the aircraft after executing the selected action-the actual value corresponding to the action; S6) updating the estimated value corresponding to the state-action pair in the intelligent agent according to the actual value; S7) judging whether the aircraft has reached the target cruise waypoint, if so, ending the step, otherwise returning to S2); S8) judging whether the aircraft has reached the set number of learning times, if so, ending the step, otherwise returning to S1); S9) determining the optimal action of the aircraft in each state according to the optimal state-action-value pair table after iteration; The flight state includes horizontal position, altitude position, speed and pitch angle; the flight state space is a discretized flight state space, which shall satisfy the following conditions: the flight state of the aircraft after a unit time changes in the discrete space and still belongs to the discrete space; Based on the aircraft performance, the optional actions of the aircraft in different flight states can be obtained, as follows: 1) Determine the speed range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: v min ≤v si +acc ai ≤v max in min =C Vmin ×V stall in max =V MO / M MO In the formula, v si is the current aircraft speed at time i, acc ai represents the acceleration decision available in the current aircraft flight state, v max 、v min They represent the maximum and minimum controllable speeds under aviation performance limitations, C Vmin 、V MO 、M MO These are the performance hyperparameters corresponding to specific models in the BADA model. MO represents the maximum controllable corrected airspeed, V stall is the stall speed of the aircraft, M MO Indicates the maximum controllable Mach number; 2) Determine the acceleration range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: 0≤acc ai ≤a max Thr maxclimb -D-mg=ma max Thr maxclimb =(Thr maxclimb ) ISA ×(1-C Tc,5 ×(ΔT-C Tc,4 )) C D =C D0,CR +C D2,CR ×(C L ) 2 In the formula, a max is the maximum acceleration of the aircraft, Thr maxclimb C is the maximum thrust available for the aircraft in climbing condition. Tc,1 , C Tc,2 , C Tc,3 , C Tc,4 , C Tc,5 are all hyperparameters set in the corresponding aircraft BADA model. ΔT is the international standard atmospheric temperature correction value, which is related to the actual local temperature. p is the pressure altitude of the aircraft, D is the drag on the aircraft, ρ is the air density, v is the true air speed of the aircraft, S is the wing area of ​​the aircraft, and C D is the aircraft drag coefficient, C L is the aircraft lift coefficient, C D0,CR , C D2,CR are the parameters in the BADA model of a specific aircraft, m is the mass of the aircraft, and g0 is the acceleration due to gravity; 3) Determine the pitch angle range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: 0≤θ si +in i ≤θ max Thr maxclimb =(Thr maxclimb ) ISA ×(1-C Tc,5 ×(ΔT-C Tc,4 )) In the formula, θ si is the climb gradient of aircraft i, w i is the pitch angle action available for aircraft i in sequence, ROC max is the maximum climb rate of the aircraft, θ max is the maximum climb gradient of the aircraft, M is the aircraft Mach number, f{M} is a piecewise function of the aircraft speed, ΔT is the standard atmospheric temperature correction value, T is the theoretical temperature calculated according to the vertical temperature decrease rate of the current aircraft pressure altitude, β T,< is the vertical temperature lapse rate, R = 287.05m 2 / (K·s 2 ), κ=1.4, is the derivative of the aircraft's vertical velocity with respect to altitude; 4) Determine the pitch velocity range corresponding to a specific aircraft model and flight altitude based on the aircraft performance database: In min ≤in i ≤in max Among them, w max is the maximum pitch angular velocity of the aircraft (rad / s), w min is the maximum nose-down angular velocity of the aircraft; Track fitting module: fits the climb track obtained based on the optimal action to obtain the four-dimensional climb track of the aircraft.

4. The aircraft four-dimensional climb trajectory planning system according to claim 3, characterized in that: The track fitting module includes: The module for determining the altitude change and level flight distance is used to determine the altitude change and level flight distance of the aircraft during step climb according to the aircraft performance. Fitting module: According to the conversion altitude and level flight distance, the climb trajectory obtained based on the optimal action is fitted to obtain the four-dimensional climb trajectory of the aircraft.