Aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generating method and system

By decomposing the obstacle avoidance problem of aircraft using a three-degree-of-freedom dynamic model and reinforcement learning methods, real-time intelligent decision-making and trajectory generation of aircraft in complex dynamic environments are realized. This solves the problems of insufficient real-time performance and environmental adaptability in existing technologies and improves the efficiency and accuracy of obstacle avoidance.

CN118938950APending Publication Date: 2024-11-12BEIHANG UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410983595.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing obstacle avoidance methods for aircraft are insufficient in terms of real-time performance and environmental adaptability, and cannot effectively meet the real-time decision-making and trajectory generation requirements of complex dynamic obstacles.

Method used

By employing a three-degree-of-freedom dynamic model combined with reinforcement learning, the obstacle avoidance problem is broken down into upper-level intelligent decision-making and lower-level trajectory generation. The deflection angle is optimized through a single-step Markov decision process to achieve dynamic obstacle avoidance.

Benefits of technology

It improves the real-time performance, environmental adaptability, and accuracy of dynamic obstacle avoidance for aircraft, reduces the training difficulty of reinforcement learning models, and enhances the efficiency and accuracy of intelligent decision-making and trajectory generation for aircraft.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118938950A_ABST
    Figure CN118938950A_ABST
Patent Text Reader

Abstract

The invention discloses an aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation method and system, and relates to the technical field of aircraft obstacle avoidance, and the method comprises the steps: forming an aircraft obstacle avoidance problem model under a multi-constraint condition through employing a three-degree-of-freedom dynamics model and obstacle avoidance constraints of a target aircraft, based on a reinforcement learning method, converting an upper-layer intelligent decision into a single-step Markov decision process, inputting a real-time flight state, a real-time obstacle position and a current obstacle radius of a target aircraft into a trained strategy network to obtain a current decision strategy; and inputting the current decision strategy into the lower layer track generation for obstacle avoidance track planning, obtaining an obstacle avoidance track corresponding to the current decision strategy, and obtaining an optimal obstacle avoidance decision strategy and an optimal obstacle avoidance track, thereby improving real-time performance, environmental adaptability, accuracy and efficiency of dynamic obstacle avoidance of the target aircraft.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of aircraft obstacle avoidance, and in particular to an intelligent decision-making and trajectory generation method and system for aircraft dynamic obstacle avoidance. Background Art

[0002] Aircraft are facing a more complex and dynamically changing mission environment. In actual flight missions, there are multiple obstacles of varying sizes and time-varying distributions. Aircraft need to make real-time decisions on obstacle avoidance strategies and plan safe and reliable flight trajectories, which brings real-time and adaptability challenges to aircraft decision-making and trajectory generation.

[0003] Existing obstacle avoidance methods for aircraft are either based on complex nonlinear programming for offline solution, which is not real-time and environmentally adaptable; or only consider simple motion models, and the performance of the resulting decision strategy may deviate greatly from the performance of the actual dynamic trajectory, or even fail to meet the constraints and cannot reliably achieve the obstacle avoidance task. Therefore, there is an urgent need for an intelligent decision-making and trajectory generation method for dynamic obstacle avoidance of aircraft that is real-time, environmentally adaptable, efficient, and accurate while meeting the actual dynamic constraints. Summary of the invention

[0004] The purpose of this application is to provide an intelligent decision-making and trajectory generation method and system for dynamic obstacle avoidance of an aircraft, which can improve the real-time, environmental adaptability, accuracy and efficiency of dynamic obstacle avoidance of a target aircraft under the premise of meeting actual dynamic constraints.

[0005] To achieve the above objectives, this application provides the following solutions.

[0006] The present application provides an intelligent decision-making and trajectory generation method for dynamic obstacle avoidance of an aircraft, comprising the following steps.

[0007] Step 201: A three-degree-of-freedom dynamics model is used to describe the motion of the aircraft, and process constraints, obstacle constraints, terminal constraints and performance indicators are established to form an aircraft obstacle avoidance problem model; the obstacle constraint is a geometric model of the obstacle changing dynamically with the aircraft; the performance indicator is the minimum cumulative lateral control force.

[0008] Step 202: Based on the reinforcement learning method, the upper-level intelligent decision is converted into a single-step Markov decision process, and the state set, behavior set and reward function corresponding to the upper-level intelligent decision are defined.

[0009] Step 203: input the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy; the current decision strategy is the deflection angle of the target aircraft.

[0010] Step 204: Input the current decision strategy into the lower-layer trajectory generation to perform obstacle avoidance trajectory planning, and obtain the obstacle avoidance trajectory corresponding to the current decision strategy.

[0011] Step 205: Repeat steps 203 to 204 to obtain an optimal obstacle avoidance decision strategy and an optimal obstacle avoidance trajectory; the optimal obstacle avoidance decision strategy includes a deflection angle of the target aircraft in each information update cycle.

[0012] In a second aspect, the present application provides an aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system, including the following modules.

[0013] The aircraft obstacle avoidance problem model establishment module is used to: use a three-degree-of-freedom dynamic model to describe the movement of the aircraft, establish process constraints, obstacle constraints, terminal constraints and performance indicators, and form an aircraft obstacle avoidance problem model; the obstacle constraint is a geometric model in which the obstacle changes dynamically with the aircraft; the performance indicator is the minimum cumulative lateral control force.

[0014] The upper-level intelligent decision conversion module is used to: convert the upper-level intelligent decision into a single-step Markov decision process based on the reinforcement learning method, and define the state set, behavior set and reward function corresponding to the upper-level intelligent decision.

[0015] The current decision strategy determination module is used to: input the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy; the current decision strategy is the deflection angle of the target aircraft.

[0016] The obstacle avoidance trajectory planning module is used to: input the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, and obtain the obstacle avoidance trajectory corresponding to the current decision strategy.

[0017] The intelligent decision-making and trajectory generation module is used to: repeat the steps of "inputting the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy" to the step of "inputting the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, and obtaining the obstacle avoidance trajectory corresponding to the current decision strategy" to obtain the optimal obstacle avoidance decision strategy and the optimal obstacle avoidance trajectory; the optimal obstacle avoidance decision strategy includes the deflection angle of the target aircraft in each information update cycle.

[0018] According to the specific embodiments provided by the present application, the present application discloses the following technical effects: the present application provides a method and system for intelligent decision-making and trajectory generation for dynamic obstacle avoidance of an aircraft, which reduces the training difficulty of the reinforcement learning model and improves the solution efficiency by reasonably splitting the complex dynamic obstacle avoidance problem into upper-level intelligent decision-making and lower-level trajectory generation; by updating the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft, the problem of the lack of real-time and environmental adaptability of the existing method is solved, and the real-time and environmental adaptability of dynamic obstacle avoidance of the target aircraft are achieved; based on the reinforcement learning method, the upper-level intelligent decision is converted into a single-step Markov decision process, which can effectively select the optimal dynamic obstacle avoidance strategy under the premise of considering the capabilities of the target aircraft, thereby improving the effectiveness of the method and the global optimization capability, that is, improving the accuracy of the intelligent decision-making and trajectory generation of the target aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is an application environment diagram of an intelligent decision-making and trajectory generation method for dynamic obstacle avoidance of an aircraft in one embodiment of the present application.

[0021] Figure 2 A flowchart of an intelligent decision-making and trajectory generation method for dynamic obstacle avoidance of an aircraft provided in one embodiment of the present application.

[0022] Figure 3 A schematic diagram of a hierarchical solution framework for intelligent decision-making and trajectory generation provided in one embodiment of the present application.

[0023] Figure 4 A schematic diagram of the dynamic changes of obstacles provided in one embodiment of the present application.

[0024] Figure 5 A schematic diagram of a target aircraft intelligent decision-making reinforcement learning training framework provided in one embodiment of the present application.

[0025] Figure 6 A schematic diagram of a real-time application framework for intelligent decision-making and trajectory generation for a target aircraft provided in one embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] Since obstacles change in real time, the obstacle avoidance strategy is tightly coupled with flight dynamics, and decision-making and trajectory generation cannot be separated and conquered. In addition, this problem is a complex nonlinear optimization problem, and the optimal strategy cannot be solved analytically. Only reinforcement learning methods with weak model dependence can be used. However, if reinforcement learning is directly applied to the solution of the entire problem, it will lead to excessive complexity and make it difficult for the algorithm to converge.

[0029] The target aircraft intelligent decision-making and trajectory generation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft to the server 104. After the server 104 receives the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft, the server 104 converts the upper-level intelligent decision into a single-step Markov decision process based on the reinforcement learning method for the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft, inputs the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network, obtains the current decision strategy, inputs the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, obtains the obstacle avoidance trajectory corresponding to the current decision strategy, and finally obtains the optimal obstacle avoidance decision strategy and the optimal obstacle avoidance trajectory. The server 104 can feedback the obtained optimal obstacle avoidance decision strategy and the optimal obstacle avoidance trajectory to the terminal 102. In addition, in some embodiments, the target aircraft intelligent decision-making and trajectory generation method can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform obstacle avoidance trajectory planning processing based on the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft, or the server 104 can obtain the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft from the data storage system and perform obstacle avoidance trajectory planning processing.

[0030] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, and tablet computers. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0031] In an exemplary embodiment, Figure 2 As shown, a method for intelligent decision-making and trajectory generation for dynamic obstacle avoidance of an aircraft is provided. The method is executed by a computer device, and specifically can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps 201 to 205.

[0032] Step 201: A three-degree-of-freedom dynamics model is used to describe the motion of the aircraft, and process constraints, obstacle constraints, terminal constraints and performance indicators are established to form an aircraft obstacle avoidance problem model; the obstacle constraint is a geometric model of the obstacle changing dynamically with the aircraft; the performance indicator is the minimum cumulative lateral control force.

[0033] Step 202: Based on the reinforcement learning method, the upper-level intelligent decision is converted into a single-step Markov decision process, and the state set, behavior set and reward function corresponding to the upper-level intelligent decision are defined.

[0034] Step 203: input the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy; the current decision strategy is the deflection angle of the target aircraft.

[0035] Step 204: Input the current decision strategy into the lower-layer trajectory generation to perform obstacle avoidance trajectory planning, and obtain the obstacle avoidance trajectory corresponding to the current decision strategy.

[0036] Step 205: Repeat steps 203 to 204 to obtain an optimal obstacle avoidance decision strategy and an optimal obstacle avoidance trajectory; the optimal obstacle avoidance decision strategy includes a deflection angle of the target aircraft in each information update cycle.

[0037] By implementing the above steps 201 to 205, the complex dynamic obstacle avoidance problem is reasonably split into upper-level intelligent decision-making and lower-level trajectory generation, thereby reducing the training difficulty of the reinforcement learning model and improving the solution efficiency; by updating the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft, the problem of the existing method not having real-time and environmental adaptability is solved, and the real-time and environmental adaptability of the dynamic obstacle avoidance of the target aircraft is achieved; based on the reinforcement learning method, the upper-level intelligent decision is converted into a single-step Markov decision process, which can effectively select the optimal dynamic obstacle avoidance strategy under the premise of considering the capabilities of the target aircraft, thereby improving the effectiveness of the method and the global optimization capability, that is, improving the accuracy of the intelligent decision-making and trajectory generation of the target aircraft.

[0038] In another exemplary embodiment of the present application, the aircraft obstacle avoidance problem model in the above step 201 is specifically as follows: The present application adopts a three-degree-of-freedom dynamic model to describe the movement of the target aircraft, uses an infinitely high cylinder to describe obstacles, establishes process constraints, obstacle constraints, terminal constraints and performance indicators, and forms an aircraft obstacle avoidance problem model.

[0039] (1) Three-degree-of-freedom dynamic model: The target aircraft center of mass motion equation is expressed as follows.

[0040]

[0041] The above equations (1) to (6) are obtained by differentiating the time t, where r = R e +h is the radial distance from the center of the earth to the center of gravity of the target aircraft, R eis the average radius of the earth, h is the altitude, θ and is the geographical longitude and latitude of the target aircraft; ψ and γ are the track deviation and track inclination of the target aircraft, V is the speed of the target aircraft relative to the earth, ω e is the angular velocity of the earth's rotation, g is the acceleration of gravity, L is the lift acceleration, D is the drag acceleration, α is the angle of attack, and σ is the roll angle.

[0042] The above-mentioned target aircraft center of mass motion equation can be written in the following compact form, that is, the three-degree-of-freedom dynamic model is shown in the following formula.

[0043]

[0044] In the formula, the state quantity x includes the distance from the center of the earth, longitude, latitude, speed, track inclination and track deviation, that is, The control variable u includes the angle of attack and the roll angle, that is, u = [α, σ] T .

[0045] (2) The process constraint is expressed as shown below.

[0046] g(x(t),u(t))≤g max (8).

[0047] In the formula, g(x,u) contains all process constraints, g max is the upper bound of each process constraint.

[0048] (3) Terminal constraints mainly include terminal speed constraints V f , terminal height constraint h f , Terminal position constraints Terminal range constraints f Etc. The terminal constraint is expressed as shown below.

[0049] V(t f )=V f , h(t f )≈h f , s(t f )≤s f (9).

[0050] In the formula, V(t f ) is the speed at the terminal moment; V f is the set terminal speed; h(t f ) is the height at the terminal moment; h f To set the terminal height; s(t f ) is the distance at the terminal time; s f is the set terminal range; s is the range, which is the distance from the current position of the aircraft to the target point The great circle distance.

[0051] (4) This application models the obstacle as a cylinder of infinite height. The obstacle constraint is expressed as follows.

[0052]

[0053] In the formula, The obstacle position after dynamic changes; is the initial obstacle position, represented by the initial obstacle center position; r o Initial radius, initial center position of obstacle and radius r o In the present application, the obstacle radius is the radius of the bottom circle of the cylinder.

[0054] (5) Performance index: In order to reduce the energy loss during obstacle avoidance, the performance index is set to the minimum cumulative lateral control force. The calculation formula for the minimum cumulative lateral control force is shown in the following formula.

[0055]

[0056] Among them, t0 and t f are the initial moment and the terminal moment respectively, a=Lsinσ is the lateral acceleration, that is, the component of the aerodynamic lift acceleration on the horizontal plane.

[0057] This application establishes a geometric model of obstacles changing dynamically with the aircraft, which is used to determine the position and radius of the obstacles after the changes. The center position and radius of the obstacle are related to the deflection angle of the aircraft to avoid the obstacle, as shown in the schematic diagram Figure 4 shown.

[0058] First, calculate the dynamically changing obstacle center position. Draw an arc with the current aircraft position as the center, the line connecting the current aircraft position and the current obstacle center as the radius, and the deflection angle of the aircraft to avoid the obstacle as the center angle, where the starting point of the arc is at the current obstacle center position. Then, the obstacle center position after dynamic change is on the line connecting the current aircraft position and the end position of the arc, where the distance between the current aircraft position and the obstacle center position after change is linearly related to the deflection angle for avoiding the obstacle. The larger the deflection angle, the larger the distance.

[0059] d b =a1Δψ+d o (12).

[0060] Among them, d o is the current distance, Δψ is the deflection angle, a1 is the linear coefficient, d b is the distance after the change.

[0061] Then, the dynamically changing obstacle radius is calculated. The obstacle radius is linearly related to the deflection angle of the aircraft to avoid the obstacle. The larger the deflection angle, the smaller the obstacle radius.

[0062] r b =-a2Δψ+r o (13).

[0063] Among them, r o is the obstacle radius after dynamic change, the current obstacle radius, Δψ is the deflection angle, a2 is the linear coefficient, r b is the obstacle radius after the change.

[0064] The dynamic obstacle avoidance of an aircraft needs to complete two tasks: selecting the deflection angle to avoid the obstacle and generating a flight trajectory to avoid the obstacle. Therefore, the solution framework for the dynamic obstacle avoidance problem of an aircraft is set up as two layers: the upper layer is intelligent decision-making, which selects the deflection angle to avoid the obstacle according to the current flight status of the aircraft and the position and radius of the obstacle; the lower layer is trajectory generation, which generates the flight trajectory according to the flight status, deflection angle, target point and other information of the aircraft to meet the dynamics, process constraints, obstacle constraints and terminal constraints. The hierarchical solution framework of the aircraft intelligent decision-making and trajectory generation is as follows: Figure 3 shown.

[0065] Specifically, the upper intelligent decision inputs the flight status of the aircraft Obstacle location and radius r o , the aircraft avoids the obstacle and outputs the deflection angle Δψ for obstacle avoidance, so that the obstacle changes dynamically to a position and size that is favorable to the aircraft, thereby reducing the energy loss of the aircraft during obstacle avoidance. The lower-level trajectory generation inputs the aircraft flight state Deflection angle Δψ, dynamic obstacle position and radius r b , Target point location Target point height h f and speed V f , plan the flight trajectory that satisfies the dynamic constraints, process constraints, obstacle constraints and terminal constraints, and output the performance indicators of the trajectory Feedback is given to the upper-level intelligent decision-making to evaluate the performance of the decision-making strategy and to modify the decision-making strategy. The upper-level intelligent decision-making and the lower-level trajectory generation are performed alternately to finally generate the obstacle avoidance decision-making strategy and flight trajectory with the best performance indicators.

[0066] In step 202, the upper-level intelligent decision is solved by reinforcement learning, and the decision on the deflection angle is converted into a single-step Markov decision process. An interactive sampling model between the upper-level intelligent decision and the lower-level trajectory generation is established. Through obstacle avoidance trajectory planning, the trajectory generated by the current strategy is simulated, and the performance index corresponding to the trajectory is used as a reward. At the same time, the random distribution of input is introduced during the training process to improve the generalization ability of the obtained strategy to uncertainty. Finally, the REINFORCE algorithm is used to optimize the above training model. The aircraft intelligent decision reinforcement learning training framework is as follows: Figure 5 shown.

[0067] (1) The single-step Markov decision model of the upper-level intelligent decision is defined by the state set S, the behavior set A and the reward function r. The single-step Markov decision (MDP) process is expressed by the following formula.

[0068] The state set S is the state of the entire system at the decision time. The state set is the flight state of the aircraft. Obstacle location and radius r o , the state set is expressed as formula (14).

[0069]

[0070] in, x represents the flight state of the target aircraft; r is the radial distance from the center of the earth to the center of gravity of the target aircraft; θ and are the geographical longitude and geocentric latitude of the target aircraft, respectively; V is the speed of the target aircraft relative to the earth; ψ and γ are the track deviation and track inclination of the target aircraft, respectively; is the initial obstacle position; r o is the radius.

[0071] The behavior set A is the decision behavior that can be taken at the decision moment, and the behavior set is the deflection angle Δψ for obstacle avoidance, that is, the behavior set is expressed as the following formula.

[0072] A=[Δψ1; Δψ2; ...; Δψ n ] (15).

[0073] In the formula, Δψ i is the i-th discretized deflection angle, i=1, 2,…, n is the discretized deflection angle level, and n is the number of discrete points.

[0074] The reward function r is related to the optimization goal and is used to describe the optimality of the current state and behavior. Here, the reward is the total energy loss of obstacle avoidance, which is obtained from the result of the lower-level trajectory generation, that is, the reward function is expressed as follows.

[0075]

[0076] Where J is the minimum cumulative lateral control force; t0 and t f are the initial moment and the terminal moment respectively, and a is the lateral acceleration.

[0077] (2) Decision strategy optimization based on the REINFORCE algorithm: In order to make the deflection angle output by the single-step MDP model the strategy with the maximum reward, the REINFORCE algorithm is used to perform gradient estimation and strategy optimization of the decision.

[0078] The trained policy network is obtained by training with the REINFORCE algorithm. The trained policy network is obtained by training with the policy network; the training process of the policy network is as follows.

[0079] 1) Using neural network π θ Fit the optimal decision strategy, where θ is the neural network parameter to be optimized; network π θ The output is the probability p of selecting each deflection angle level θ (A|S); define the loss function L(θ|π θ ) is the expected value of reward r, which satisfies the calculation rule shown in formula (17).

[0080]

[0081] 2) To update the gradient of the neural network parameter θ, the REINFORCE algorithm is used to estimate the gradient until the iteration stop condition is met to obtain a trained policy network. The loss function gradient can be written as follows.

[0082]

[0083] 3) The neural network parameters are further updated along the gradient direction to complete the iterative optimization of the strategy. The update equation can be written as shown in the following formula (19).

[0084]

[0085] Among them, α is the learning rate and γ is the decay factor, which gradually decreases as the number of iterations k increases.

[0086] Step 204 specifically includes the following steps 301 to 302 .

[0087] Step 301: Obtain a deflection angle according to the current decision strategy to avoid dynamically changing obstacles, specifically including: calculating the center position of the obstacle circle after dynamic changes according to the deflection angle; calculating the radius of the obstacle after dynamic changes according to the deflection angle; avoiding the dynamically changing obstacles according to the center position of the obstacle circle after dynamic changes and the obstacle radius after dynamic changes.

[0088] Step 302: Convert the lower-level trajectory generation into a path point tracking guidance problem, take minimizing the energy consumption of path point tracking as the optimization goal, design the optimal guidance instruction, and simulate to obtain the obstacle avoidance trajectory corresponding to the current decision strategy; the optimal guidance instruction is the lateral acceleration; the path point is tangent to the obstacle boundary.

[0089] Based on the deflection angle obtained by the upper-level intelligent decision, the position and size of the obstacle after dynamic change are obtained, and the avoidance trajectory is generated for the obstacle. Specifically, a geometric model of the obstacle changing dynamically with the aircraft is established. The center position and radius of the obstacle are related to the deflection angle of the aircraft to avoid the obstacle. First, the center position of the obstacle that changes dynamically is calculated. With the current aircraft position as the center, the line connecting the current aircraft position and the current obstacle center as the radius, and the deflection angle of the aircraft to avoid the obstacle as the center angle, an arc is drawn, wherein the starting point of the arc is at the current obstacle center position. Then, the center position of the obstacle after dynamic change is on the line connecting the current aircraft position and the end position of the arc, and the distance between the current aircraft position and the center position of the obstacle after change is linearly related to the deflection angle of the obstacle to avoid the obstacle. The larger the deflection angle, the larger the distance. Then, the radius of the obstacle after dynamic change is calculated based on the deflection angle based on formula (13). The radius of the obstacle is linearly related to the deflection angle of the aircraft to avoid the obstacle. The larger the deflection angle, the smaller the radius of the obstacle.

[0090] The trajectory generation is transformed into a path point tracking and guidance problem, in which a path point tangent to the obstacle boundary is set in the direction of the side where the aircraft has a smaller lateral maneuver to avoid the obstacle as the path point for obstacle avoidance. A minimum control force path point tracking and guidance method is adopted, with the energy consumption of path point tracking minimized as the optimization goal, and the optimal guidance law is designed to obtain the obstacle avoidance trajectory through simulation.

[0091] Specifically, without loss of generality, assuming that the target aircraft needs to pass through N path points, according to the relative geometric relationship between the target aircraft and the i-th path point, its kinematic model is shown in the following formula.

[0092]

[0093] Among them, s i and ψ LOSi represents the relative great circle distance and the line of sight angle, Δψ i Indicates the heading error.

[0094] Assume that N path points are arranged according to their corresponding arrival time t f,i Increase the sort, that is, t f,i <t f,i+1 When the aircraft approaches a waypoint, the arrival time can be approximated by formula (21).

[0095]

[0096] Using the zero miss conversion and the Schwarz inequality extension lemma, we can get that there are N Lagrange multipliers λ i , i = 1, 2, ..., N, so that the lateral acceleration a is optimal, and the lateral acceleration a satisfies the calculation rule shown in formula (22).

[0097]

[0098] Among them, the Lagrange multiplier vector λ=[λ1,λ2,…,λ N ] T It can be solved analytically, k1, k2, ..., k N is the guidance law coefficient.

[0099] Using the guidance instruction a, the flight trajectory for obstacle avoidance can be obtained through numerical integration simulation, and the performance indicators can be calculated.

[0100] In actual flight, the trained strategy network is reused at the beginning of each information update cycle to update the obstacle position information, make real-time decisions on the obstacle avoidance deflection angle, and generate guidance instructions based on the underlying trajectory generation method to simulate the obstacle avoidance trajectory. The real-time application framework of aircraft intelligent decision-making and trajectory generation is as follows: Figure 6 shown.

[0101] The present application also provides an application scenario, which applies the above-mentioned target aircraft intelligent decision-making and trajectory generation method. Specifically: the target aircraft intelligent decision-making and trajectory generation method provided in this embodiment, for the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft, the server 104 converts the upper-level intelligent decision into a single-step Markov decision process based on the reinforcement learning method, and inputs the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy, and inputs the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, and obtains the obstacle avoidance trajectory corresponding to the current decision strategy, and finally obtains the optimal obstacle avoidance decision strategy and the optimal obstacle avoidance trajectory. The target aircraft intelligent decision-making and trajectory generation method provided in this embodiment belongs to the link of aircraft intelligent decision-making and trajectory generation. Specifically, when the aircraft intelligent decision-making and trajectory generation are generated, obstacle avoidance can be performed based on the upper-level intelligent decision and the lower-level trajectory generation.

[0102] In an exemplary embodiment, the present application provides an aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system, including the following modules.

[0103] The aircraft obstacle avoidance problem model establishment module is used to: use a three-degree-of-freedom dynamic model to describe the movement of the aircraft, establish process constraints, obstacle constraints, terminal constraints and performance indicators, and form an aircraft obstacle avoidance problem model; the obstacle constraint is a geometric model in which the obstacle changes dynamically with the aircraft; the performance indicator is the minimum cumulative lateral control force.

[0104] The upper-level intelligent decision conversion module is used to: convert the upper-level intelligent decision into a single-step Markov decision process based on the reinforcement learning method, and define the state set, behavior set and reward function corresponding to the upper-level intelligent decision.

[0105] The current decision strategy determination module is used to: input the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy; the current decision strategy is the deflection angle of the target aircraft.

[0106] The obstacle avoidance trajectory planning module is used to: input the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, and obtain the obstacle avoidance trajectory corresponding to the current decision strategy.

[0107] The intelligent decision-making and trajectory generation module is used to: repeat the steps of "inputting the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy" to the step of "inputting the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, and obtaining the obstacle avoidance trajectory corresponding to the current decision strategy" to obtain the optimal obstacle avoidance decision strategy and the optimal obstacle avoidance trajectory; the optimal obstacle avoidance decision strategy includes the deflection angle of the target aircraft in each information update cycle.

[0108] The single-step Markov decision process is expressed by the following formula.

[0109] The state set is represented as: in,

[0110] In the formula, x represents the flight state of the target aircraft; r is the radial distance from the center of the earth to the center of gravity of the target aircraft; θ and are the geographical longitude and geocentric latitude of the target aircraft, respectively; V is the speed of the target aircraft relative to the earth; ψ and γ are the track deviation and track inclination of the target aircraft, respectively; is the initial obstacle position; r o is the radius.

[0111] The behavior set is represented as: A = [Δψ1; Δψ2; ...; Δψ n ].

[0112] In the formula, Δψ iis the i-th discretized deflection angle, i=1, 2,…, n is the discretized deflection angle level, and n is the number of discrete points.

[0113] The reward function is expressed as:

[0114] Where J is the minimum cumulative lateral control force; t0 and t f are the initial moment and the terminal moment respectively, and a is the lateral acceleration.

[0115] The trained strategy network is obtained by training the strategy network using the REINFORCE algorithm; the aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system also includes a training module, and the training module is used to perform the following steps.

[0116] 1) Using neural network π θ Fit the optimal decision strategy, where θ is the neural network parameter to be optimized; network π θ The output is the probability p of selecting each deflection angle level θ (A|S).

[0117] 2) Define the loss function is the expected value of reward r.

[0118] 3) To update the gradient of the neural network parameter θ, the REINFORCE algorithm is used to estimate the gradient until the iteration stop condition is met to obtain a trained policy network.

[0119] The obstacle avoidance trajectory planning module includes an obstacle avoidance unit and an obstacle avoidance trajectory planning unit.

[0120] The obstacle avoidance unit is used to obtain a deflection angle according to the current decision strategy to avoid dynamically changing obstacles.

[0121] The obstacle avoidance trajectory planning unit is used to: transform the lower-level trajectory generation into a path point tracking guidance problem, take minimizing the energy consumption of path point tracking as the optimization goal, design the optimal guidance instruction, and simulate the obstacle avoidance trajectory corresponding to the current decision strategy; the optimal guidance instruction is the lateral acceleration; the path point is tangent to the obstacle boundary.

[0122] Wherein, the obstacle avoidance unit includes an obstacle center position calculation subunit and a dynamic change obstacle avoidance subunit.

[0123] The obstacle center position calculation subunit is used to calculate the obstacle center position after dynamic changes according to the deflection angle.

[0124] The dynamically changed obstacle avoidance subunit is used to: calculate the dynamically changed obstacle radius according to the deflection angle; and avoid the dynamically changed obstacle according to the dynamically changed obstacle center position and the dynamically changed obstacle radius.

[0125] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An intelligent decision-making and trajectory generation method for dynamic obstacle avoidance of an aircraft, characterized in that: include: Step 201: A three-degree-of-freedom dynamics model is used to describe the motion of the aircraft, and process constraints, obstacle constraints, terminal constraints and performance indicators are established to form an aircraft obstacle avoidance problem model; the obstacle constraint is a geometric model of the obstacle changing dynamically with the aircraft; the performance indicator is the minimum cumulative lateral control force; Step 202: Based on the reinforcement learning method, the upper-level intelligent decision is converted into a single-step Markov decision process, and the state set, behavior set and reward function corresponding to the upper-level intelligent decision are defined; Step 203: input the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy; The current decision strategy is the deflection angle of the target aircraft; Step 204: input the current decision strategy into the lower-layer trajectory generation to perform obstacle avoidance trajectory planning, and obtain the obstacle avoidance trajectory corresponding to the current decision strategy; Step 205: Repeat steps 203 to 204 to obtain an optimal obstacle avoidance decision strategy and an optimal obstacle avoidance trajectory; the optimal obstacle avoidance decision strategy includes a deflection angle of the target aircraft in each information update cycle.

2. The method for intelligent decision-making and trajectory generation for dynamic obstacle avoidance of an aircraft according to claim 1, characterized in that: The single-step Markov decision process is expressed by the following formula: The state set is represented as: in, In the formula, x represents the flight state of the target aircraft; r is the radial distance from the center of the earth to the center of gravity of the target aircraft; θ and are the geographical longitude and geocentric latitude of the target aircraft, respectively; V is the speed of the target aircraft relative to the earth; ψ and γ are the track deviation and track inclination of the target aircraft, respectively; is the initial obstacle position; r o is the radius; The behavior set is represented as: A=[Δψ1;Δψ2;...;Δψ n ]; In the formula, Δψ i is the i-th discretized deflection angle, i = 1, 2, ..., n is the discretized deflection angle level, and n is the number of discrete points; The reward function is expressed as: Where J is the minimum cumulative lateral control force; t0 and t f are the initial moment and the terminal moment respectively, and a is the lateral acceleration.

3. The method for intelligent decision-making and trajectory generation for dynamic obstacle avoidance of an aircraft according to claim 1, characterized in that: The trained policy network is trained using the REINFORCE algorithm; the training process of the policy network is as follows: Using neural network π θ Fit the optimal decision strategy, where θ is the neural network parameter to be optimized; network π θ The output is the probability p of selecting each deflection angle level θ (A|S); Define the loss function is the expected value of reward r; In order to update the gradient of the neural network parameter θ, the REINFORCE algorithm is used to estimate the gradient until the iteration stop condition is met to obtain a trained policy network.

4. The method for intelligent decision-making and trajectory generation for dynamic obstacle avoidance of an aircraft according to claim 1, characterized in that: Step 204 specifically includes: Obtain the deflection angle based on the current decision-making strategy to avoid obstacles after dynamic changes; The lower-level trajectory generation is transformed into a path point tracking guidance problem. Taking minimizing the energy consumption of path point tracking as the optimization goal, the optimal guidance instruction is designed, and the obstacle avoidance trajectory corresponding to the current decision strategy is simulated; the optimal guidance instruction is the lateral acceleration; the path point is tangent to the obstacle boundary.

5. The method for intelligent decision-making and trajectory generation for dynamic obstacle avoidance of an aircraft according to claim 4, characterized in that: Obtain the deflection angle based on the current decision strategy to avoid obstacles after dynamic changes, including: Calculate the obstacle center position after dynamic changes according to the deflection angle; The radius of the dynamically changing obstacle is calculated according to the deflection angle; the dynamically changing obstacle is avoided according to the center position of the dynamically changing obstacle circle and the dynamically changing obstacle radius.

6. An intelligent decision-making and trajectory generation system for aircraft dynamic obstacle avoidance, characterized in that: include: The aircraft obstacle avoidance problem model establishment module is used to: use a three-degree-of-freedom dynamics model to describe the movement of the aircraft, establish process constraints, obstacle constraints, terminal constraints and performance indicators, and form an aircraft obstacle avoidance problem model; the obstacle constraint is a geometric model of the obstacle changing with the dynamic changes of the aircraft; the performance indicator is the minimum cumulative lateral control force; The upper-level intelligent decision conversion module is used to: convert the upper-level intelligent decision into a single-step Markov decision process based on the reinforcement learning method, and define the state set, behavior set and reward function corresponding to the upper-level intelligent decision; The current decision strategy determination module is used to: input the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy; The current decision strategy is the deflection angle of the target aircraft; The obstacle avoidance trajectory planning module is used to: input the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, and obtain the obstacle avoidance trajectory corresponding to the current decision strategy; The intelligent decision-making and trajectory generation module is used to: repeat the steps of "inputting the real-time flight status, real-time obstacle position and current obstacle radius of the target aircraft into the trained strategy network to obtain the current decision strategy" to the step of "inputting the current decision strategy into the lower-level trajectory generation to perform obstacle avoidance trajectory planning, and obtaining the obstacle avoidance trajectory corresponding to the current decision strategy", and obtain the optimal obstacle avoidance decision strategy and the optimal obstacle avoidance trajectory; the optimal obstacle avoidance decision strategy includes the deflection angle of the target aircraft in each information update cycle.

7. The aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system according to claim 6, characterized in that: The single-step Markov decision process is expressed by the following formula: The state set is represented as: in, In the formula, x represents the flight state of the target aircraft; r is the radial distance from the center of the earth to the center of gravity of the target aircraft; θ and are the geographical longitude and geocentric latitude of the target aircraft, respectively; V is the speed of the target aircraft relative to the earth; ψ and γ are the track deviation and track inclination of the target aircraft, respectively; is the initial obstacle position; r o is the radius; The behavior set is represented as: A=[Δψ1;Δψ2;...;Δψ n ]; In the formula, Δψ i is the i-th discretized deflection angle, i = 1, 2, ..., n is the discretized deflection angle level, and n is the number of discrete points; The reward function is expressed as: Where J is the minimum cumulative lateral control force; t0 and t f are the initial moment and the terminal moment respectively, and a is the lateral acceleration.

8. The aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system according to claim 6, characterized in that: The trained policy network is obtained by training the policy network using the REINFORCE algorithm; the aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system also includes a training module, and the training module is used to: Using neural network π θ Fit the optimal decision strategy, where θ is the neural network parameter to be optimized; network π θ The output is the probability p of selecting each deflection angle level θ (A|S); Define the loss function is the expected value of reward r; In order to update the gradient of the neural network parameter θ, the REINFORCE algorithm is used to estimate the gradient until the iteration stop condition is met to obtain a trained policy network.

9. The aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system according to claim 6, characterized in that: The obstacle avoidance trajectory planning module includes: The obstacle avoidance unit is used to: obtain the deflection angle according to the current decision strategy to avoid dynamically changing obstacles; The obstacle avoidance trajectory planning unit is used to: transform the lower-level trajectory generation into a path point tracking guidance problem, take minimizing the energy consumption of path point tracking as the optimization goal, design the optimal guidance instruction, and simulate the obstacle avoidance trajectory corresponding to the current decision strategy; the optimal guidance instruction is the lateral acceleration; the path point is tangent to the obstacle boundary.

10. The aircraft dynamic obstacle avoidance intelligent decision-making and trajectory generation system according to claim 9, characterized in that: The obstacle avoidance unit comprises: The obstacle circle center position calculation subunit is used to calculate the obstacle circle center position after dynamic change according to the deflection angle; The dynamically changed obstacle avoidance subunit is used to: calculate the dynamically changed obstacle radius according to the deflection angle; and avoid the dynamically changed obstacle according to the dynamically changed obstacle center position and the dynamically changed obstacle radius.

Citation Information

Cited By

  • Unmanned aerial vehicle evasion strategy determination method and device based on reinforcement learning, equipment and medium

    CN119861744A

  • Time-optimal spacecraft attitude trajectory planning method and system under time-varying constraint

    CN120233790A