Aircraft cluster game confrontation online optimization decision-making method based on pseudo-common method

Abstract

The invention provides a rolling optimization method based on a pseudo-spectral method. The rolling optimization method is used for collaborative obstacle avoidance of an aircraft cluster and online autonomous decision making of approaching a target. With the expansion of an aircraft application scene, a traditional optimization method is difficult to deal with a complex flight environment. According to the method, online optimization is realized by establishing an optimization constraint model and comprehensively considering angle and time constraints, so that the decision-making efficiency and task execution capability of an aircraft cluster are remarkably improved, and powerful support is provided for development of an aircraft autonomous control technology.

Description

Technical Field

[0001] The present invention relates to the technical field of cooperative autonomous flight of unmanned aerial vehicle clusters, and in particular proposes an online autonomous decision-making method for cooperative obstacle avoidance and target approach of unmanned aerial vehicle clusters using a pseudo-spectral method combining the idea of ​​rolling time domain. Background Art

[0002] Aircraft technology is developing rapidly. Its features of no need for human piloting, flexibility, and high cost-effectiveness have made it widely used in many fields such as military, agriculture, and logistics. With the continuous advancement of intelligent and automated technologies, the remote control and autonomous control systems of aircraft are becoming more and more perfect, further broadening their application scenarios and value. However, with the improvement of control technology, the flight environment faced by aircraft is becoming increasingly complex. In order to better adapt to the execution of tasks in diverse scenarios, the research on autonomous control flight technology of aircraft has become the focus of global attention.

[0003] With the continuous expansion of aircraft application scenarios, their flight environment is becoming increasingly complex, which places higher demands on the aircraft's ability to perform tasks. Simply improving the autonomous decision-making ability of a single aircraft is no longer able to cope with the increasingly severe mission challenges. In this context, aircraft collaborative flight has shown significant advantages and can complete tasks more effectively, so this research direction has received increasing attention. However, the optimization problem involved in multi-aircraft collaborative flight is no longer limited to the state of a single aircraft, but has become more complex. The optimization process takes a long time and is easily affected by factors such as dynamic environment, communication limitations, computational complexity, and safety.

[0004] In order to effectively solve the online optimization problem of rapid autonomous decision-making during coordinated flight of aircraft clusters, which contains complex constraints, improve the coordination between aircraft clusters, and ensure the task completion rate in specific scenarios, the research on the rolling optimization method based on pseudo-spectral method for autonomous decision-making of coordinated obstacle avoidance and approaching targets of aircraft clusters will provide strong support for improving the research on aircraft cluster problems.

[0005] The current methods are:

[0006] 1) UAV swarm flight path planning based on pseudo-spectral method [J]. Li Zheng, Chen Jianwei, Peng Bo. Aerospace Defense, 2021, 4(01): 52-59. This study proposes a UAV swarm path planning method based on Radau pseudo-spectral method, aiming to solve the path planning problem in multi-UAV collaborative flight. This method first establishes the UAV dynamics model, considers the constraints in the swarm flight process, and transforms them into nonlinear constraints. Then, the performance indicators of path planning are determined, and the optimal control model of UAV swarm path planning is established. Finally, the Radau pseudo-spectral method is used to solve the optimal control problem, and the feasibility and effectiveness of this method are verified by simulation. The simulation results show that this method can obtain a flight trajectory that satisfies state constraints, path constraints and control constraints, meets the requirements of multi-UAV collaborative flight, and has strong practical value. However, its disadvantages are: (1) This method performs RPM optimization on the entire motion route and cannot achieve the effect of online optimization (2) No more constraints are established, such as angle constraints, time constraints, etc., and the UAV swarm approaching the target is not effective.

[0007] 2) A Q-learning-based collaborative trajectory planning method for multiple UAVs[J]. Yin Yiyi, Wang Xiaofang, Zhou Jian. Acta Ordnance, 2023, 44(02): 484-495. This study proposes a Q-learning-based solution to the trajectory planning problem of multiple UAVs reaching the target at the same time. This solution uses the Q-learning algorithm to find the optimal trajectory with the shortest range by establishing a battlefield environment model and a Markov decision model for trajectory planning of a single UAV. Then, the experience matrix obtained by the Q-learning algorithm is used to quickly solve the shortest trajectory of each UAV, and the trajectory group that satisfies time coordination is obtained by adjusting the action selection strategy of the bypassing UAV. In order to solve the collision problem between multiple UAVs, this method designs a backoff parameter to determine the local replanning area, and based on the deep Q-learning theory, a neural network is used to replace the Q-table to replan the local trajectory, avoiding the dimensionality explosion problem. In addition, this method also designs the obstacle Q matrix with reference to the idea of ​​the artificial potential field method, and superimposes it on the original Q matrix, realizing the collision avoidance of UAVs against undetected obstacles. The simulation results show that this method can obtain a coordinated trajectory that satisfies time coordination and collision avoidance, and can avoid obstacles that are not discovered during environment modeling. (1) When there are a large number of drones, the training time and space complexity of the Q-learning algorithm are still high. (2) The collision detection module and local replanning method are relatively simple and cannot handle complex collision scenarios and dynamic obstacles. (3) Without establishing more constraints, such as angle constraints, time constraints, etc., the drone cluster is not effective in approaching the target. Summary of the invention

[0008] Aiming at the technical field of coordinated autonomous flight of aircraft clusters, the present invention proposes a pseudo-spectral method combined with the rolling time domain idea to make online autonomous decisions for coordinated obstacle avoidance and target approach of aircraft clusters. This method establishes an optimal control problem based on pseudo-spectral optimization, and designs the corresponding objective function, taking into account factors such as angle constraints and time constraints. Through rolling optimization, online optimization of aircraft formations is achieved, thereby significantly improving the efficiency of autonomous decision-making for coordinated obstacle avoidance and target approach of aircraft clusters, which specifically includes the following steps:

[0009] S1. Obtain the relevant parameters required for the aircraft cluster to perform the target task, and define the state update expression of the aircraft cluster as:

[0010] In the initial stage, determine the number of aircraft clusters performing the mission, the number of enemy interceptors, the number, position and radius of threat zones in the environment, the position of the target point, the initial position and initial state of the aircraft cluster, the angle of attack setting of the aircraft cluster collaboratively approaching the target, etc. Then define the aircraft cluster state update expression as follows.

[0011] Select the ground coordinate system (O g x g y g z g ) Choose any spatial point O on the ground g , select any direction of the horizontal plane as O g x g The direction of the axis, O g z g The direction of the axis is perpendicular to the horizontal plane and upward. g y g Perpendicular to O g x g z g plane, whose orientation is determined by the right-hand rule.

[0012] Consider a group of N fixed-wing aircraft, and the kinematic model of the i-th fixed-wing aircraft is (O g x g y g z g ) can be expressed in the ground coordinate system as:

[0013]

[0014] Among them, (x i ,y i ,z i ) is the position coordinate of the UAV in the ground coordinate system, n i Represents the overload factor of the drone, n x，i represents normal overload, g is the acceleration due to gravity, c is i is the track deflection angle, γi is the track inclination angle, φ i is the velocity roll angle.

[0015] Based on this kinematic model, the state update equation of the aircraft cluster can be expressed. From the above model, it can be seen that the state vector of the i-th aircraft can be expressed as follows:

[0016] x i (t) = [x i ,y i ,x i ,v i ,c i ,γ i ] T

[0017] The input control quantity vector is:

[0018] u i (t) = [n xi ,n i ,φ i ] T

[0019] Therefore, the state update expression of the aircraft cluster at any time can be expressed as follows:

[0020]

[0021] S2. The optimal control problem of establishing a cluster of aircraft is as follows:

[0022]

[0023] x i (0) = x i 0

[0024] C x (x i (t))≤0

[0025] C u (u i (t))≤0

[0026] ε>0

[0027]

[0028] Among them, J p (x i (t),u i (t),ε) is the augmented objective function after using the exact penalty function to process the relevant constraints; Update expression for the status of the aircraft cluster; x i(0) = x i 0 is the initial state constraint of the initial cluster; C x (x i (t))≤0 and C u (u i (t))≤0 are the boundary constraints of the state quantity and the control quantity respectively; ε>0 is the range constraint of the relaxed decision variable; In order to optimize the time domain range of the solution, the derivation of these quantities is introduced in detail below.

[0029] Since the collaborative trajectory planning problem of aircraft clusters contains complex nonlinear constraints, the computational dimensions of conventional solution methods are very large, and the solution speed of the problem is very slow. In order to improve the solution efficiency of the trajectory planning problem, the present invention introduces an accurate penalty function method to deal with the nonlinear constraints of the trajectory planning problem of drone clusters. Specifically, the threat zone constraint, distance constraint and interceptor avoidance constraint of the aircraft are augmented into the objective function, that is, J in the above p (x i (t),u i (t),ε) term, which is specifically expressed as follows:

[0030]

[0031] g i,j (x i (t)) = D s -d i,j (t)≤0

[0032] l i,j (x i (t)) = d i,j (t)-D c ≤0

[0033] r k (x i (t)) = R threat,k -||(x i (t)-x threat,k ),(y i (t)-y threat,k )||≤0; k=1,2,…,S

[0034] q k (x i (t)) = D safe -||p i (t)-p IC,k (t)||≤0; k=1,2,…,M

[0035] In the formula, α>0, β>2, m>0, s∈(0,1) is a constant, and δ>0 is a penalty coefficient. Obviously, when the degree of state constraint dissatisfaction and the time interval are larger, the values ​​of △1, △2, and △3 are larger. i,j (x i (t)) is the minimum safe distance constraint between any two aircraft in the aircraft cluster, l i,j (x i (t)) is the maximum communication distance constraint between any two aircraft in the aircraft cluster, D s is the minimum safe distance between the two aircraft, D c is the maximum communication distance between the two machines, d i,j (t)=||p i (t)-p j (t)|| represents the distance between the current aircraft and the jth aircraft. k (x i (t)) is the threat zone constraint, (x thraet,k ,y threat,k ) represents the coordinates of the center of the kth threat zone, R threat,k is the radius of the threat zone plane circle, and S is the number of threat zones. k (x i (t)) is the interceptor constraint, p IC,k (t) represents the current position of the kth interceptor, p i (t) represents the current position of the aircraft, D safe It represents the minimum safe distance between the aircraft and the interceptor, N is the number of aircraft in the aircraft cluster, and M is the number of interceptors.

[0036] It can be seen that in the augmented objective function, when the constraints are fully satisfied, the optimal solution of the augmented objective function after the exact penalty function is the original problem J(x i (t),u i (t)) is the optimal solution. i (t),u i (t)) includes three optimization items: approaching the target, time and angle coordination, which are expressed as follows:

[0037] J(x i (t),u i (t)) = κ1J1+κ2J2+κ3J3

[0038]

[0039] Among them, J1 is the optimization item approaching the target, (x d ,y d ,z d) represents the desired target point position coordinates, ||…|| represents the Euclidean distance between two points. J2 is the optimization term of time coordination, p i (t), p j (t), p d They represent the current position of the aircraft, the position of the aircraft, and the position of the target point. J3 is the optimization item of angle coordination, c i represents the angle between the i-th aircraft and the target vector and the xy plane, γ i represents the angle between the projection of the i-th aircraft to the target vector on the xy plane and the x-axis, represents the expected angle between the i-th aircraft and the target vector and the xy plane, It represents the expected angle between the projection of the i-th aircraft to the target vector on the xy plane and the x-axis. κ1, κ2, κ3 are the weight coefficients of J1, J2, and J3 respectively.

[0040] The differential equations in the optimal control problem are established It has been defined and explained in S1, x i 0 is the state quantity of the initial cluster given by S1. x (x i (t))≤0 and C u (u i The boundary constraints of these two state quantities and control quantities are specifically expressed as follows:

[0041]

[0042] Among them, (z min ,z max )、(v min ,v max )、(g min ,γ max )、(c min ,c max ) represent the boundary ranges of the aircraft’s flight altitude, flight speed, track deviation angle, and track inclination angle, respectively. (n x,min ,n x,max )、(n min ,n max )、(φ min ,φ max ) represent the boundary ranges of the tangential overload, normal overload and roll angle of the aircraft respectively.

[0043] S3. Solve the optimization problem based on the pseudo-spectral method combined with the rolling time domain, specifically:

[0044] (1) Select the number of nodes N, the initial time t0, and the initial state x0 of the pseudo-spectral discrete interpolation method, and use the Radau pseudo-spectral method to calculate the open-loop optimal control in the 4 s time domain [t0, t0+4]

[0045] (2) In the interval [t0, t1], the optimal control quantity Applied to the system, the state measurement value at time t1 is Let i = 1;

[0046] (3) In the interval [t i ,t i+1 ], the open loop optimal control Applied to the system; at the same time, according to t i The state measurement value x(t i ), calculate the open-loop optimal control Assume △t i To optimize the computation time, t i+1 =t i +△t i ;

[0047] (4) Let i=i+1 and return to (3).

[0048] The aircraft is continuously controlled through the above-mentioned rolling time domain update method until the mission is completed or the aircraft dies. In the above-mentioned update process, the Radau pseudo-spectral method is used to solve the optimal control problem established previously. The solution process of the Radau pseudo-spectral method is introduced below:

[0049] First, for the time interval [t0,t f ] state, and then use the following formula to map it to the interval [-1, 1] to optimize the problem solution. The mapping formula is as follows:

[0050]

[0051] Then, by discretizing the control quantity and state quantity using LGR points and using Lagrange polynomial interpolation, the approximate functions of the state quantity and control quantity can be obtained, where the basis function of Lagrange polynomial interpolation is:

[0052]

[0053] The approximate expressions of state quantity and control quantity are:

[0054]

[0055] Among them, X i and U iAfter the above-mentioned time transformation, x(t) and u(t) are discretized into a series of points using LGR points on τ∈[-1,1], and i=1,2,…,N.

[0056] Taking the derivative of the state variable x(τ) we get

[0057]

[0058] The derivative of the state quantity x(τ) can also be expressed in another form through the differential equation established in S1:

[0059]

[0060] After the above method is used, the state differential equation in the S2 optimal control problem can be expressed as the following equality constraint:

[0061]

[0062]

[0063] j=0,1,…,N

[0064] Therefore, the pseudo-spectral method converts the optimal control problem of S2 into a nonlinear programming problem through this discrete method. There are many mature methods for solving nonlinear programming problems. Using the SNOPT package to solve, the solution to the optimal control problem proposed by S2 can be obtained, that is, the optimal control quantity u * (t). Apply the solved optimal control quantity to the aircraft system. The nonlinear programming problem transformed by the above method can be described as follows:

[0065]

[0066] x(0)=x 0

[0067] C u (U i )≤0,i=0,1,…,N

[0068] C x (X i )≤0,i=0,1,…,N

[0069] ε>0

[0070] This continuous update process of open-loop optimal control ensures that the algorithm has real-time and online application capabilities, and uses rolling optimization to achieve the effect of online optimization of the aircraft group. Figure 1 shown.

[0071] S4, state perception and update:

[0072] In S3, the solution algorithm and update iteration framework for the optimal control problem are introduced to complete the given task. It can be seen that after solving the optimal control problem proposed in S2, the solution to the problem is obtained, that is, the optimal control quantity, which is applied to the aircraft system and the solution is updated in the rolling time domain until the task is completed. Therefore, it is necessary to add a judgment on whether the task is completed in the algorithm. If the task has been completed, the algorithm ends; if the task is not completed in the current optimization update, the current state of the aircraft is detected, and the temporary information and environmental information perception are updated. This information is used as the initial setting of S2, and the optimal control problem is solved again. The overall flow chart of the solution is shown below. Figure 2 shown.

[0073] Beneficial effects of the present invention:

[0074] Based on the rolling time domain pseudo-spectral method, the present invention realizes the online optimization of coordinated obstacle avoidance and approaching targets of aircraft clusters, significantly improving the efficiency of autonomous decision-making. By optimizing the optimal control problem through pseudo-spectral method, multiple constraints such as angle and time are integrated to enhance the control accuracy and dynamic response capability of aircraft clusters, thereby improving the adaptability and safety of coordinated flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 Schematic diagram of the continuous application process of open-loop optimal control.

[0076] Figure 2 This is the flow chart of trajectory optimization of aircraft clusters based on pseudo-spectral method of rolling time domain.

[0077] Figure 3 This is the process of aircraft cluster approaching.

[0078] Figure 4 This is the aircraft overload curve.

[0079] Figure 5 is the aircraft speed curve. DETAILED DESCRIPTION

[0080] The present invention uses the aircraft motion model mentioned in S1 to verify the pseudo-spectral method in the rolling time domain mentioned above. Considering five aircrafts collaboratively approaching a static target point and avoiding the opponent's interceptor during the approach process as an example, the algorithm is used to plan the approach trajectory online and complete the relevant collaborative actions. Figure 3-5 shown.

Claims

1. An online optimization decision-making method for aircraft cluster game confrontation based on pseudo-legal popularization, characterized by: The following steps are involved: S1. Obtain the relevant parameters required for the aircraft group to perform the target mission, and define the state update expression of the aircraft group at any time as: Among them, x(t) is the state vector of the aircraft group, and u(t) is the control vector of the aircraft group; S2. The optimal control problem of establishing a cluster of aircraft is as follows: Among them, J p (x i (t),u i (t),ε) is the augmented objective function after using the exact penalty function to process the relevant constraints; is the initial state constraint of the initial cluster; C x (x i (t))≤0 and C u (u i (t))≤0 are the boundary constraints of the state quantity and the control quantity respectively; ε>0 is the range constraint of the relaxation decision variable; The time domain scope for optimization solution; J p (x i (t),u i The specific expression of (t),ε) is: in, g i,j (x i (t))=D s -d i,j (t)≤0 l i,j (x i (t))=d i,j (t)-D c ≤0 r k (x i (t))=R threat,k -||(x i (t)-x threat,k ),(y i (t)-y threat,k )||≤0;k=1,2,…,S q k (x i (t))=D safe -||p i (t)-p IC,k (t)||≤0;k=1,2,…,M Among them, α>0, β>2, m>0, σ∈(0,1) is a constant, δ>0 is a penalty coefficient, g i,j (x i (t)) is the minimum safe distance constraint between any two aircraft in the aircraft cluster, l i,j (x i (t)) is the maximum communication distance constraint between any two aircraft in the aircraft cluster, D s is the minimum safe distance between the two aircraft, D c is the maximum communication distance between the two machines, d i,j (t)=||p i (t)-p j (t)|| represents the distance between the current aircraft and the jth aircraft, r k (x i (t)) is the threat zone constraint, (x thraet,k ,y threat,k ) represents the coordinates of the center of the kth threat zone, R threat,k is the radius of the threat zone plane circle, S is the number of threat zones, q k (x i (t)) is the interceptor constraint, p IC,k (t) represents the current position of the kth interceptor, p i (t) represents the current position of the aircraft, D safe It represents the minimum safe distance between the aircraft and the interceptor, N is the number of aircraft in the aircraft cluster, and M is the number of interceptors; J(x i (t),u i (t)) contains three optimization items: approaching the target, time and angle coordination, which can be expressed as: Among them, J1 is the optimization item approaching the target, (x d ,y d ,z d ) represents the desired target point position coordinates, ||…|| represents the Euclidean distance between two points; J2 is the optimization term of time coordination, p i (t), p j (t), p d They represent the current position of the aircraft, the position of the aircraft on the spot, and the position of the target point respectively; J3 is the optimization item of angle coordination, χ i represents the angle between the i-th aircraft and the target vector and the xy plane, γ i represents the angle between the projection of the i-th aircraft to the target vector on the xy plane and the x-axis, represents the expected angle between the i-th aircraft and the target vector and the xy plane, represents the expected angle between the projection of the i-th aircraft to the target vector on the xy plane and the x-axis; κ1, κ2, κ3 are the weight coefficients of J1, J2, and J3 respectively; S3. Solve the optimal control problem based on the pseudo-spectral method combined with the idea of ​​rolling time domain, specifically: (1) Select the number of nodes N, the initial time t0, and the initial state x0 of the pseudo-spectral discrete interpolation method, and use the Radau pseudo-spectral method to calculate the open-loop optimal control in the 4 s time domain [t0, t0+4] (2) In the interval [t0, t1], the optimal control quantity Applied to the system, the state measurement value at time t1 is Let i = 1; (3) In the interval [t i ,t i+1 ], the open loop optimal control Applied to the system; at the same time, according to t i The state measurement value x(t i ), calculate the open-loop optimal control Assume △t i To optimize the computation time, t i+1 =t i +△t i ; (4) Let i=i+1 and return to (3); The aircraft is continuously controlled by the above-mentioned rolling time domain update method until the mission is completed or the aircraft dies. In the above-mentioned update process, the Radau pseudospectral method is used to solve the established optimal control problem. The specific solution process is: First, for the time interval [t0,t f ], and then use the following formula to map it to the interval [-1, 1] to optimize the problem solution. The mapping formula is as follows: Then, by discretizing the control quantity and state quantity using LGR points and using Lagrange polynomial interpolation, the approximate functions of the state quantity and control quantity can be obtained, where the basis function of Lagrange polynomial interpolation is: The approximate expressions of state quantity and control quantity are: Among them, X i and U i x(t) and u(t) are respectively a series of points discretized using LGR points at t∈[-1,1] after the above time transformation, and i=1,2,…,N; x(t) is derived: The derivative of the state quantity x(t) can also be expressed in another form through the differential equation established in S1: The above two derivatives of the state quantity x(t) are approximately equivalent, so the following equality constraint can be established: The pseudo-spectral method converts the optimal control problem of S2 into a nonlinear programming problem through this discrete method. The solution of the optimal control problem proposed by S2 can be obtained by using the SNOPT package, that is, the optimal control quantity u * (t); Apply the solved optimal control quantity to the aircraft system; the nonlinear programming problem is expressed as: S4, state perception and update: After obtaining the solution to the optimal control problem through S3, determine whether the target mission of the aircraft cluster is completed. If so, end; otherwise, detect the current state of the aircraft, and obtain the temporary information and environmental information perception update, use this information as the initial setting, and re-solve the optimal control problem.

Images