A multi-aircraft cooperative guidance method based on MAPPO algorithm

Abstract

The present invention relates to a multi-aircraft collaborative guidance method based on the MAPPO algorithm, comprising: establishing a multi-aircraft collaborative guidance model, the multi-aircraft collaborative guidance model including a dynamic model of each aircraft, a process constraint model, a terminal constraint model, and a roll angle amplitude constraint model; when the remaining flight range is greater than or equal to 500 km, designing a collaborative guidance law based on an improved prediction correction to solve the aircraft's angle of attack and roll angle guidance instructions; when the remaining range is less than 500 km, switching to a collaborative guidance law based on the MAPPO algorithm to solve the aircraft's angle of attack and roll angle guidance instructions. The present invention can accurately reach the mid-terminal handover point within a large time difference and successfully complete the time coordination task. The present invention has strong robustness against state disturbances, a larger time coordination range, and better time coordination guidance capabilities.

Description

Technical Field

[0001] The invention belongs to the field of aircraft guidance and control, and relates to a multi-aircraft collaborative guidance method based on a MAPPO algorithm. Background Art

[0002] In recent years, the rapid development of hypersonic vehicles has attracted widespread attention. Thanks to their advantages, such as long range, high speed, and diverse and unpredictable trajectories, they hold enormous potential for military applications. Countries like the United States, Russia, Germany, and the United Kingdom have implemented numerous research programs and achieved a series of results. However, in the face of increasingly sophisticated anti-missile defense systems, traditional single-vehicle operations are no longer sufficient for combat missions in complex battlefield environments. Multi-vehicle coordinated operations are poised to become a defining operational strategy in future wars.

[0003] Multi-vehicle collaboration refers to the coordinated flight of multiple homogeneous or heterogeneous aircraft in formation, both in time and space, according to mission requirements. Currently, research on multi-vehicle collaboration focuses primarily on drone swarms, while research on hypersonic vehicle collaboration is relatively limited. Therefore, integrating the strengths of hypersonic aircraft to conduct research on multi-vehicle collaborative guidance is crucial for the rapid advancement of hypersonic aircraft. Summary of the Invention

[0004] The technical problem solved by the present invention is to overcome the deficiencies of the prior art and propose a multi-aircraft collaborative guidance method based on the MAPPO algorithm.

[0005] The solution of the present invention is:

[0006] A multi-aircraft cooperative guidance method based on the MAPPO algorithm, comprising:

[0007] Establishing a multi-aircraft collaborative guidance model, wherein the multi-aircraft collaborative guidance model includes a dynamic model of each aircraft, a process constraint model, a terminal constraint model, and a roll angle amplitude constraint model;

[0008] When the remaining flight range is greater than or equal to 500 km, a collaborative guidance law based on improved prediction and correction is designed to solve the aircraft's angle of attack and bank angle guidance instructions:

[0009] When the remaining range is less than 500km, the system switches to the collaborative guidance law based on the MAPPO algorithm to solve the aircraft's angle of attack and roll angle guidance instructions.

[0010] Preferably, in multiple aircraft, the dynamic model of each aircraft is as follows:

[0011]

[0012] Where r is the distance from the center of the Earth to the aircraft, θ and φ represent the longitude and latitude of the aircraft, respectively, V is the aircraft velocity, γ and ψ are the track angle and heading angle of the aircraft, respectively, σ is the roll angle of the aircraft, m is the mass of the aircraft, g is the acceleration due to gravity, and L and D are the lift and drag of the aircraft, respectively.

[0013] Preferably, the process constraint model of each aircraft includes a heat flow rate constraint model, an overload constraint model, a dynamic pressure constraint model, and a quasi-equilibrium gliding condition constraint model;

[0014] The heat flow rate constraint model is as follows:

[0015]

[0016] Where, represents the heat flux rate at the stagnation point of the aircraft; k Q is the heat flow rate model coefficient; is the maximum allowable heat flux rate at the aircraft stagnation point, ρ is the atmospheric density at the aircraft's altitude, and V is the aircraft's speed;

[0017] The overload constraint model is as follows:

[0018]

[0019] Where n is the aircraft overload; n max is the maximum allowable overload; g0 is the gravitational acceleration at sea level, m is the mass of the aircraft, g is the acceleration due to gravity, L and D are the lift and drag of the aircraft respectively;

[0020] The dynamic pressure constraint model is as follows:

[0021]

[0022] Where q is the dynamic pressure of the aircraft; q max is the maximum allowable dynamic pressure;

[0023] The quasi-balanced gliding condition constraint model is as follows:

[0024]

[0025] r is the distance from the center of the earth to the aircraft, and σ is the roll angle of the aircraft.

[0026] Preferably, a collaborative guidance law based on improved prediction and correction is designed to solve the aircraft angle of attack and bank angle guidance instructions as follows:

[0027] Design a preset angle of attack profile for each aircraft;

[0028] Calculate the roll angle amplitude of each aircraft;

[0029] Solve for the sign of the roll angle for each vehicle.

[0030] Preferably, the specific form of the preset attack angle α profile is as follows:

[0031]

[0032] Where: α max is the maximum angle of attack; α LDmax is the angle of attack corresponding to the maximum lift-to-drag ratio; V0 is the initial speed of the aircraft; V aoa1 and V aoa2 are the speed of the aircraft at the segmentation points, It is the ideal speed for the middle and end handover points.

[0033] Preferably, the method for solving the roll angle amplitude of each aircraft is as follows:

[0034] The remaining range error and time error are predicted, and the overall error is obtained by weighting them. The secant method is used to obtain the roll angle amplitude that makes the overall error zero.

[0035] Preferably, the method for solving the sign of the roll angle of each aircraft is as follows:

[0036] A heading angle error corridor is designed for each aircraft, and the corridor width is adjusted based on the remaining flight time error. When the aircraft heading angle error exceeds the aircraft heading angle error corridor boundary, the aircraft roll angle is reversed, that is, the sign changes, thereby determining the sign of the aircraft roll angle.

[0037] Preferably, the cooperative guidance law based on the MAPPO algorithm is as follows:

[0038] (1) Design the preset angle of attack profile of each aircraft;

[0039] (2) Design the observation quantity, terminal reward and process reward of each aircraft;

[0040] (3) Use a fully connected neural network to establish the action network and evaluation network of all aircraft, and train the action network. The process is as follows:

[0041] S1: The aircraft action network outputs the aircraft roll angle command (including amplitude and sign) in real time based on the observed value, and simultaneously receives the process reward of environmental feedback. It also stores the data of interaction with the environment in the experience pool until the aircraft reaches the mid-terminal handover point and obtains the terminal reward. After all aircraft reach the terminal position, a learning cycle ends and the experience pool completes data collection.

[0042] S2. Use the data collected from the experience pool to train the aircraft action network using a centralized training method: During the training process, data is randomly sampled from the aircraft's experience pool data to update the parameters of the evaluation network, and the parameters of the action network are updated based on the value function generated by the evaluation network;

[0043] S3. After multiple update iterations, the data in the experience pool is cleared;

[0044] S4, return to step S1, and restart a new round of data collection and algorithm training process until the MAPPO algorithm converges and the action network parameters are trained;

[0045] (4) Each aircraft adopts a distributed execution mode and outputs the roll angle command in real time based on the trained action network until the aircraft arrives at the mid-terminal handover point.

[0046] Preferably, in step (1), the specific form of the preset attack angle α profile is as follows:

[0047]

[0048] Where: α max is the maximum angle of attack; α LDmax is the angle of attack corresponding to the maximum lift-to-drag ratio; V0 is the initial speed of the aircraft; V aoa1 and V aoa2 are the speed of the aircraft at the segmentation points; V f * It is the ideal speed for the middle and end handover points.

[0049] Preferably, in step (2), the observation quantity of each aircraft is The first six items represent the state of the aircraft; Δr = rr f is the difference between the current aircraft altitude and the terminal altitude; ΔV = VV f Indicates the difference between the current aircraft speed and the terminal speed; Δθ=θ-θ f Indicates the difference between the current aircraft longitude and the terminal longitude; It represents the difference between the current aircraft latitude and the terminal latitude; S is the remaining range of the aircraft; r is the altitude change rate.

[0050] Preferably, in step (2), the terminal reward for each aircraft is as follows:

[0051]

[0052] rr goal The reward for reaching the terminal position, rr dev Indicates the penalty value if the longitude and latitude of the aircraft deviate too far from the mid-terminal handover point when arriving at the terminal position.

[0053] Preferably, in step (2), the process reward function of each aircraft includes a distance reward, a roll angle constraint reward, and a time constraint reward:

[0054] Distance Reward rr goto as follows:

[0055]

[0056] Where S represents the remaining range of the aircraft; R0 is the radius of the earth; represents normalization processing; k togo >0 is a constant;

[0057] Roll angle constraint reward rr diff as follows:

[0058]

[0059] where σ pre_cor represents the roll angle generated by the predictive correction guidance; σ mappo is the roll angle generated by the MAPPO algorithm; k comp >0 is a constant;

[0060] Time constraint reward rr time as follows:

[0061] rr time =-k time |t togo -t co |

[0062] The time constraint reward enables multiple aircraft to meet the coordinated time constraints.

[0063] Preferably, in the cooperative guidance law based on the MAPPO algorithm, t togo The remaining flight time of the aircraft is calculated by analytical expression. The analytical expression is as follows:

[0064]

[0065] Where: R0 is the radius of the earth; g0 is the gravitational acceleration at sea level; V is the current speed of the aircraft; V f is the speed at the middle and end handover points; σ mappo is the roll angle generated by the cooperative guidance law based on the MAPPO algorithm, L and D are the lift and drag of the aircraft, respectively.

[0066] The beneficial effects of the present invention compared with the prior art are:

[0067] This invention addresses the problem of time-coordinated guidance for multiple hypersonic aircraft. It designs a time-coordinated guidance law based on improved prediction and correction, and a coordinated guidance law based on the MAPPO algorithm. This composite guidance method uses the remaining range of the aircraft as the switching logic. The composite guidance law proposed in this invention can accurately reach the mid-terminal handover point within a large time difference and successfully achieve the time-coordinated mission. The composite guidance law of this invention is highly robust to state disturbances, has a larger time-coordinated range, and offers improved time-coordinated guidance capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is the longitude and latitude change curve;

[0069] Figure 2 is the curve of height changing with speed;

[0070] Figure 3 is the curve of the roll angle changing with time;

[0071] Figure 4 is the curve of heading angle changing with time;

[0072] Figure 5 is the curve of track angle changing with time. DETAILED DESCRIPTION

[0073] The present invention will be further described below with reference to the accompanying drawings.

[0074] The present invention designs a phased time cooperative guidance law based on the remaining range. When the remaining range is large, an improved predictive correction guidance is adopted, while considering the remaining range error, the remaining cooperative flight time error and the terminal height error, and combining the heading angle error corridor to obtain the roll angle instruction, so as to realize the re-entry guidance of the aircraft under the time constraint. When the remaining range is small, in order to improve the guidance accuracy and time coordination capability, a multi-agent reinforcement learning method is switched to, by designing the observation space and action space, and considering the distance between the aircraft and the mid-terminal handover point, and the time error between aircraft to design the reward function, the reward weight and algorithm parameters are adjusted to train the aircraft, and finally the time cooperative guidance of the aircraft is realized. The specific method is as follows (each aircraft in the multi-aircraft is taken as an example below):

[0075] (1) Hypersonic vehicle modeling

[0076] (a) Dynamic modeling of each aircraft:

[0077]

[0078] Where r is the distance from the center of the Earth to the aircraft, θ and φ represent the longitude and latitude of the aircraft, respectively, V is the aircraft velocity, γ and ψ are the track angle and heading angle of the aircraft, respectively, σ is the roll angle of the aircraft, m is the mass of the aircraft, g is the acceleration due to gravity, and L and D are the lift and drag of the aircraft, respectively.

[0079]

[0080] Where ρ = ρ0e -βH is the atmospheric density at the aircraft's altitude, ρ0 is the atmospheric density at sea level; β = 1 / H MCP , H MCP =7.11km is the base altitude; S ref is the characteristic area of ​​the aircraft; C L and C D They are lift coefficient and drag coefficient respectively, which are related to the aircraft's angle of attack α and speed V. The present invention selects the American general aviation aircraft CAV-H as the research object. The lift coefficient C L and the drag coefficient C D The expression is:

[0081]

[0082] Where C L0 、C L1 、C L2 and C L3 is the fitting coefficient of the lift expression; C D0 、C D1 、C D2 and C D3 The lift coefficient C is the fitting coefficient of the drag expression L and the drag coefficient C D The relevant parameters are as follows:

[0083]

[0084] (b) Process Constraint Modeling

[0085] When a glider is flying at high speed, the high-speed airflow will generate a large amount of heat energy in the aircraft's stationary area. For aircraft safety, the heat flow rate constraint must be met:

[0086]

[0087] Where, represents the heat flux rate at the stagnation point of the aircraft; k Q is the heat flow rate model coefficient; is the maximum allowable heat flux rate at the aircraft stagnation point, ρ is the atmospheric density at the aircraft's altitude, and V is the aircraft's speed;

[0088] Overload refers to the multiple of the combined force of aerodynamic force and thrust acting on the aircraft relative to the aircraft's gravity. Due to the limited structural bearing capacity of the aircraft and considering safety issues, the aircraft overload cannot exceed the maximum overload limit. Since the hypersonic aircraft in the present invention has no thrust in the gliding phase, its overload constraint is

[0089]

[0090] Where n is the aircraft overload; n max is the maximum allowable overload; g0 is the gravitational acceleration at sea level.

[0091] Considering the structural bearing capacity of the aircraft, the aircraft must also meet the dynamic pressure constraint. During high-speed flight, the air flows in the opposite direction of the aircraft body at high speed. At this time, the air itself has kinetic energy, and the pressure presented by this kinetic energy is called dynamic pressure:

[0092]

[0093] Where q is the dynamic pressure of the aircraft; q max is the maximum dynamic pressure.

[0094] The upper limit of the aircraft's fuselage and wing structural strength, as well as the bearing capacity of the aerodynamic control surface hinges, determine the maximum allowable overload and dynamic pressure during reentry flight. Heat flux, overload, and dynamic pressure constraints are "hard constraints" that must be met during flight. The quasi-equilibrium gliding condition is a soft constraint designed to ensure the stability of the aircraft's altitude and flight path angle, avoiding large sudden changes. The mathematical form of this constraint is as follows:

[0095]

[0096] Substituting this into the expression of the track angle in the dynamic model, we can obtain the motion equation describing the quasi-balanced gliding:

[0097]

[0098] It can be seen from the above formula that when the aircraft satisfies the above formula, the combined force of its gravity and aerodynamic lift provides the required centripetal force. At this time, the change in flight altitude is very small, and the rate of change of the track angle γ is close to zero.

[0099] (c) Terminal constraint modeling: The terminal constraints of a hypersonic vehicle include the altitude, speed, and longitude and latitude constraints at the mid-terminal handover point:

[0100]

[0101] Where h f 、V f ,θ f 、φ fThey are the actual status (altitude, speed, longitude and latitude) of the aircraft when it arrives at the mid-term and final handover points; It is the ideal state for the mid-to-end handover point; h , ε V , ε θ , ε φ are the maximum terminal errors allowed for each state variable (altitude, speed, longitude and latitude).

[0102] (d) Modeling of bank angle amplitude constraints: During flight, the aircraft must meet the aforementioned hard constraints of heat flux, overload, and dynamic pressure. However, repeated calculations of the aircraft's heat flux, overload, and dynamic pressure will result in a large amount of computation. Therefore, the hard constraints of heat flux, overload, and dynamic pressure can be converted into constraints on the bank angle amplitude, namely:

[0103]

[0104] The constraint form of the roll angle amplitude is:

[0105]

[0106] They are respectively used to convert the aircraft's thermal flow, overload, and dynamic pressure process constraints into constraints on the roll angle amplitude.

[0107] In addition, to further suppress the oscillation of the reentry trajectory, especially to ensure that the heat flux constraint is met at the first trough after reentry, feedback control of the altitude change rate is added on the basis of the above-mentioned bank angle constraint. The purpose is to automatically adjust the amplitude of the bank angle according to the change of the aircraft altitude, so that the projection of the lift in the negative gravity direction, Lcosσ, can better meet the quasi-balanced gliding condition. Specifically:

[0108]

[0109] Where: σ(V) represents the roll angle of the aircraft as the speed changes, Indicates the current aircraft altitude change rate; is the reference height change rate; k is the feedback coefficient.

[0110] (2) Cooperative guidance law based on improved prediction and correction

[0111] When the remaining flight range is greater than 500km, the collaborative guidance law based on improved prediction and correction is used to obtain the flight angle of attack and bank angle of each aircraft.

[0112] (a) Angle of attack profile design: The control variables of a hypersonic vehicle during flight mainly include the angle of attack and the bank angle. However, considering that designing both the angle of attack and the bank angle simultaneously is computationally intensive and difficult, the present invention adopts a preset angle of attack-velocity profile. The specific form of the angle of attack α profile is as follows:

[0113]

[0114] Where: α max is the maximum angle of attack; α LDmax is the angle of attack corresponding to the maximum lift-to-drag ratio; V0 is the initial speed of the aircraft; V aoa1 and V aoa2 are the speeds of the aircraft at the segmentation points. V f * It is the ideal speed for the middle and end handover points.

[0115] (b) Design of roll angle amplitude

[0116] Convert the dynamic equation in (1) into the form of energy E integral and ignore the state variables that are irrelevant to the calculation of the remaining range S, and we can get:

[0117]

[0118] Integrate the above equation and set the integration termination condition to E≤E f , E f is the terminal energy of the aircraft when it arrives at the mid-terminal handover point, that is, the optimal remaining range S(|σ opt |), σ opt Indicates the roll angle that makes the overall error zero. In addition, the actual remaining range of the aircraft S f The current latitude and longitude of the aircraft can be And the longitude and latitude of the mid-to-end handover point θ f 、 Direct access to:

[0119]

[0120] Therefore, the remaining range error is expressed as

[0121] ΔS(|σ opt |)=S(|σ opt |)-S f .

[0122] Remaining flight time t togo Through integration, we can get:

[0123]

[0124] During the interval between two prediction and correction algorithm updates of the remaining flight time, the aircraft's roll angle profile remains unchanged, so the aircraft's real-time remaining flight time is equal to the remaining flight time of the last prediction and correction algorithm update minus the flight time. Based on this, the collaborative remaining flight time t is defined as co is the average value of the remaining flight time corresponding to the current state of each aircraft t co :

[0125]

[0126] N is the total number of aircraft, t togo,i is the remaining flight time of the i-th aircraft.

[0127] The coordinated time error of each aircraft can be expressed as:

[0128] Δt=t togo -t co

[0129] In summary, the overall error Δe(|σ opt |) is represented as:

[0130] Δe(|σ opt |)=k S ΔS(|σ opt |)+k t Δt

[0131] Where: k S 、k t are the weights corresponding to the remaining range error and the coordinated remaining flight time error respectively. Then, the secant method can be used to obtain the roll angle amplitude that makes the overall error zero:

[0132]

[0133] Where: k is the number of iterations required for the secant method iterative process to converge.

[0134] (c) Roll angle symbol design

[0135] The sign of the roll angle is mainly determined by the heading angle error corridor:

[0136]

[0137] Where: Δψ corridor is the width of the heading angle error corridor; Δψ out and Δψ in are the outer and inner boundary widths of the heading error corridor, Δψ out >Δψ in >0;V a and V bis a parameter to be set. When the aircraft's heading angle error exceeds the boundary of the aircraft's heading angle error corridor, the aircraft's roll angle reverses, i.e., its sign changes. Therefore, the heading angle error corridor can be used to determine the sign of the aircraft's roll angle:

[0138]

[0139] Where: T hor is the lateral guidance period, t is the current moment, and the heading angle error Δψ is defined as the deviation between the aircraft heading angle ψ and the aircraft line of sight angle Φ:

[0140] Δψ=ψ-Φ

[0141] The aircraft's sight angle Φ can be obtained through the spherical triangle:

[0142]

[0143] In addition, according to the current remaining flight time t of the aircraft togo and the coordinated remaining flight time t co The error dynamically adjusts the heading angle error corridor width:

[0144]

[0145] Where: T corridor The time error threshold for adjusting the width of the heading error corridor; Δψ ε In addition, to ensure that the lateral corridor has sufficient time to adjust while ensuring that the aircraft's roll angle reversal times are not excessive and that the aircraft can successfully reach the mid-term handover point, the heading angle error corridor width should be limited to a certain range:

[0146]

[0147] Where: Δψ min(1,2,3) , Δψ max(1,2,3) are the minimum and maximum values ​​of the heading angle error corridor width corresponding to the aircraft speed at different stages.

[0148] (3) Cooperative guidance law based on MAPPO algorithm

[0149] When the remaining flight range is less than 500km, the cooperative guidance law based on the MAPPO algorithm is adopted. In this algorithm, the observation quantity of each aircraft is designed to be The first six items represent the state of the aircraft; Δr = rr f is the difference between the current aircraft altitude and the terminal altitude; ΔV = VV f Indicates the difference between the current aircraft speed and the terminal speed; Δθ=θ-θ fIndicates the difference between the current aircraft longitude and the terminal longitude; Indicates the difference between the current aircraft latitude and the terminal latitude; S is the remaining range of the aircraft; In the MAPPO algorithm, the action amount of each aircraft is a = σ, which is the roll angle of the aircraft.

[0150] Take each aircraft as an example:

[0151] (a) Terminal reward function

[0152]

[0153] The terminal reward function is divided into two parts, which represent the reward value obtained by the aircraft when it reaches the terminal position, rr goal The reward for reaching the terminal position is determined by judging whether the aircraft can meet the terminal constraints when arriving at the mid-terminal handover point. If so, a large reward value is given to encourage the aircraft to complete the time coordination task while meeting the terminal accuracy. dev This value represents the penalty for aircraft longitude and latitude deviating too far from the mid-terminal handover point when arriving at the terminal position. The terminal reward is set so that the aircraft meets the specified accuracy when reaching the mid-terminal handover point.

[0154] (b) Process reward function

[0155] The process reward function is mainly used to guide the aircraft to reach the mid-term and final handover points and meet the coordination time constraints:

[0156]

[0157] rr goto is the process reward, which is mainly used to guide the aircraft to fly to the mid-term handover point. S represents the remaining range of the aircraft; R0 is the radius of the earth; represents normalization processing; k togo >0 is a constant. The closer the aircraft is to the longitude and latitude of the mid-term and final handover points, the greater the reward for this part.

[0158]

[0159] rr diff represents the roll angle constraint reward, which uses the roll angle generated by the improved predictive correction guidance to constrain the roll angle generated by the MAPPO algorithm, where σ pre_cor represents the roll angle generated by the predictive correction guidance; σ mappo is the roll angle generated by the cooperative guidance law based on the MAPPO algorithm; k compThe purpose of this reward is to speed up the algorithm training by mimicking the predicted corrective guidance roll angle, while also leaving some room for improvement. That is, only when the difference between the two roll angles is greater than Δδ>0 will the penalty value constrain the roll angle generated by the MAPPO algorithm.

[0160] rr time =-k time |t togo -t co |

[0161] rr time represents the time constraint reward, which is mainly used to make multiple aircraft meet the coordinated time constraint conditions, where t togo The remaining flight time of the aircraft is calculated by analytical expression. The analytical expression is as follows:

[0162]

[0163] Where: R0 is the radius of the earth; g0 is the gravitational acceleration at sea level; V is the current speed of the aircraft; V f is the speed at the mid-term handover point. After obtaining the remaining flight time of all aircraft, average them to get t co , k time > 0 is a constant. The purpose of this reward is to influence the roll angle generated by the MAPPO algorithm through the penalty term, so that the remaining flight time of all aircraft is close to the average coordination time, thereby satisfying the multi-aircraft coordination time constraint. Finally, the process reward received by each aircraft is the weighted sum of these three rewards.

[0164] (c) Training of MAPPO algorithm:

[0165] S1. Use a fully connected neural network to build the motion network of all aircraft;

[0166] S2: The aircraft action network outputs the aircraft roll angle command (including amplitude and sign) in real time based on the observed value, while receiving rewards from the environment and storing the data of the interaction with the environment in the experience pool until the aircraft reaches the mid-terminal handover point and obtains the terminal reward. After all aircraft reach the terminal position, a learning cycle ends and the experience pool completes data collection.

[0167] S3. Using the data collected from the experience pool, the aircraft action network is trained using a centralized training method. During the training process, data is randomly sampled from the aircraft's experience pool data to update the parameters of the evaluation network. The parameters of the action network are then updated based on the value function generated by the evaluation network.

[0168] S4. After multiple update iterations, the data in the experience pool is cleared;

[0169] S5. Return to step S2 and restart a new round of data collection and algorithm training process until the MAPPO algorithm converges and the action network parameters are trained.

[0170] Finally, each aircraft uses a distributed execution method to output roll angle commands in real time based on the trained action network until the aircraft reaches the mid-term handover point.

[0171] Simulation Verification

[0172] The present invention takes the general aviation aircraft CAV-H as the research object, and the relevant parameters are set as follows: Earth radius R0 = 6378 km; Sea level atmospheric density ρ0 = 1.2258 kg / m 3 ; Gravity acceleration at sea level g0 = 9.8 m / s 2 ; Aircraft mass m = 907.2 kg; Reference area S ref =0.4839m 2 ; Heat flow model coefficient k Q =5×10 -5 ; Maximum allowable heat flow rate Maximum allowable overload n max =3; Maximum allowable dynamic pressure q max =100kPa. In addition, the maximum angle of attack of the aircraft is α max =20°; Maximum lift-to-drag ratio corresponds to angle of attack α LDmax =20°; angle of attack profile velocity node V aoa1 =5000m / s, V aoa2 =2000m / s; lateral guidance period T hor = 0.1s. In the process of updating the roll angle amplitude through longitudinal prediction and correction, in order to ensure the terminal accuracy while reducing the amount of calculation for online correction of the roll angle, a prediction correction period T is designed to be shortened as the remaining range decreases. preco :

[0173]

[0174] In addition, the outer boundary of the aircraft heading angle error corridor Δψ out =21°; inner boundary Δψ in =3°; Node V a =5000m / s, V b =2500m / s. Dynamic adjustment threshold of heading corridor T corridor =0.8; heading angle corridor boundary width adjustment value When the remaining range of each aircraft in the composite guidance law reaches 400 km, the aircraft switches to the cooperative guidance law based on the MAPPO algorithm. The number of aircraft N = 8, and the initial positions are as shown in the following table:

[0175]

[0176] The MAPPO algorithm uses a distributed reinforcement learning method for training. The structure of the action network and the evaluation network uses a 2-layer 128-node fully connected network. The learning rate of the action network and the evaluation network is set to lr = 5e-4, and the network is trained using the Adam optimizer, with the discount factor set to 0.99 and the GAE set to 0.95. Since the MAPPO algorithm is an on-policy algorithm, the data stored in the experience pool is discarded after the update. Therefore, the batch size of the experience pool is set to 2000 to store all the trajectory data of the aircraft, the minibatch size is set to 600, and the number of reuses of the experience data is set to 6. The simulation results are shown in Figures 1 to 5 From the simulation results, it can be seen that the method of the present invention can enable multiple aircraft to arrive at the same target point at the same time, and compared with the traditional method, the longitude and latitude errors and the time coordination errors are significantly reduced.

[0177] The core concept of the present invention lies in integrating the advantages of improved predictive correction guidance and a collaborative guidance law based on the MAPPO algorithm. When the remaining range is large, the improved predictive correction guidance can solve the collaborative flight trajectory with the minimum combined error between the remaining range error and the remaining flight time error through iterative optimization, and under the constraints of the heading angle corridor, the aircraft can stably and reliably arrive near the mid-terminal handover point after a long and long flight. When the remaining range of the aircraft reaches a certain distance, in order to achieve higher spatial and temporal collaborative guidance accuracy for the aircraft cluster, it is necessary to further increase the update frequency of the guidance instructions. The ballistic integration process of the improved predictive correction guidance is computationally intensive and has a long guidance cycle. The collaborative guidance law based on the MAPPO algorithm has the advantages of faster intelligent algorithm calculation speed, shorter guidance cycle, and higher degrees of freedom in outputting roll angle instructions, which can further improve the collaborative guidance accuracy of the aircraft cluster.

[0178] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A multi-aircraft cooperative guidance method based on the MAPPO algorithm, characterized in that: include: Establishing a multi-aircraft collaborative guidance model, wherein the multi-aircraft collaborative guidance model includes a dynamic model of each aircraft, a process constraint model, a terminal constraint model, and a roll angle amplitude constraint model; When the remaining flight range is greater than or equal to 500 km, a collaborative guidance law based on improved prediction and correction is designed to solve the aircraft's angle of attack and bank angle guidance instructions: When the remaining range is less than 500km, the system switches to the collaborative guidance law based on the MAPPO algorithm to solve the aircraft's angle of attack and roll angle guidance instructions.

2. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 1, characterized in that: In multiple aircraft, the dynamic model of each aircraft is as follows: Where r is the distance from the center of the Earth to the aircraft, θ and φ represent the longitude and latitude of the aircraft, respectively, V is the aircraft velocity, γ and ψ are the track angle and heading angle of the aircraft, respectively, σ is the roll angle of the aircraft, m is the mass of the aircraft, g is the acceleration due to gravity, and L and D are the lift and drag of the aircraft, respectively.

3. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 1, characterized in that: The process constraint model of each aircraft includes heat flux constraint model, overload constraint model, dynamic pressure constraint model, and quasi-equilibrium gliding condition constraint model; The heat flow rate constraint model is as follows: Where, represents the heat flux rate at the stagnation point of the aircraft; k Q is the heat flow rate model coefficient; is the maximum allowable heat flux rate at the aircraft stagnation point, ρ is the atmospheric density at the aircraft's altitude, and V is the aircraft's speed; The overload constraint model is as follows: Where n is the aircraft overload; n max is the maximum allowable overload; g0 is the gravitational acceleration at sea level, m is the mass of the aircraft, g is the acceleration due to gravity, L and D are the lift and drag of the aircraft respectively; The dynamic pressure constraint model is as follows: Where q is the dynamic pressure of the aircraft; q max is the maximum allowable dynamic pressure; The quasi-balanced gliding condition constraint model is as follows: r is the distance from the center of the earth to the aircraft, and σ is the roll angle of the aircraft.

4. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 1, characterized in that: Design a collaborative guidance law based on improved predictive correction to solve the aircraft's angle of attack and bank angle guidance commands. The method is as follows: Design a preset angle of attack profile for each aircraft; Calculate the roll angle amplitude of each aircraft; Solve for the sign of the roll angle for each vehicle.

5. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 4, characterized in that: The specific form of the preset attack angle α profile is as follows: Where: α max is the maximum angle of attack; α LDmax is the angle of attack corresponding to the maximum lift-to-drag ratio; V0 is the initial speed of the aircraft; V aoa1 and V aoa2 are the speed of the aircraft at the segmentation points, It is the ideal speed for the middle and end handover points.

6. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 4, characterized in that: The method for solving the roll angle amplitude of each aircraft is as follows: The remaining range error and time error are predicted, and the overall error is obtained by weighting them. The secant method is used to obtain the roll angle amplitude that makes the overall error zero.

7. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 4, characterized in that: The method to solve for the sign of the roll angle for each aircraft is as follows: A heading angle error corridor is designed for each aircraft, and the corridor width is adjusted based on the remaining flight time error. When the aircraft heading angle error exceeds the aircraft heading angle error corridor boundary, the aircraft roll angle is reversed, that is, the sign changes, thereby determining the sign of the aircraft roll angle.

8. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 1, characterized in that: The cooperative guidance law based on the MAPPO algorithm is as follows: (1) Design the preset angle of attack profile of each aircraft; (2) Design the observation quantity, terminal reward and process reward of each aircraft; (3) Use a fully connected neural network to establish the action network and evaluation network of all aircraft, and train the action network. The process is as follows: S1: The aircraft action network outputs the aircraft roll angle command (including amplitude and sign) in real time based on the observed value, and simultaneously receives the process reward of environmental feedback. It also stores the data of interaction with the environment in the experience pool until the aircraft reaches the mid-terminal handover point and obtains the terminal reward. After all aircraft reach the terminal position, a learning cycle ends and the experience pool completes data collection. S2. Use the data collected from the experience pool to train the aircraft action network using a centralized training method: During the training process, data is randomly sampled from the aircraft's experience pool data to update the parameters of the evaluation network, and the parameters of the action network are updated based on the value function generated by the evaluation network; S3. After multiple update iterations, the data in the experience pool is cleared; S4, return to step S1, and restart a new round of data collection and algorithm training process until the MAPPO algorithm converges and the action network parameters are trained; (4) Each aircraft adopts a distributed execution mode and outputs the roll angle command in real time based on the trained action network until the aircraft arrives at the mid-terminal handover point.

9. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 8, characterized in that: In step (1), the specific form of the preset attack angle α profile is as follows: Where: α max is the maximum angle of attack; α LDmax is the angle of attack corresponding to the maximum lift-to-drag ratio; V0 is the initial speed of the aircraft; V aoa1 and V aoa2 are the speeds of the aircraft at the segmentation points respectively; It is the ideal speed for the middle and end handover points.

10. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 8, characterized in that: In step (2), the observation quantity of each aircraft is The first six items represent the state of the aircraft; Δr = rr f is the difference between the current aircraft altitude and the terminal altitude; ΔV = VV f Indicates the difference between the current aircraft speed and the terminal speed; Δθ=θ-θ f Indicates the difference between the current aircraft longitude and the terminal longitude; Indicates the difference between the current aircraft latitude and the terminal latitude; S is the remaining range of the aircraft; is the rate of change of height.

11. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 8, characterized in that: In step (2), the terminal reward for each aircraft is as follows: rr goal The reward for reaching the terminal position, rr dev Indicates the penalty value if the longitude and latitude of the aircraft deviate too far from the mid-terminal handover point when arriving at the terminal position.

12. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 8, characterized in that: In step (2), the process reward function of each aircraft includes distance reward, roll angle constraint reward, and time constraint reward: Distance Reward rr goto as follows: Where S represents the remaining range of the aircraft; R0 is the radius of the earth; represents normalization processing; k togo >0 is a constant; Roll angle constraint reward rr diff as follows: where σ pre_cor represents the roll angle generated by the predictive correction guidance; σ mappo is the roll angle generated by the MAPPO algorithm; k comp >0 is a constant; Time constraint reward rr time as follows: rr time =-k time |t togo -t co | The time constraint reward enables multiple aircraft to meet the coordinated time constraints.

13. The multi-aircraft cooperative guidance method based on the MAPPO algorithm according to claim 12, characterized in that: In the cooperative guidance law based on the MAPPO algorithm, t togo The remaining flight time of the aircraft is calculated by analytical expression. The analytical expression is as follows: Where: R0 is the radius of the earth; g0 is the gravitational acceleration at sea level; V is the current speed of the aircraft; V f is the speed at the middle and end handover points; σ mappo is the roll angle generated by the cooperative guidance law based on the MAPPO algorithm, L and D are the lift and drag of the aircraft, respectively.

Images