Method for evading missile of split-body aircraft based on deep reinforcement learning and split-body aircraft

Abstract

This invention discloses a method for missile evasion by a split-type aircraft based on deep reinforcement learning, and the split-type aircraft itself. The method includes the following steps: S1. For the control of the split-type aircraft, a decision network and an evaluation network based on the DDPG algorithm are constructed and trained using reinforcement learning; wherein, the split-type aircraft includes a front sub-aircraft, a rear sub-aircraft, and a front-rear sub-aircraft separation device, the front sub-aircraft has a nose landing gear, a spare main landing gear, and a spare engine, and the rear sub-aircraft has a main landing gear and a main engine; S2. During flight, the x-axis distance, y-axis distance, and directional angle actually observed by the aircraft radar are used as state inputs into the decision network, which controls the aircraft's steering and the separation of the rear sub-aircraft from the front sub-aircraft. This invention can control the split-type aircraft's steering to align the aircraft with the missile's direction and separate the rear sub-aircraft from the front sub-aircraft before the missile hits the aircraft, thus achieving the function of split-type missile evasion.

Description

Technical Field

[0001] This invention relates to the field of aircraft technology, and in particular to a method for a split-type aircraft to evade missiles based on deep reinforcement learning, and the split-type aircraft itself. Background Technology

[0002] A morphing aircraft is a new type of aircraft capable of sensing the external environment in real time and autonomously changing its shape and layout according to information such as flight mission, flight status, and flight environment to achieve optimal performance flight under different missions. As a new concept, compared with fixed-shape aircraft, morphing aircraft can change shape to have greater range and stronger environmental adaptability, and is one of the current research hotspots in the aerospace field.

[0003] In the late 1970s and early 1980s, NASA and the U.S. Air Force jointly conducted the Mission Adaptive Wing (MAW) project. Subsequently, NASA launched the Aircraft Morphing project to focus on researching intelligent device components for airframe applications. In 2003, DARPA (Defense Advanced Research Projects Agency) launched the Modular Aircraft Structure (MAS) project, aiming to study larger-scale morphing methods to achieve multi-mission flight of modular aircraft. The proposed design concept of this project is to extend the leading and trailing edge control surfaces into fully adaptive wings, allowing the aircraft to achieve flight control through wing morphing, similar to birds. NASA's "21st Century Aviation Vision" has already included specific ideas for modular aircraft, with the expectation of making them a reality around 2030.

[0004] Since the beginning of the 21st century, my country has focused its research on the concepts related to morphing aircraft. Some researchers have conducted in-depth studies on the coordinated control of morphing and flight. Others have studied the unsteady aerodynamic characteristics during continuous airfoil deformation. Some have considered factors such as time delay and packet loss in sampling communication, researched distributed cooperative control strategies, and built a distributed-drive intelligent morphing wing simulation platform. The Shenyang Aircraft Design and Research Institute proposed a novel morphing control UAV, employing the Q-Learning (reinforcement learning algorithm) method to implement the reinforcement learning control module of the morphing UAV. Xi'an Jiaotong University, drawing inspiration from the ability of birds to retract their wings during high-speed flight and extend them during low-speed flight, has researched a novel biomimetic morphing UAV.

[0005] Compared with traditional fixed-structure aircraft, morphing aircraft have the ability to change shape, which improves aerodynamic characteristics. At the same time, they inevitably bring new technical problems. The following will further discuss the current research status of morphing aircraft, mainly focusing on the design of deformable structures and the design of flight control systems.

[0006] a. Deformable structure design of aircraft

[0007] Currently, scholars and industry professionals both domestically and internationally have proposed various aircraft morphing structure design schemes. Among them, the wing morphing mechanism has received the most attention. Its working principle is to change the aerodynamic lift of the aircraft by altering the shape of the wing. According to the scale of deformation, wing morphing can be divided into three categories: small-scale deformation, medium-scale deformation, and large-scale deformation. Small-scale deformation refers to local changes to the wing, such as spoilers and bulges; medium-scale deformation refers to changes to the wing airfoil, such as changes in thickness, camber, and torsion; large-scale deformation involves changes to the entire wing, such as folding, sweeping, and extension / retraction. Notably, the University of Florida in the United States has developed a deformable micro-aircraft that resembles the flight of a seagull, which can be divided into three forms: neutral, positively deflected, and negatively deflected.

[0008] b. Aircraft control technology

[0009] The control principle of morphing aircraft is basically the same as that of conventional aircraft. However, the changes in aerodynamic characteristics, center of gravity, and moment of inertia caused by their deformation alter the parameters in the conventional six-degree-of-freedom twelve-state equations for aircraft, posing significant challenges to the control of morphing aircraft. Generally speaking, there are two main research approaches to the relationship between the morphing process and the control system of morphing aircraft, and their impact. The first approach treats deformation as an externally given command, with the aircraft deforming over time according to the given command, and then designing the control system based on this. The second approach treats structural deformation as an input to the control system, where deformation and flight state are coupled and influence each other. Commonly used control methods include robust adaptive control, backstepping-based control, and sliding mode control.

[0010] Split-body aircraft, as a type of variator aircraft, allow parts of the fuselage to detach and move freely during flight, achieving enhanced combat performance. Their advantages include, but are not limited to:

[0011] 1. It can actively split into parts to avoid obstacles and missile attacks. The actively split parts can be used as decoys to confuse the enemy's missile tracking system.

[0012] 2. By splitting the aircraft, its aerodynamic characteristics can be altered to achieve higher flight performance. Unlike traditional split-type rockets, the separate parts of a split-type aircraft, when not split, not only provide power but can also carry munitions and launch missiles.

[0013] 3. The modular aircraft adopts a modular design that allows it to be connected to another functional sub-module to achieve different tasks, thus saving costs. For example, the main propulsion flight module can be combined with the forest fire fighting module to achieve the forest fire fighting function, while the main propulsion module can be repeatedly "combined" with other modules to achieve different functions, saving costs.

[0014] However, existing control methods for morphing aircraft have the following shortcomings:

[0015] 1. Q-learning-based variant aircraft: The Q-learning method cannot fully represent the flight state of the aircraft, and the actions can only be set to specific values;

[0016] 2. The research object of the adaptive morphing strategy and flight control method for morphing aircraft based on reinforcement learning is a morphing aircraft with a variable airfoil. It uses the aircraft's dive, cruise, and climb as three states, and the aircraft's sweep angle as the action, exploring the optimal strategy for aircraft morphing using Q-learning. However, the action space executed by the morphing aircraft under this control method is discrete, limiting its applicability. It cannot be applied to the control of split-type aircraft.

[0017] Existing variant aircraft and their control methods are not suitable for controlling split-type aircraft to achieve the function of split-type missile evasion. Summary of the Invention

[0018] The purpose of this invention is to propose a method for missile evasion by a split-type aircraft based on deep reinforcement learning, and to provide a split-type aircraft that effectively solves the structural design and control problems of missile evasion by split-type aircraft.

[0019] The technical problem of this invention is solved by the following technical solution:

[0020] A method for evading missiles using a split-type aircraft based on deep reinforcement learning includes the following steps:

[0021] S1. For the control of split-type aircraft, a decision network and evaluation network based on the DDPG algorithm are constructed and trained by reinforcement learning.

[0022] The split aircraft includes a front sub-unit, a rear sub-unit, and a front-to-rear sub-unit separation device. The front sub-unit has a nose landing gear, a spare main landing gear, and a spare engine. The rear sub-unit has a main landing gear and a main engine.

[0023] During training, the observed distances between the missile and the aircraft on the xy plane of the aircraft's body coordinate system, the distances between the missile and the aircraft on the x-axis, and the directional angle are used as state inputs into the decision network. The decision network outputs actions to the evaluation network. The evaluation network performs reward scoring and updates network parameters based on the state and actions. The decision network performs gradient ascent based on the scores to obtain an updated strategy. The updated strategy is used to control the aircraft's turning so that the aircraft's direction is consistent with the missile's direction, and to separate the rear sub-aircraft from the front sub-aircraft before the missile hits the aircraft.

[0024] S2. During flight, the x-axis distance, y-axis distance, and directional angle actually observed by the aircraft radar are input into the decision network as state, and the decision network controls the aircraft to turn and separate the rear sub-machine from the front sub-machine.

[0025] In some embodiments, the present invention further includes the following technical features:

[0026] When the rear sub-unit separates from the front sub-unit, the backup engine of the front sub-unit is activated, and the front sub-unit uses the nose landing gear and the backup main landing gear to complete the landing.

[0027] In step S1, the step size period for controlling the aircraft's steering is 0.1s.

[0028] In step S1, the angle of aircraft turning is controlled from -1 degree to 1 degree each time.

[0029] In step S1, controlling the aircraft's direction to align with the missile's direction means controlling the angle between the aircraft's and the missile's directions to be no greater than 1 degree.

[0030] In step S1, when the angular difference between the speed direction of the aircraft and the missile is less than 1 degree, the reward is increased; otherwise, the reward remains unchanged.

[0031] The evaluation network is trained using the TD method, which backpropagates the TD error for training; the decision network uses a value function as the maximization objective and is trained using the gradient ascent method; the network parameters are determined after training.

[0032] The present invention also provides a split-type aircraft, including a front sub-aircraft, a rear sub-aircraft, a front-rear sub-aircraft separation device, and a control module. The front sub-aircraft has a nose landing gear, a spare main landing gear, and a spare engine. The rear sub-aircraft has a main landing gear and a main engine. The control module is configured to perform the method for missile avoidance of a split-type aircraft based on deep reinforcement learning as described in any one of claims 1 to 7.

[0033] In some embodiments, both the front sub-engine and the rear sub-engine are provided with twin vertical tail fins.

[0034] In some embodiments, the front and rear submachine separation device is a device with thrust.

[0035] The beneficial effects of this invention compared to the prior art include:

[0036] This invention proposes a method for missile evasion using a split-type aircraft based on deep reinforcement learning, and the split-type aircraft itself. The split-type aircraft includes a front sub-aircraft, a rear sub-aircraft, and a separation device for the front and rear sub-aircraft. The front sub-aircraft has a nose landing gear, a spare main landing gear, and a spare engine. The rear sub-aircraft has a main landing gear and a main engine. This invention constructs a decision network and an evaluation network based on the DDPG algorithm and trains them using reinforcement learning. This allows the split-type aircraft to separate before being hit by a missile, ensuring the missile can only strike the rear sub-aircraft, while the front sub-aircraft improves its aerodynamic characteristics to escape. By using a deep reinforcement learning control method, the aircraft is made to align its velocity direction with the incoming missile as quickly as possible before separating the front and rear sub-aircraft. This ensures the missile can only strike the rear sub-aircraft from behind, avoiding a large velocity angle that could cause the missile to hit the front sub-aircraft. It also allows the front sub-aircraft to maintain stable flight as quickly as possible, enabling the pilot to escape and safely transfer supplies and data from the rear sub-aircraft. The structural design of this invention ensures that after the front and rear sub-aircraft separate during missile evasion, the front sub-aircraft can fly normally and land smoothly. Attached Figure Description

[0037] Figure 1 This is a flowchart of a method for a split-type aircraft to evade missiles based on deep reinforcement learning in an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of the entire split-type aircraft in an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the front sub-engine of the split-type aircraft in an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the rear sub-unit of the split-type aircraft in an embodiment of the present invention;

[0041] Figure 5 This is a control flowchart of the split-type aircraft in an embodiment of the present invention;

[0042] Figure 6 This is a trajectory diagram of the missile and the aircraft when the missile locks on in an embodiment of the present invention;

[0043] Figure 7 This is a diagram of the reinforcement learning framework in an embodiment of the present invention;

[0044] Figure 8 This is a round-to-round total reward graph of the DDPG algorithm in this embodiment of the invention;

[0045] Figure 9This is a diagram showing the change in the flight angle of the missile and the split-type aircraft in an embodiment of the present invention;

[0046] Figure 10 This is a trajectory diagram of the missile and the split-type aircraft in an embodiment of the present invention;

[0047] Figure 11a This is a schematic diagram of the split-type aircraft in the demonstration animation video according to an embodiment of the present invention;

[0048] Figure 11b This is a schematic diagram of the front and rear sub-units after they are separated in the demonstration animation video of an embodiment of the present invention. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0050] It should be noted that the directional terms such as left, right, up, down, top, and bottom used in this embodiment are only relative concepts or are based on the normal use of the product, and should not be considered as restrictive.

[0051] For detachable aircraft, reinforcement learning is employed for control. Reinforcement learning, a major research area in artificial intelligence, has shown excellent performance in recent years in fields such as game AI, autonomous driving, and robot control. The aircraft control utilizes the DDPG (Deep Deterministic Policy Gradient) algorithm, a classic reinforcement learning algorithm. Reinforcement learning, also known as reward learning, evaluation learning, or enhancement learning, is one of the paradigms and methodologies of machine learning. In the reinforcement learning framework, an agent explores the environment and receives rewards based on its state and actions. The agent's learning task is to find a policy that selects actions that maximize rewards in the long run. This learning task requires not only selecting actions that yield the highest reward for the current state but also considering long-term consequences, selecting actions that maximize the agent's global reward. Currently, reinforcement learning algorithms are mainly divided into three types: value-based reinforcement learning, policy-based reinforcement learning, and the actor-critic framework. The most representative algorithms of the three reinforcement learning frameworks are DQN (deep Q-network), TRPO (TrustRegion Policy Optimization), and DDPG. Deep reinforcement learning is a combination of deep learning and reinforcement learning, using neural networks to better represent policies or value functions. Deep reinforcement learning is now widely used in fields such as fault diagnosis and autonomous driving, and is a relatively complete algorithm. However, its disadvantages include difficulty in convergence and high computational cost, requiring optimization for industrial applications. This invention demonstrates how the DDPG algorithm achieves relatively stable control of a split-type aircraft by establishing a simple scenario.

[0052] In particular, this invention addresses the problem of aircraft struggling to evade missiles by designing a split-type aircraft, comprising a front sub-unit and a rear sub-unit. This aircraft can split apart after being locked on by a missile. This splitting allows the missile to hit the rear sub-unit, while the front sub-unit improves its aerodynamic characteristics to escape.

[0053] Specifically, addressing the control problem of a split-type aircraft evading missiles, this embodiment of the invention employs a reinforcement learning control method to enable the aircraft to align its speed and direction with the incoming missile as quickly as possible. This avoids the missile striking the leading sub-unit due to a large angle between its speed and direction, and also allows the aircraft to maintain stable flight as quickly as possible, enabling the transfer of materials and data from the trailing sub-unit.

[0054] like Figure 1 As shown, this embodiment of the invention provides a method for a split-type aircraft to evade missiles based on deep reinforcement learning, including the following steps:

[0055] S1. For the control of split-type aircraft, a decision network and evaluation network based on the DDPG algorithm are constructed and trained by reinforcement learning.

[0056] The split aircraft includes a front sub-unit, a rear sub-unit, and a front-to-rear sub-unit separation device. The front sub-unit has a nose landing gear, a spare main landing gear, and a spare engine, while the rear sub-unit has a main landing gear and a main engine.

[0057] During training, the observed distances between the missile and the aircraft on the xy plane of the aircraft's body coordinate system, along with the distances between their x-axis and y-axis and the angle of their directions, are used as state inputs into the decision network. The decision network outputs actions to the evaluation network, which then assigns rewards and updates network parameters based on the state and actions. The decision network performs gradient ascent based on the scores to obtain an updated strategy. This updated strategy is used to control the aircraft's steering so that it aligns with the missile's direction, and to separate the rear sub-aircraft from the front sub-aircraft before the missile hits it.

[0058] Furthermore, the step period for controlling the aircraft's steering is 0.1s.

[0059] Furthermore, the angle at which the aircraft is turned each time is between -1 degree and 1 degree.

[0060] Furthermore, controlling the aircraft's turn to align the aircraft's direction with the missile's direction means controlling the angle between the aircraft's and the missile's directions to be no greater than 1 degree.

[0061] Furthermore, when the angular difference between the speed direction of the aircraft and the missile is less than 1 degree, the reward is increased; otherwise, the reward remains unchanged.

[0062] S2. During flight, the x-axis distance, y-axis distance, and directional angle actually observed by the aircraft radar are used as state inputs into the decision network, which controls the aircraft's steering and the separation of the rear sub-aircraft from the front sub-aircraft.

[0063] In some embodiments, when the rear submachine gun separates from the front submachine gun, the backup engine of the front submachine gun is activated, and the front submachine gun uses the nose landing gear and the backup main landing gear to complete the landing.

[0064] In some embodiments, the evaluation network is trained using the TD method, which backpropagates the TD error for training; the decision network is trained using the gradient ascent method with the value function as the maximization objective; and the network parameters are determined after training.

[0065] This invention also provides a split-type aircraft, including a front sub-aircraft, a rear sub-aircraft, a front-rear sub-aircraft separation device, and a control module. The front sub-aircraft has a nose landing gear, a spare main landing gear, and a spare engine, while the rear sub-aircraft has a main landing gear and a main engine. The control module is configured to execute the aforementioned method for split-type aircraft to evade missiles based on deep reinforcement learning.

[0066] In some embodiments, both the front and rear sub-engines are equipped with twin vertical tail fins.

[0067] In some embodiments, the front and rear submachine separation device is a device with thrust.

[0068] The following further describes a method for missile evasion by a split-type aircraft based on deep reinforcement learning, and specific implementations of the split-type aircraft.

[0069] 1. Structural design of split-type aircraft

[0070] First, the structural design of the split-type aircraft is carried out: the front part of the split-type aircraft is the core part, called the front sub-aircraft, which has a complete control system and a retractable, separate backup main landing gear, allowing for independent takeoff and landing; the rear part mainly stores ammunition and fuel, lacks a front landing gear, and cannot takeoff and land independently, called the rear sub-aircraft. This structure ensures that the front sub-aircraft has better maneuverability than the whole aircraft, and in the event of a missile attack, the rear sub-aircraft, with its larger size, can intercept the missile. These two points give the front sub-aircraft a greater chance of escape when the split-type aircraft is attacked. Compared with existing split-type aircraft, the method in this embodiment can use reinforcement learning control to separate the rear sub-aircraft and detonate the missile, thus intelligently protecting the front sub-aircraft. Some split-type aircraft are designed with separate escape pods to protect passengers; these separated structures do not have power. The front sub-aircraft in this embodiment has an independent backup engine, and after separation, it can transport passengers or supplies to a safe area.

[0071] In this embodiment, CATIA (a 3D design software) was used for the parametric design and model building of the split-type aircraft. First, the wings of the split-type aircraft were determined to be delta wings. This provides better aerodynamic characteristics at supersonic speeds and allows for greater fuel storage, extending the aircraft's combat radius. Both delta wings of the split-type aircraft use the LG10SC IL (NASA 0710) airfoil, with root chord lengths of 11.4m and 5.32m respectively, and a sweep angle of 45 degrees for both. Then, to maintain the directional balance, stability, and controllability of the split-type aircraft, a vertical tail was installed, with an airfoil of NACA 0010. Based on the delta wings and tail, the length of the forward core section of the aircraft was set at 17.2m, and the length of the rear section (excluding the detachment section) was set at 11.9m. When the two parts of the split-type aircraft are not separated, there is an overlap. The overall length of the split-type aircraft is 27.8m, and the wingspan is 31.0m. Finally, based on the fuselage parameters of the modular aircraft, appropriately sized landing gear, nose, air intakes, and other components are designed to complete the assembly of the modular aircraft.

[0072] A schematic diagram of the modular aircraft is shown below. Figure 2 As shown, it includes a front sub-engine 1, a rear sub-engine 2, a front-rear sub-engine separation device, and a control module. Since the aircraft needs to perform various missions at supersonic speeds most of the time, a delta wing, which is more adapted to supersonic flight, is chosen for the aircraft's wings. Its shape characteristics allow the wing to generate greater lift at high speeds, and the aerodynamic efficiency of the delta wing is also higher. In addition, the delta wing has high structural strength, light weight, and good rigidity, which is conducive to the retraction of landing gear and the placement of fuel and other equipment.

[0073] A schematic diagram of the front and rear sub-units of the split-type aircraft is shown below. Figures 3 to 4 As shown, the front sub-unit 1 includes a nose landing gear, a spare main landing gear 13, a spare engine, and a spare engine air intake 12, while the rear sub-unit 2 includes a main landing gear, a main engine 22, and an air intake 24. To improve the stability of the aircraft during supersonic flight, twin vertical tail fins 11 are added to the front sub-unit 1, and twin vertical tail fins 21 are added to the rear sub-unit 2. This maintains the high stability of the entire split aircraft while ensuring that the front sub-unit retains high maneuverability to complete different missions after separation. The rear sub-unit may need to maintain its original speed and direction of flight for a period of time after separation; the twin vertical tail fins ensure its stability.

[0074] The main design features of the split-section aircraft are as follows: Missiles are mounted on the wings of the rear sub-unit. Firstly, this complements the larger wingspan of the rear sub-unit, resulting in greater overall aircraft balance. Secondly, in the event of a dangerous separation, the front sub-unit is lighter and has a higher escape capability. A front-to-rear sub-unit separation device 23 is installed at the connection point between the front and rear sub-units, providing a certain amount of thrust. This separation device 23 ensures that the front and rear sub-units can be separated quickly in emergencies, greatly improving the emergency avoidance capability of the split-section aircraft. The rear of the front sub-unit has a spare engine, which can independently support the high-speed flight of the separated front sub-unit after separation. To ensure successful recovery of the front sub-unit, a spare main landing gear 13 is also housed in the front sub-unit cabin, allowing the front sub-unit to land normally after separation.

[0075] 2. Control of split-type aircraft

[0076] The control flowchart of the split-type aircraft is as follows: Figure 5 As shown, the states of the missile and the aircraft are input into the decision network, which then outputs actions to the evaluation network. The evaluation network scores the states and actions, updates its parameters using the TD algorithm, and transmits the scores to the decision network. The decision network performs gradient ascent based on the scores and transmits the updated strategy to the separate aircraft.

[0077] The control of the split-body aircraft adopts the DDPG method. In this embodiment, when the missile hits the split-body aircraft, the projections of their velocities onto the xy-plane of the split-body aircraft's coordinate system are consistent, so the two-dimensional case of missile chasing the split-body aircraft is considered. When the missile locks onto the split-body aircraft, the missile's velocity is directed towards the split-body aircraft. As the positions of the split-body aircraft and the missile change, the missile continuously adjusts its velocity direction, eventually hitting the split-body aircraft. Since the missile's velocity is often many times greater than that of the split-body aircraft, the split-body aircraft will always be hit unless emergency measures are taken. The trajectory of the missile hitting the split-body aircraft after locking onto it is as follows. Figure 6 As shown, the dashed line represents the missile's trajectory, and the solid line represents the trajectory of the split-type aircraft. In this scenario, the missile and the split-type aircraft encounter each other at a large angle, potentially hitting the side of the aircraft. When the aircraft separates, the missile can easily affect or even hit the front sub-unit, which is the core component. Therefore, it is desirable for the missile and the split-type aircraft to maintain the same speed and direction, so that the missile hits the rear sub-unit from behind without affecting the escape of the front sub-unit.

[0078] Due to the high speeds of the split-plane aircraft and missile, a control step size of 0.1s is chosen. Aircraft radar typically detects limited data, including the x-axis and y-axis distances between the missile and aircraft, and the directional angle between them. This radar data is used as the state input. The split-plane aircraft's turning speed is 16.5 degrees / second, and the action chosen is the aircraft's turning. With a step size of 0.1s, the angle range for each turn is -1 to 1 degree. The Tomahawk cruise missile has a range of 90 meters. Based on the estimated width of the front and rear sub-planes, to ensure the rear sub-plane blocks the missile and the front sub-plane is not affected by the explosion, the angle difference between the rear and front sub-planes relative to the missile's velocity direction must be less than 1 degree, and the distance between the front and rear sub-planes must be greater than 100 meters. Since there is also a requirement to ensure the split-plane aircraft flies smoothly as quickly as possible, the reward is designed as follows: when the angle difference between the split-plane and missile's velocity direction is less than 1 degree, the reward increases by 1; otherwise, the reward remains unchanged. The above reinforcement learning framework is as follows: Figure 7 As shown, when the distance between the split aircraft and the missile is less than 1km, it is considered that the missile will hit the split aircraft in the next second (because the speed of the missile is set to 4000km / h and the speed of the aircraft is set to 1000km / h), and the next training session begins.

[0079] After the environment, observations, actions, and rewards are all designed, reinforcement learning training can begin. First, the four neural networks for the DDPG algorithm are initialized: actor_eval, actor_target, q_eval, and q_target. The DDPG algorithm uses an actor-critic architecture, with actor_eval and actor_target as the actor networks, and q_eval and q_target as the critic networks. actor_eval and q_eval are primarily used for exploration and updating, while actor_target and q_target networks update the eval network gradually to avoid a sharp drop in rewards due to large updates. These four networks share the same intermediate structure: two fully connected layers with 32 nodes each, using ReLU activation. Then, starting from the first round, in each round, the actor_eval network selects the agent's (split-plane) action (rotation angle) based on the current state. The current state, the next state, the action taken, and the corresponding reward are stored in an array in the replay buffer (with a capacity of 10,000 such arrays) for subsequent training. When the replay buffer is full, training begins: the q_eval network uses the TD method with gamma set to 0.9, and the TD error is backpropagated for training. The actor_eval network uses the output of the q_eval network (i.e., the value function Q(s, a)) as the target to be maximized, and is trained using gradient ascent. The learning rate for both networks during training is set to 0.001. After training, the network parameters are propagated to the actor_target and q_target networks according to the definition of hyperparameter τ (set to 0.001): target = τ * eval + (1 - τ) * target. Training ends after 150 epochs. The actor_target network is the trained agent decision network, and the q_target network is the trained evaluation network.

[0080] 3. Experimental Results

[0081] The training results of DDPG (reward-episode graph of DDPG algorithm) are as follows: Figure 8 As shown, since the initial distances between the split-plane aircraft and the missile are different each time, the reward is normalized by taking the ratio of the total reward within the time interval to the time taken for the split-plane aircraft and the missile to meet. After 150 training runs, DDPG converged very well, with fluctuations within an acceptable range. The upper limit of the reward value is 10 because the control step size is chosen to be 0.1s.

[0082] Save the actor_target network from the trained DDPG, randomly select an initial distance between the split-type aircraft and the missile, and the angle after the split-type aircraft takes action is as follows: Figure 9 As shown, the solid lines represent the flight angles of the split-type aircraft relative to the coordinate system, and the dashed lines represent the flight angles of the missile relative to the coordinate system. After the split-type aircraft aligns its speed and direction with the missile, it will experience slight oscillations. In practical applications, the design conditions should keep it stationary.

[0083] In this case, the flight trajectory of the split-type aircraft is as follows: Figure 10 As shown in the diagram. The solid lines represent the aircraft's flight trajectory relative to the coordinate system, and the dashed lines represent the missile's flight trajectory relative to the coordinate system. It is quite clear that the aircraft made rapid movements to ensure its speed and direction were aligned with the missile's.

[0084] The above control system implements a simple function to adjust the aircraft's angle. By controlling the split-plane to separate approximately 0.5 seconds before it encounters the missile (at a distance of about 500 meters), it is possible to ensure that the rear split-plane is hit by the missile while the front split-plane remains unaffected. Based on practical considerations, more elements can be added to the state, and corresponding rewards can be designed to encourage the split-plane to fly more efficiently.

[0085] This invention also includes an animated video demonstration of a split-plane dodging missiles. The demonstration video was created using the Kerbal Space Program game on the STEAM platform. The specific steps are as follows:

[0086] First, draw the front sub-plane in the game based on the CAD model, then set the rear landing gear of the front sub-plane to the retracted state. Next, connect the front and rear sub-planes of the split aircraft using a tiered device, thus allowing the split aircraft to separate. Finally, equip the weapons and set the activation status of the engines of the front and rear sub-planes to trigger after separation and trigger before separation, respectively.

[0087] Then, the split-type aircraft is placed on the runway, its engines are started, and the throttle and elevators are operated to take off. Once the split-type aircraft reaches a certain altitude, its landing gear is retracted. After stabilizing, the split-type aircraft enters cruise mode, after which it can engage missiles and launch machine guns. Ground-based missile launch units fire Patriot missiles at the split-type aircraft. After the missile locks onto the split-type aircraft, the missile's speed is much greater than the aircraft's. Before the missile hits the entire split-type aircraft, it jettisons its rear sub-engine and activates the inactive backup engine of the front sub-engine. At this point, the missile will hit the rear sub-engine, while the front sub-engine continues to fly. Finally, the split-type aircraft uses the main landing gear of the entire aircraft and the separate backup main landing gear of the front sub-engine to complete the landing, achieving the function of split-type missile evasion. The animated demonstration video shows schematics of the entire aircraft and the front and rear sub-engines after splitting, as shown below. Figure 11a and Figure 11b As shown.

[0088] Typically, aircraft escape by releasing flares to mislead infrared missiles. Once the flares are depleted, the aircraft can only be hit. However, the split-type aircraft in this invention can evade missiles by splitting up when the flares are exhausted.

[0089] This invention employs the DDPG method to ensure the directional alignment of the split-unit aircraft and the missile, guaranteeing that the missile can only hit the rear sub-unit, allowing the front sub-unit to escape. Reinforcement learning methods can incorporate observations into the state based on the split-unit aircraft's needs and adjust the reward, enabling the aircraft to fly in the optimal manner.

[0090] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or purpose, should be considered within the scope of protection of the present invention.

Claims

1. A method for missile evasion by a split-type aircraft based on deep reinforcement learning, characterized in that, Includes the following steps: S1. For the control of split-type aircraft, a decision network and evaluation network based on the DDPG algorithm are constructed and trained by reinforcement learning. The split aircraft includes a front sub-unit, a rear sub-unit, and a front-to-rear sub-unit separation device. The front sub-unit has a nose landing gear, a spare main landing gear, and a spare engine. The rear sub-unit has a main landing gear and a main engine. During training, the observed distances between the missile and the aircraft along the x-axis, y-axis, and directional angle on the xy-plane of the aircraft's coordinate system are input into the decision network as state. The decision network outputs actions to the evaluation network, which then assigns rewards and updates network parameters based on the state and actions. The decision network performs gradient ascent based on the scores to obtain an updated strategy. This updated strategy is used to control the aircraft's turning so that it aligns with the direction of the incoming missile, ensuring that the missile can only hit the rear sub-plane from behind. This avoids the missile hitting the front sub-plane due to a large speed angle between the missile and the aircraft. Furthermore, the rear sub-plane is separated from the front sub-plane before the missile hits the aircraft, allowing the front sub-plane to maintain stable flight as quickly as possible. S2. During flight, the x-axis distance, y-axis distance, and directional angle actually observed by the aircraft radar are input into the decision network as status. The decision network controls the aircraft to turn and separate the rear sub-engine from the front sub-engine. When the rear sub-engine separates from the front sub-engine, the backup engine and the previously retracted backup main landing gear of the front sub-engine are activated, and the front sub-engine uses the nose landing gear and the backup main landing gear to complete the landing.

2. The method as described in claim 1, characterized in that, In step S1, the step size period for controlling the aircraft's steering is 0.1s.

3. The method as described in claim 1, characterized in that, In step S1, the angle of aircraft turning is controlled from -1 degree to 1 degree each time.

4. The method as described in claim 1, characterized in that, In step S1, controlling the aircraft's direction to align with the missile's direction means controlling the angle between the aircraft's and the missile's directions to be no greater than 1 degree.

5. The method as described in claim 1, characterized in that, In step S1, when the angular difference between the speed direction of the aircraft and the missile is less than 1 degree, the reward is increased; otherwise, the reward remains unchanged.

6. The method according to any one of claims 1 to 5, characterized in that, The evaluation network is trained using the TD method, with the TD error backpropagated for training; the decision network uses a value function as the maximization objective and is trained using the gradient ascent method. Determine the network parameters after training is complete.

7. A split-type aircraft, characterized in that, The device includes a front sub-engine, a rear sub-engine, a front and rear sub-engine separation device, and a control module. The front sub-engine has a nose landing gear, a spare main landing gear, and a spare engine. The rear sub-engine has a main landing gear and a main engine. The control module is configured to perform the method for missile avoidance by a split-type aircraft based on deep reinforcement learning as described in any one of claims 1 to 6.

8. The split-type aircraft as described in claim 7, characterized in that, Both the front sub-engine and the rear sub-engine are equipped with twin vertical tail fins.

9. The split-type aircraft as described in claim 7 or 8, characterized in that, The front and rear submachine separation device is a device with thrust.

Images