Aircraft collision angle defense method and system based on reinforcement learning and sliding mode control

Abstract

The invention relates to the technical field of aircraft interception, and provides an aircraft collision angle defense method and system based on reinforcement learning and sliding mode control. Constructing a three-body dynamic model of an incoming attack aircraft, a target and a defense aircraft; constructing a guidance law design target according to the three-body dynamics model; constructing a defense aircraft sliding mode guidance law according to the three-body dynamics model and the guidance law design target; performing deep learning on the defense aircraft sliding mode guidance law to obtain a defense model; and intercepting the aircraft according to the defense model. According to the method, deep reinforcement learning and traditional sliding mode control are combined, an intelligent agent is guided to carry out non-blind exploration in the early stage, an integer reward function is added in the design of the reward function, rapid convergence of reinforcement learning is achieved, and the method has the advantages of being simple in structure and convenient to use. Active defense guidance of defense aircrafts in three-body confrontation is completed, and the incoming aircrafts are intercepted with high precision and angle constraints.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft interception, and provides an aircraft collision angle defense method and system based on reinforcement learning and sliding mode control. Background Art

[0002] As the use of drones becomes more and more common, the flight of drones may be unrestricted due to drone malfunctions. If a drone intrudes into the approach area of ​​an aircraft or even the flight path of an aircraft, the consequences will be disastrous. Therefore, the protection of airports is imminent.

[0003] At present, the main methods in the field of UAV interception include proportional navigation guidance (PNG), optimal control method, sliding mode control method and differential game method, etc. In proportional guidance, the acceleration of the UAV is proportional to the line of sight angular rate of the aircraft target connection line, and it is widely used in the guidance field because of its simple structure.

[0004] In the optimal control method, the guidance law of the UAV aircraft to intercept the target is designed by selecting the optimal indicators of different performances. The differential game method is based on the optimal control theory, considering the optimal control problems of both parties, and achieving the optimal goals of the UAV aircraft and the target. Sliding mode control theory, in nonlinear systems, has been greatly promoted because it is not affected by model uncertainty and external disturbances, and does not require accurate maneuvering information of the target in advance.

[0005] With the development of artificial intelligence technology, deep reinforcement learning is used in the guidance field. The guidance problem is regarded as a Markov decision process. By setting the reward function and optimizing the strategy network, the adaptive online output of the acceleration of the UAV aircraft to intercept the target is realized. Summary of the invention

[0006] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides an aircraft collision angle defense method and system based on reinforcement learning and sliding mode control to achieve rapid interception of incoming aircraft.

[0007] The present invention provides an aircraft collision angle defense method based on reinforcement learning and sliding mode control, comprising the following steps: S1: Construct a three-body dynamics model of the attacking aircraft, the target and the defensive aircraft; S2: constructing a guidance law design target according to the three-body dynamics model; S3: constructing a sliding mode guidance law for a defense aircraft according to the three-body dynamics model and the guidance law design target; S4: Performing GAIL-SAC learning on the sliding mode guidance law of the defense aircraft to obtain a defense model; S5: The position, velocity and acceleration of the attacking aircraft; the position, velocity and acceleration of the target; and the position, velocity and acceleration of the defending aircraft are input into the defense model to intercept the aircraft.

[0008] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S1 includes: S11: Construct a two-dimensional kinematic model between the incoming aircraft and the target: in, is the relative speed between the incoming aircraft and the target, is the speed of the incoming aircraft, is the heading angle of the incoming aircraft velocity relative to the inertial coordinate system, is the line of sight angle between the incoming aircraft and the target relative to the inertial coordinate system, is the heading angle of the target velocity relative to the inertial coordinate system, is the target speed, for The first derivative of for The second derivative of is the acceleration of the incoming aircraft, is the relative distance between the incoming aircraft and the target, is the target acceleration, for The first derivative of for The first derivative of S12: Construct a two-dimensional kinematic model between the attacking aircraft and the defending aircraft: in, represents the relative speed between the attacking aircraft and the defending aircraft, It represents the line of sight angle between the attacking aircraft and the defending aircraft relative to the inertial coordinate system, Indicates the speed of the defending aircraft, It represents the heading angle of the defense aircraft relative to the inertial coordinate system. for The first derivative of for The second derivative of Indicates the relative distance between the attacking aircraft and the defending aircraft. represents the acceleration of the defending aircraft, for The first derivative of S13: The two-dimensional kinematic model between the incoming aircraft and the target and the two-dimensional kinematic model between the incoming aircraft and the defending aircraft are combined to form a three-body dynamic model of the incoming aircraft-target-defending aircraft.

[0009] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S2 includes: S21: Design the interception conditions for the defensive aircraft to intercept the incoming aircraft: in, The desired sight angle for the defending aircraft; S22: Define the guidance law design objective according to the interception condition: in, To prevent the aircraft’s line of sight angle tracking error, for The first derivative of .

[0010] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S3 includes: S31: Define the sliding surface: in, is the designed sliding surface, is the first adjustment parameter, is the second adjustment parameter, is the first exponential parameter, is the second exponential parameter, To define a symbolic function, for The independent variable of the function, To take the absolute value, is a symbolic function; S32: Design convergence law: in, is the first derivative of the sliding surface, is a gain that takes a positive value in the reaching law; S33: Design a sliding mode guidance law for the defense aircraft according to the sliding mode surface and the reaching law: in, is the scale factor of the proportional guidance.

[0011] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S31 includes: The value range of is: ; The value range of is: ; The value range of is: ; .

[0012] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S4 includes: S41: constructing a SAC algorithm model according to the sliding mode guidance law of the defensive aircraft, and generating a strategy according to the SAC algorithm model; S42: using the generated strategy as an expert strategy, inputting the state space and action space of the SAC algorithm model and the expert strategy into a discriminator, and obtaining the possibility derived from the state space and action space; S43: Update the SAC algorithm model according to the possibilities derived from the state space and the action space to obtain the defense model.

[0013] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S41 includes: S411: Set the state space, action space and reward function of the SAC algorithm: in, is the state space, is the action space, is the reward function, is the first reward parameter, The distance between the defending aircraft and the attacking aircraft at the initial moment, is the second reward parameter, is the third reward parameter, represents the distance between the incoming aircraft and the target at the initial moment, is the fourth reward parameter, is the fifth reward parameter, is the sixth reward parameter; S412: Design the optimal strategy function for the SAC algorithm: in, the strategy to adopt for the next moment; The strategy adopted at the current moment, To find the maximum value function, For expectations, For the current moment, is the expected reward after taking an action in the current state following the strategy, is the current state, is the action at the current moment, is the regularization coefficient, is the information entropy, The state of following the policy Take Action ; S413: Constructing the value of the objective function and value function : in, The expected objective function value of the state at the current moment, The expected value obtained after taking an action in the current state of following the strategy, is the discount factor, for Always follow the strategy The expected value of the state after taking the action; To follow the strategy state Take Action ; S414: Calculate the loss function of the objective function value and the loss function of the value function : in, is the reward at the current moment, To obtain the minimum value, is the marking ordinal number, is the expected value corrected by the loss function of the value function, is the Gaussian distribution function, is the standard normal distribution noise.

[0014] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S42 includes: S421: and As the input of the discriminator, As a generator, it is trained to obtain the possibilities from the state space and action space; S422: According to the loss function of the discriminator Modify the training: in, is the input function of the discriminator.

[0015] According to an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention, step S43 includes: S431: Activate the possibilities from the state space and the action space through an activation function to obtain an automatically adjusted entropy regularization term ; S432: Modify the reinforcement learning objective and rewrite it as: in, To obtain the maximum value, is the target entropy, Indicates restrictions; S433: Loss function based on automatic adjustment of entropy regularization term Modify the training: .

[0016] The present invention also provides an aircraft collision angle defense system based on reinforcement learning and sliding mode control, comprising: a model building module: constructing a three-body dynamics model of an incoming aircraft-a target-a defensive aircraft; constructing a guidance law design target according to the three-body dynamics model; constructing a sliding mode guidance law of the defensive aircraft according to the three-body dynamics model and the guidance law design target; Model learning module: performing deep learning on the sliding mode guidance law of the defense aircraft to obtain a defense model; Aircraft interception module: the position, velocity and acceleration of the attacking aircraft; the position, velocity and acceleration of the target; the position, velocity and acceleration of the defending aircraft are input into the defense model to intercept the aircraft.

[0017] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The present invention provides an aircraft collision angle defense method and system based on reinforcement learning and sliding mode control. By combining deep reinforcement learning with traditional sliding mode control, the intelligent agent is guided to conduct non-blind exploration in the early stage. In the design of the reward function, an integer reward function is added to achieve rapid convergence of reinforcement learning, and the active defense guidance of the defensive aircraft in the three-body confrontation is completed to intercept the incoming aircraft with higher accuracy and angle constraints.

[0018] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention; Figure 2 This is a schematic diagram of GAIL-SAC learning; Figure 3 is a schematic diagram of active defense trajectory of an embodiment of the present invention; Figure 4 is a graph showing changes in the sight angle between the defending aircraft and the attacking aircraft over time according to an embodiment of the present invention; Figure 5 It is a structural block diagram of an aircraft collision angle defense device based on reinforcement learning and sliding mode control provided by the present invention; Reference numerals: 101. Model building module; 102. Model learning module; 103. Aircraft interception module. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme in the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0022] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0023] Combine the following Figures 1 to 5 The present invention is described.

[0024] Example like Figure 1 As shown, Figure 1 A schematic flow chart of an aircraft collision angle defense method based on reinforcement learning and sliding mode control provided by the present invention comprises the following steps: S1: Construct a three-body dynamics model of the attacking aircraft, the target and the defensive aircraft; S2: constructing a guidance law design target according to the three-body dynamics model; S3: constructing a sliding mode guidance law for a defense aircraft according to the three-body dynamics model and the guidance law design target; S4: Performing GAIL-SAC learning on the sliding mode guidance law of the defense aircraft to obtain a defense model; S5: The position, velocity and acceleration of the attacking aircraft; the position, velocity and acceleration of the target; and the position, velocity and acceleration of the defending aircraft are input into the defense model to intercept the aircraft.

[0025] Specifically, step S1 includes: S11: Construct a two-dimensional kinematic model between the incoming aircraft and the target: in, is the relative speed between the incoming aircraft and the target, is the speed of the incoming aircraft, is the heading angle of the incoming aircraft velocity relative to the inertial coordinate system, is the line of sight angle between the incoming aircraft and the target relative to the inertial coordinate system, is the heading angle of the target velocity relative to the inertial coordinate system, is the target speed, for The first derivative of for The second derivative of is the acceleration of the incoming aircraft, is the relative distance between the incoming aircraft and the target, is the target acceleration, for The first derivative of for The first derivative of S12: Construct a two-dimensional kinematic model between the attacking aircraft and the defending aircraft: in, represents the relative speed between the attacking aircraft and the defending aircraft, It represents the line of sight angle between the attacking aircraft and the defending aircraft relative to the inertial coordinate system, Indicates the speed of the defending aircraft, It represents the heading angle of the defense aircraft relative to the inertial coordinate system. for The first derivative of for The second derivative of Indicates the relative distance between the attacking aircraft and the defending aircraft. represents the acceleration of the defending aircraft, for The first derivative of S13: The two-dimensional kinematic model between the incoming aircraft and the target and the two-dimensional kinematic model between the incoming aircraft and the defending aircraft are combined to form a three-body dynamic model of the incoming aircraft-target-defending aircraft.

[0026] Specifically, step S2 includes: S21: Design the interception conditions for the defensive aircraft to intercept the incoming aircraft: in, The desired sight angle for the defending aircraft; S22: Define the guidance law design objective according to the interception condition: in, To prevent the aircraft’s line of sight angle tracking error, for The first derivative of .

[0027] Since the present invention takes into account the problem of limited terminal acceleration of the aircraft, it is defined as follows: in, is the acceleration ordinal number, , It is the maximum maneuverability that each aircraft actuator can provide during the guidance process.

[0028] Specifically, step S3 includes: S31: Define the sliding surface: in, is the designed sliding surface, is the first adjustment parameter, is the second adjustment parameter, is the first exponential parameter, is the second exponential parameter, To define a symbolic function, for The independent variable of the function, To take the absolute value, is a symbolic function; The value range of is: ; The value range of is: ; The value range of is: ; .

[0029] S32: Design convergence law: in, is the first derivative of the sliding surface, is a gain that takes a positive value in the reaching law; S33: Design a sliding mode guidance law for the defense aircraft according to the sliding mode surface and the reaching law: in, is the scale factor of the proportional guidance.

[0030] In the three-body active defense problem, the motion status information of the three aircraft is abstracted into a Markov decision process. The intelligent agent selects actions by continuously interacting with the environment and optimizes its own strategy by maximizing rewards.

[0031] In this paper, the Generative Adversarial Imitation Learning (GAIL) in reinforcement learning is combined with the traditional SAC (Soft Actor-Critic, flexible action evaluation algorithm) to construct an intelligent agent for defending aircraft in a three-body confrontation environment. The algorithm network is as follows Figure 1 As shown in Figure 1, the generator in GAIL-SAC is composed of a strategy network formed by the SAC algorithm for the defense aircraft, and the discriminator is composed of a neural network. According to the guidance law of the defense aircraft designed by the traditional sliding mode control in step S3, the initial positions and performance states of the three aircraft in the random initial confrontation environment based on the Monte Carlo idea are collected to collect expert data sets for training the network.

[0032] In the GAIL-SAC algorithm, the generator uses the set state as the input of the network, the action to be performed by the defense aircraft as the output, and collects the generated data trajectory into the memory, and updates the network parameters of the generator so that the data generated by the generator is as close to the expert data as possible. In the discriminator network, the expert data set and the three-body environment aircraft generated by the generator use the intelligent body state and action as the input of the neural network. At the same time, the neural network uses the Sigmoid function as the activation function of the network output layer, sets the data label generated by the generator to 0, and the expert data label to 1, calculates the loss degree of the discriminator model through the BCELoss function, and uses the algorithm RMSprop to optimize the network parameters, so that the discriminator can accurately evaluate the gap between the generator and the expert experience, and then guide the training of the generator.

[0033] like Figure 2 As shown, Figure 2 This is a schematic diagram of GAIL-SAC learning. Specifically, step S4 includes: S41: constructing a SAC algorithm model according to the sliding mode guidance law of the defensive aircraft, and generating a strategy according to the SAC algorithm model; S411: Set the state space, action space and reward function of the SAC algorithm: in, is the state space, is the action space, is a reward function, which is an integer reward function, is the first reward parameter, The distance between the defending aircraft and the attacking aircraft at the initial moment, is the second reward parameter, is the third reward parameter, represents the distance between the incoming aircraft and the target at the initial moment, is the fourth reward parameter, is the fifth reward parameter, is the sixth reward parameter; S412: Design the optimal strategy function for the SAC algorithm: in, the strategy to adopt for the next moment; The strategy adopted at the current moment, To find the maximum value function, For expectations, For the current moment, is the expected reward after taking an action in the current state following the strategy, is the current state, is the action at the current moment, is the regularization coefficient, is the information entropy, The state of following the policy Take Action ; S413: Constructing the value of the objective function and value function : in, The expected objective function value of the state at the current moment, The expected value obtained after taking an action in the current state of following the strategy, is the discount factor, for Always follow the strategy The expected value of the state after taking the action; To follow the strategy state Take Action ; S414: Calculate the loss function of the objective function value and the loss function of the value function : in, is the reward at the current moment, To obtain the minimum value, is the marking ordinal number, is the expected value corrected by the loss function of the value function, is the Gaussian distribution function, is the standard normal distribution noise.

[0034] In reinforcement learning, the PPO algorithm (Proximal Policy Optimization) represented by the online policy algorithm requires a large number of samples for learning, and has the problem of low sample efficiency. The DDPG algorithm (deep deterministic policy gradient) is an offline policy algorithm. Although it improves the utilization rate of samples, DDPG is a deterministic strategy, that is, it only considers the best action in each state. It has the problems of insufficient exploration ability, unstable training, poor convergence, sensitivity to hyperparameters, and difficulty in adapting to different complex environments. The SAC algorithm is an offline policy algorithm with a random strategy. Compared with the PPO algorithm, it has higher sampling efficiency and stronger exploration ability than DDPG.

[0035] S42: using the generated strategy as an expert strategy, inputting the state space and action space of the SAC algorithm model and the expert strategy into a discriminator, and obtaining the possibility derived from the state space and action space; S421: and As the input of the discriminator, As a generator, it is trained to obtain the possibilities from the state space and action space; S422: According to the loss function of the discriminator Modify the training: in, is the input function of the discriminator.

[0036] S43: Update the SAC algorithm model according to the possibilities derived from the state space and the action space to obtain the defense model.

[0037] S431: Activate the possibilities from the state space and the action space through an activation function to obtain an automatically adjusted entropy regularization term ; S432: Modify the reinforcement learning objective and rewrite it as: in, To obtain the maximum value, is the target entropy, Indicates restrictions; S433: Loss function based on automatic adjustment of entropy regularization term Modify the training: .

[0038] The effectiveness of the embodiments of the present invention is demonstrated below: Firstly, a three-system guided simulation environment is constructed, and the initial state parameters are shown in Table 1.

[0039] Table 1 Monte Carlo simulation environment parameters in, is the position coordinate of the target aircraft, are the coordinates of the incoming aircraft, is the acceleration due to gravity.

[0040] In the Monte Carlo experiment, the defense aircraft initially adopted the sliding mode guidance law designed by the present invention to collect expert data sets. The range is , velocity inclination , the defense aircraft overload range is . A dataset of 3,000 experts is constructed and used in the GAIL-SAC algorithm to guide the agent to imitate the expert experience, thereby protecting the target and achieving active defense.

[0041] Table 2 shows the settings of network parameters in the GAIL-SAC algorithm for intelligent networks.

[0042] Table 2 GAIL-SAC network parameter setting table In order to verify the algorithm proposed above, the optimal agent neural network parameters and weights obtained by GAIL-SAC training are saved, the initial state of the three bodies is reinitialized, and a simulation test experiment is performed. In the test experiment, the defense aircraft and the target start from the same position at the initial moment, and the initial position is (0, 0). The initial speed of the defense aircraft is 750 , the heading angle is , the target's initial speed is 400 , the heading angle is , the initial position of the incoming aircraft is (10000, 0), and the initial speed is 400 , the heading angle is The acceleration of the target is , the attacking aircraft intercepts the target with proportional guidance, where the proportional coefficient is 3. The expected sight angle of the defensive aircraft intercepting the raiding aircraft is .

[0043] like Figure 3 As shown, Figure 3 is an active defense trajectory diagram of an embodiment of the present invention, from Figure 3 It can be seen that the defensive aircraft can intercept the incoming aircraft within the specified time to ensure that the target is attacked, and its active defense miss margin is 0.0924, which meets the requirements.

[0044] like Figure 4 As shown, Figure 4 The graph of the sight angle between the defensive aircraft and the attacking aircraft according to the embodiment of the present invention changes with time. It can be seen from the graph that the defensive aircraft has a desired sight angle value. , to intercept the incoming aircraft.

[0045] like Figure 5 As shown, an aircraft collision angle defense system based on reinforcement learning and sliding mode control provided by the present invention is described below. The aircraft collision angle defense system based on reinforcement learning and sliding mode control described below and the aircraft collision angle defense method based on reinforcement learning and sliding mode control described above can refer to each other.

[0046] Model building module 101: constructing a three-body dynamics model of an attacking aircraft, a target, and a defensive aircraft; constructing a guidance law design target based on the three-body dynamics model; constructing a sliding mode guidance law for a defensive aircraft based on the three-body dynamics model and the guidance law design target; Model learning module 102: performing deep learning on the sliding mode guidance law of the defense aircraft to obtain a defense model; Aircraft interception module 103: The position, velocity and acceleration of the incoming aircraft; the position, velocity and acceleration of the target; the position, velocity and acceleration of the defending aircraft are input into the defense model to intercept the aircraft. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0047] The beneficial effects of the present invention include: combining traditional sliding mode guidance with reinforcement learning, and proposing an active defense guidance method based on reinforcement learning GAIL-SAC combined with sliding mode control for complex three-body confrontation environments. Under the guidance of the expert strategy composed of sliding mode control, the guidance method ensures that in a complex three-body confrontation environment, the defense aircraft can complete the interception of the incoming aircraft under the terminal angle constraint within the specified maximum time, and has a high hit accuracy, preventing the target from being attacked by the incoming aircraft.

[0048] The present invention can accelerate the convergence speed of the network, realize the online autonomous decision-making of the defense aircraft, and improve the success rate of the defense aircraft in intercepting the incoming aircraft.

Claims

1. A method for aircraft collision angle defense based on reinforcement learning and sliding mode control, characterized in that: The following steps are involved: S1: Construct a three-body dynamics model of the attacking aircraft, the target and the defensive aircraft; S2: constructing a guidance law design target according to the three-body dynamics model; S3: constructing a sliding mode guidance law for a defense aircraft according to the three-body dynamics model and the guidance law design target; S4: Performing GAIL-SAC learning on the sliding mode guidance law of the defense aircraft to obtain a defense model; S5: The position, velocity and acceleration of the attacking aircraft; the position, velocity and acceleration of the target; and the position, velocity and acceleration of the defending aircraft are input into the defense model to intercept the aircraft.

2. The method for preventing collision angle of aircraft based on reinforcement learning and sliding mode control according to claim 1, characterized in that: Step S1 includes: S11: Construct a two-dimensional kinematic model between the incoming aircraft and the target: in, is the relative speed between the incoming aircraft and the target, is the speed of the incoming aircraft, is the heading angle of the incoming aircraft velocity relative to the inertial coordinate system, is the line of sight angle between the incoming aircraft and the target relative to the inertial coordinate system, is the target speed, is the heading angle of the target velocity relative to the inertial coordinate system, for The first derivative of for The second derivative of is the acceleration of the incoming aircraft, is the relative distance between the incoming aircraft and the target, is the target acceleration, for The first derivative of for The first derivative of S12: Construct a two-dimensional kinematic model between the attacking aircraft and the defending aircraft: in, represents the relative speed between the attacking aircraft and the defending aircraft, It represents the line of sight angle between the attacking aircraft and the defending aircraft relative to the inertial coordinate system, Indicates the speed of the defending aircraft. It represents the heading angle of the defense aircraft relative to the inertial coordinate system. for The first derivative of for The second derivative of Indicates the relative distance between the attacking aircraft and the defending aircraft. represents the acceleration of the defending aircraft, for The first derivative of S13: The two-dimensional kinematic model between the incoming aircraft and the target and the two-dimensional kinematic model between the incoming aircraft and the defending aircraft are combined to form a three-body dynamic model of the incoming aircraft-target-defending aircraft.

3. The method for preventing collision angle of aircraft based on reinforcement learning and sliding mode control according to claim 2, characterized in that: Step S2 includes: S21: Design the interception conditions for the defensive aircraft to intercept the incoming aircraft: in, The desired sight angle for the defending aircraft; S22: Define the guidance law design objective according to the interception condition: in, To prevent the aircraft’s line of sight angle tracking error, for The first derivative of .

4. The method for aircraft collision angle defense based on reinforcement learning and sliding mode control according to claim 3, characterized in that: Step S3 includes: S31: Define the sliding surface: in, is the designed sliding surface, is the first adjustment parameter, is the second adjustment parameter, is the first exponential parameter, is the second exponential parameter, To define a symbolic function, for The independent variable of the function, To take the absolute value, is a symbolic function; S32: Design convergence law: in, is the first derivative of the sliding surface, is a gain that takes a positive value in the reaching law; S33: Design a sliding mode guidance law for the defense aircraft according to the sliding mode surface and the reaching law: in, is the scale factor of the proportional guidance.

5. The method for aircraft collision angle defense based on reinforcement learning and sliding mode control according to claim 4, characterized in that: Step S31 includes: The value range of is: ; The value range of is: ; The value range of is: ; .

6. The method for aircraft collision angle defense based on reinforcement learning and sliding mode control according to claim 4, characterized in that: Step S4 includes: S41: constructing a SAC algorithm model according to the sliding mode guidance law of the defensive aircraft, and generating a strategy according to the SAC algorithm model; S42: using the generated strategy as an expert strategy, inputting the state space and action space of the SAC algorithm model and the expert strategy into a discriminator, and obtaining the possibility derived from the state space and action space; S43: Update the SAC algorithm model according to the possibilities derived from the state space and the action space to obtain the defense model.

7. The method for aircraft collision angle defense based on reinforcement learning and sliding mode control according to claim 6, characterized in that: Step S41 includes: S411: Set the state space, action space and reward function of the SAC algorithm: in, is the state space, is the action space, is the reward function, is the first reward parameter, The distance between the defending aircraft and the attacking aircraft at the initial moment, is the second reward parameter, is the third reward parameter, represents the distance between the incoming aircraft and the target at the initial moment, is the fourth reward parameter, is the fifth reward parameter, is the sixth reward parameter; S412: Design the optimal strategy function for the SAC algorithm: in, the strategy to adopt for the next moment; The strategy adopted at the current moment, To find the maximum function, For expectations, For the current moment, is the expected reward after taking an action in the current state following the strategy, is the current state, is the action at the current moment, is the regularization coefficient, is the information entropy, The state of following the policy Take Action ; S413: Constructing the value of the objective function and value function : in, The expected objective function value of the state at the current moment, The expected value obtained after taking an action in the current state of following the strategy, is the discount factor, for Always follow the strategy The expected value of the state after taking the action; To follow the strategy state Take Action ; S414: Calculate the loss function of the objective function value and the loss function of the value function : in, is the reward at the current moment, To obtain the minimum value, is the marking ordinal number, is the expected value corrected by the loss function of the value function, is the Gaussian distribution function, is the standard normal distribution noise.

8. The method for aircraft collision angle defense based on reinforcement learning and sliding mode control according to claim 7, characterized in that: Step S42 includes: S421: and As the input of the discriminator, As a generator, it is trained to obtain the possibilities from the state space and action space; S422: According to the loss function of the discriminator Modify the training: in, is the input function of the discriminator.

9. The method for aircraft collision angle defense based on reinforcement learning and sliding mode control according to claim 8, characterized in that: Step S43 includes: S431: Activate the possibilities from the state space and the action space through an activation function to obtain an automatically adjusted entropy regularization term ; S432: Modify the reinforcement learning objective and rewrite it as: in, To obtain the maximum value, is the target entropy, Indicates restrictions; S433: Loss function based on automatic adjustment of entropy regularization term Modify the training: 。 10. An aircraft collision angle defense system based on reinforcement learning and sliding mode control, used to execute an aircraft collision angle defense method based on reinforcement learning and sliding mode control as claimed in any one of claims 1 to 9, characterized in that: include: Model building module: constructing a three-body dynamics model of the attacking aircraft, the target and the defensive aircraft; constructing the guidance law design target according to the three-body dynamics model; Constructing a sliding mode guidance law for a defense aircraft according to the three-body dynamics model and the guidance law design objective; Model learning module: performing deep learning on the sliding mode guidance law of the defense aircraft to obtain a defense model; Aircraft interception module: the position, velocity and acceleration of the attacking aircraft; the position, velocity and acceleration of the target; the position, velocity and acceleration of the defending aircraft are input into the defense model to intercept the aircraft.

Images