Intelligent guidance law based on deep neural network and prediction method thereof

Through the intelligent guidance law based on deep neural network, the target maneuvering ballistics and guidance points are predicted, and the problem of the reduction in accuracy and performance of traditional guidance algorithms in dynamic environments is solved, and efficient prediction and dynamic scene adaptation are achieved under small data samples, improving the efficiency of prediction and dynamic scene adaptation. intercepting performance of interceptor missiles.

CN119002536APending Publication Date: 2024-11-22NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411076687.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Traditional guidance algorithms have reduced guidance accuracy and performance when facing dynamic environments, and cannot effectively adapt to the reverse track interception guidance problem under the law of uncertain target maneuver prediction.

Method used

Adopt intelligent guidance law based on deep neural networks, including target maneuver branch prediction model, predict guidance point calculation model and deep neural network model. Through the Seq2Seq model and DQN algorithm, target maneuver ballistic data and guidance points are predicted, and the guidance control quantity is calculated, and the guidance control quantity is calculated. Dynamic environment and improve interception performance.

Benefits of technology

Predict specific ballistics in advance under small data samples, learn dynamic scenarios and optimal guidance paths through a large amount of training data, improve the interception performance of interceptor missiles and adapt to complex interception scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002536A_ABST
    Figure CN119002536A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of guidance, and relates to an intelligent guidance law based on a deep neural network and a prediction method thereof.The intelligent guidance law comprises a target maneuvering branch prediction model, a prediction guidance point calculation model and a deep neural network model.Target maneuvering trajectory data are predicted through the target maneuvering branch prediction model; calculating a guidance point by a prediction guidance point calculation model according to the target maneuvering trajectory data, and calculating a guidance control quantity by a deep neural network model according to the guidance point; target maneuvering is predicted in a data driving mode, so that a certain type of specific trajectory can be predicted in advance under a small data sample, and a predicted interception point is calculated according to the trajectory; meanwhile, a deep neural network is adopted, a change scene for predicting interception point maneuvering and an optimal guidance path can be learned through a large amount of training data, it is ensured that the interception missile can adapt to a dynamic and complex interception scene, and then the interception performance of the interception missile is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of guidance technology, and specifically relates to an intelligent guidance law based on a deep neural network and a method for predicting guidance points thereof. Background Art

[0002] Guidance law refers to an algorithm that calculates the path of aircraft such as missiles and drones based on navigation information so that they can reach the target point effectively. Traditional guidance algorithms such as optimal guidance, sliding mode guidance, and pseudo-spectral guidance introduce constraints such as terminal angle, minimum energy consumption, and shortest time in order to meet other performance indicators while reaching the target point.

[0003] Traditional guidance algorithms usually assume that the predicted hit point is stationary during optimization, thus abstracting the dynamic guidance process into a static two-point boundary value problem for simplified solution. This results in the decline of guidance accuracy and guidance performance of traditional guidance algorithms based on static modeling when facing environmental changes. Compared with traditional guidance algorithms based on static modeling, deep reinforcement learning algorithms can better adapt to dynamic environments. Zheng Chengchen et al. proposed a biased proportional guidance law based on deep reinforcement learning for the interception collision angle constraint problem, and introduced the remaining time as a state quantity to improve the learning convergence speed. Kang Bingbing et al. used a deep deterministic policy gradient algorithm to construct a reinforcement learning guidance law model for the design of guidance laws for attacking surface targets at specific angles, so that the missile can hit fixed targets at the set angle of impact and hit low-speed moving surface targets with a smaller angle of impact error. Zhang Hao et al. used the Soft Actor-Cr it ic (SAC) algorithm to design an adaptive intelligent interception guidance law for high-maneuverability target anti-missile. Although the above research set up a relatively complex dynamic scene, the trajectory of the target missile is visible throughout the entire process, which is not suitable for the reverse-track interception guidance problem under the target's uncertain maneuver prediction law. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide an intelligent guidance law based on a deep neural network and a method for predicting guidance points thereof.

[0005] In order to achieve the purpose of the present invention, the present invention will be implemented by adopting the following technical solutions.

[0006] An intelligent guidance law based on a deep neural network, the intelligent guidance law comprising a target maneuver branch prediction model, a prediction guidance point calculation model and a deep neural network model, wherein the target maneuver branch prediction model predicts target maneuver trajectory data, the prediction guidance point calculation model calculates the guidance point according to the target maneuver trajectory data, and the deep neural network model calculates the guidance control amount according to the guidance point;

[0007] in:

[0008] The construction of the target maneuver branch prediction model is based on the Seq2Seq model, and a two-layer gated recurrent unit network is used to build an encoder and a decoder respectively, and a planned sampling method is used to randomly select the real input of the previous moment or the predicted input of the previous moment to predict the output of the current moment. After the ballistic coordinates of the previous period are input into the target maneuver branch prediction model, the target maneuver trajectory is predicted in a data-driven manner;

[0009] The predicted guidance point calculation model uses the interception height to traverse the interception window, calculates the guidance time for the target missile and the interceptor missile to reach the height coordinate, compares the two guidance times, selects the interception height whose absolute value of the difference between the two guidance times is less than the interception time tolerance, and calculates the guidance point according to the target maneuver trajectory data;

[0010] The deep neural network model is trained by the DQN algorithm and can learn the changing scenarios and guidance paths of the predicted guidance point maneuvers through training data;

[0011] The deep neural network model is composed of an input layer, a hidden layer and an output layer. Each layer is composed of at least two neurons. Each neuron is connected to all neurons in the previous layer, and the connection weights are adjusted and information is transmitted through weight and bias parameters.

[0012] As a preferred solution of the present invention, the gated cycle unit is composed of an update gate and a reset gate.

[0013] A method for predicting a guidance point using an intelligent guidance law comprises the following steps:

[0014] S1. Use the target maneuver branch prediction model to predict the target maneuver trajectory;

[0015] S2, according to the predicted target maneuver trajectory data of step S1, using the predicted guidance point calculation model to calculate the predicted guidance point;

[0016] S3. Construct the state space, action and reward and punishment functions of the intelligent guidance law based on the Markov process to ensure the guidance performance of the interceptor missile;

[0017] S4, initialize the parameters of the deep neural network model so that the neural network can start selecting actions and controlling the interaction between the agent and the environment;

[0018] S5. Use the DQN algorithm to train the deep neural network model so that the deep neural network model can learn to predict the changing scenarios and guidance paths of the guidance point maneuvers.

[0019] As a preferred embodiment of the present invention, the training steps of the target maneuver branch prediction model are:

[0020] S51, construct 1000 trajectory data by pulling the standard trajectory and adding observation noise;

[0021] S52, divide the 1000 trajectory data into a training set, a validation set, and a test set in a ratio of 5:3:2;

[0022] S53, using the trajectory data of the training set to train the target maneuver branch prediction model, the objective function used is:

[0023]

[0024] S54. Use the trained target maneuver branch prediction model to predict the target maneuver trajectory and obtain prediction data of the target maneuver trajectory.

[0025] As a preferred embodiment of the present invention, the calculation step of predicting the interception point is:

[0026] S61. Determine the initial interception height H f , calculate the time tf required from the initial interception moment to the initial interception height and the corresponding target position (x f ,y f );

[0027] S62. Assuming that the interceptor missile is guided and intercepted according to the proportional guidance law, calculate the time t required for the missile to move from the initial condition (x0, y0) to the target position. go , let: Δt = t f -t go ;

[0028] S63, setting error ε, where:

[0029] When |Δt|≤ε, it means that the interceptor missile and the target will arrive at the point at the same time, and this point is the predicted hit point;

[0030] When |Δt>ε, the interception height is reselected and recalculated and iterated until |Δt| is less than the set error. At this time, the target position at the interception height is the predicted interception point.

[0031] As a preferred solution of the present invention, the state space is the position coordinates (x f ,y f ); the action space selects the guidance bias term γ of the interceptor missile; the reward function should not only encourage the interceptor missile to reach the middle and end handover points within certain constraints, but also punish the interceptor missile for colliding with the boundary.

[0032] As a preferred solution of the present invention, the certain constraint is a proportional guidance law with a variable bias term controlled by a neural network.

[0033] As a preferred solution of the present invention, the deep neural network model is a three-layer fully connected neural network, and the ReLU function is selected as the activation function in the layer, wherein the three-layer fully connected neural network are respectively an input layer, a hidden layer and an output layer.

[0034] As a preferred embodiment of the present invention, the simulation training steps of the deep neural model are:

[0035] S91. Assuming that the initial predicted interception point calculated before the interceptor missile is launched remains unchanged, after the target missile is launched, the interceptor missile conducts guided interception based on the initial predicted interception point;

[0036] S92: When the interceptor missile is about 10 km away from the predicted interception point, the predicted interception point changes randomly within the square with a length of 4 km in the lower right corner due to the maneuvering of the target missile;

[0037] S93. After the guidance task is changed online, the intelligent agent is required to quickly adjust the interception missile posture in the dynamic adjustment section so that it can meet the handover performance requirements as soon as possible, and finally successfully complete the handover to enable terminal guidance and hit the predicted interception point with the largest possible relative terminal velocity.

[0038] As a preferred embodiment of the present invention, during the guidance process, the upper limit of the overload of the intercepting missile is 20g.

[0039] As a preferred solution of the present invention, a data filtering method is used to optimize the experience pool: when the cumulative reward of the current round is greater than the cumulative reward of all previous rounds, the experience of the current time step is added to the experience pool.

[0040] The present invention has the following advantages:

[0041] The target maneuvers are predicted in a data-driven manner, so that a certain type of specific trajectory can be predicted in advance with a small data sample, and the predicted interception point can be calculated based on the trajectory; at the same time, a deep neural network can be used to learn the changing scenarios of the predicted interception point maneuvers and the optimal guidance path through a large amount of training data, ensuring that the interceptor missile can adapt to dynamic and complex interception scenarios, thereby improving the interception performance of the interceptor missile. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a time series diagram of the real trajectory samples of the present invention, wherein: (a) is a diagram of the real trajectory samples of Xt, and (b) is a diagram of the real trajectory samples of Yt;

[0043] Figure 2 The training loss curve diagram of the present invention, wherein (a) is a training loss curve diagram of time series data in the X direction, and (b) is a training loss curve diagram of time series data in the Y direction;

[0044] Figure 3It is a structural diagram of the deep neural network model of the present invention;

[0045] Figure 4 It is a simulation scene graph of the present invention;

[0046] Figure 5 It is the reward curve diagram of the present invention. DETAILED DESCRIPTION

[0047] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] An intelligent guidance law based on a deep neural network, the intelligent guidance law comprising a target maneuver branch prediction model, a prediction guidance point calculation model and a deep neural network model, wherein the target maneuver branch prediction model predicts target maneuver trajectory data, the prediction guidance point calculation model calculates the guidance point according to the target maneuver trajectory data, and the deep neural network model calculates the guidance control amount according to the guidance point;

[0049] in:

[0050] The construction of the target maneuver branch prediction model is based on the Seq2Seq model, and a two-layer gated recurrent unit network is used to build an encoder and a decoder respectively, and a planned sampling method is used to randomly select the real input of the previous moment or the predicted input of the previous moment to predict the output of the current moment. After the ballistic coordinates of the previous period are input into the target maneuver branch prediction model, the target maneuver trajectory is predicted in a data-driven manner;

[0051] The predicted guidance point calculation model uses the interception height to traverse the interception window, calculates the guidance time for the target missile and the interceptor missile to reach the height coordinate, compares the two guidance times, selects the interception height whose absolute value of the difference between the two guidance times is less than the interception time tolerance, and calculates the guidance point according to the target maneuver trajectory data;

[0052] The deep neural network model is trained by the DQN algorithm and can learn the changing scenarios and guidance paths of the predicted guidance point maneuvers through training data;

[0053] The deep neural network model is composed of an input layer, a hidden layer and an output layer. Each layer is composed of at least two neurons. Each neuron is connected to all neurons in the previous layer, and the connection weights are adjusted and information is transmitted through weight and bias parameters.

[0054] As an embodiment of the present invention, the gated cycle unit is composed of an update gate and a reset gate.

[0055] A method for predicting a guidance point using an intelligent guidance law comprises the following steps:

[0056] S1. Use the target maneuver branch prediction model to predict the target maneuver trajectory;

[0057] S2, according to the predicted target maneuver trajectory data of step S1, using the predicted guidance point calculation model to calculate the predicted guidance point;

[0058] S3. Construct the state space, action and reward and punishment functions of the intelligent guidance law based on the Markov process to ensure the guidance performance of the interceptor missile;

[0059] S4, setting deep neural network model parameters;

[0060] S5. Use the DQN algorithm to train the deep neural network model so that the deep neural network model can learn to predict the changing scenarios and guidance paths of the guidance point maneuvers.

[0061] As an embodiment of the present invention, the training steps of the target maneuver branch prediction model are:

[0062] S51, by pulling the standard trajectory and adding observation noise to construct 1000 trajectory data, the trajectory data set is as follows Figure 1 As shown;

[0063] S52, divide the 1000 trajectory data into a training set, a validation set, and a test set in a ratio of 5:3:2;

[0064] S53, using the trajectory data of the training set to train the target maneuver branch prediction model, the objective function used is:

[0065]

[0066] The training loss curve is as follows Figure 2 As shown, it can be seen that: with the increase of training rounds, the fitting accuracy gradually increases.

[0067] S54. Use the trained target maneuver branch prediction model to predict the target maneuver trajectory and obtain prediction data of the target maneuver trajectory.

[0068] As an embodiment of the present invention, the calculation step of the predicted interception point is:

[0069] S61. Determine the initial interception height H f , calculate the time t required from the initial moment of interception to the initial interception height based on the known target ballistic equation fAnd the corresponding target position (x f ,y f );

[0070] S62. Assuming that the interceptor missile is guided and intercepted according to the proportional guidance law, calculate the time t required for the missile to move from the initial condition (x0, y0) to the target position. go , let: Δt = t f -t go ;

[0071] S63, setting error ε, where:

[0072] When |Δt|≤ε, it means that the interceptor missile and the target will arrive at the point at the same time, and this point is the predicted hit point;

[0073] When |Δt|>ε, the interception height is reselected and recalculated and iterated until |Δt| is less than the set error. At this time, the target position at the interception height is the predicted interception point.

[0074] As an embodiment of the present invention, the Markov process includes states, actions and rewards, wherein the state space S contains the attributes and information of the agent in the environment, and the action to be performed is selected based on this. f ,y f ) as the state space of the DQN agent; the guidance bias term γ of the interceptor missile is selected as the main action to effectively alleviate the problems of slow convergence of the end-to-end reinforcement learning algorithm and the results that do not conform to the laws of physics. At the same time, when necessary, the missile is restrained from flying straight to the predicted hit point to achieve better guidance performance.

[0075] In order to ensure that the interceptor missile guidance performance is good enough, it can smoothly complete the handover and successfully intercept the target at the mid-terminal handover time. This means that the design of the reward function should not only encourage the interceptor missile to reach the mid-terminal handover point with certain constraints, but also punish the interceptor missile for colliding with the boundary. When the interceptor missile collides, a penalty of -50 is given and the round is stopped; when the interceptor missile successfully reaches the mid-terminal handover point, a reward is given according to the performance of the terminal guidance and the round is stopped.

[0076] As an embodiment of the present invention, the certain constraint is a proportional guidance law with a variable bias term controlled by a neural network.

[0077] As an embodiment of the present invention, Figure 3 As shown, the deep neural network model is a three-layer fully connected neural network, and the ReLU function is selected as the activation function in the layer, wherein the three-layer fully connected neural network are respectively an input layer, a hidden layer and an output layer, wherein the input layer is a state layer, and the output layer is an action layer; the calculation process is fully simplified and the convergence speed of the gradient descent is accelerated.

[0078] As an embodiment of the present invention, Figure 4 As shown, the simulation training steps of the deep neural model are:

[0079] S91. Assuming that the initial predicted interception point (60km, 30km) calculated before the interceptor missile is launched remains unchanged, after the target missile is launched, the interceptor missile conducts guided interception based on the initial predicted interception point;

[0080] S92, when the interceptor missile is about 10km away from the predicted interception point, the predicted interception point changes randomly within the square with a length of 4km in the lower right corner of (60km, 30km) due to the maneuvering of the target missile;

[0081] S93. After the guidance task is changed online, the intelligent agent is required to quickly adjust the interception missile posture in the dynamic adjustment section so that it can meet the handover performance requirements as soon as possible, and finally successfully complete the handover to enable terminal guidance and hit the predicted interception point with the largest possible relative terminal velocity.

[0082] As an embodiment of the present invention, during the guidance process, the upper limit of the overload of the intercepting missile is 20g.

[0083] As an embodiment of the present invention, in order to accelerate training, a data filtering method is used to optimize the experience pool: when the cumulative reward of the current round is greater than the cumulative reward of all previous rounds, the experience of the current time step is added to the experience pool.

[0084] The data filtering method and non-data filtering method are used for training respectively. The reward curve of the training process is as follows Figure 5 As shown, it can be seen that with almost the same experience in the first 30 rounds, the agent obtains a higher average reward after data filtering, while the agent without data filtering frequently touches a lower reward level during training.

[0085] The technical solution of the present invention is described in detail above in conjunction with the embodiments / drawings, but the present invention is not limited to the above technical solution. For ordinary technicians in this technical field, after knowing the contents recorded in the present invention, they can make several equivalent changes and substitutions without departing from the principle of the present invention. These equivalent changes and substitutions should also be regarded as belonging to the protection scope of the present invention.

Claims

1. An intelligent guidance law based on deep neural network, It is characterized in that The intelligent guidance law includes a target maneuver branch prediction model, a prediction guidance point calculation model and a deep neural network model, wherein the target maneuver branch prediction model predicts the target maneuver trajectory data, the prediction guidance point calculation model calculates the guidance point according to the target maneuver trajectory data, and the deep neural network model calculates the guidance control amount according to the guidance point; in: The construction of the target maneuver branch prediction model is based on the Seq2Seq model, and a two-layer gated recurrent unit network is used to build an encoder and a decoder respectively, and a planned sampling method is used to randomly select the real input of the previous moment or the predicted input of the previous moment to predict the output of the current moment. After the ballistic coordinates of the previous period are input into the target maneuver branch prediction model, the target maneuver trajectory is predicted in a data-driven manner; The predicted guidance point calculation model uses the interception height to traverse the interception window, calculates the guidance time for the target missile and the interceptor missile to reach the height coordinate, compares the two guidance times, selects the interception height whose absolute value of the difference between the two guidance times is less than the interception time tolerance, and calculates the guidance point according to the target maneuver trajectory data; The deep neural network model is trained by the DQN algorithm and can learn the changing scenarios and guidance paths of the predicted guidance point maneuvers through training data; The deep neural network model is composed of an input layer, a hidden layer and an output layer. Each layer is composed of at least two neurons. Each neuron is connected to all neurons in the previous layer, and the connection weights are adjusted and information is transmitted through weight and bias parameters.

2. According to claim 1, an intelligent guidance law based on a deep neural network, It is characterized in that The gated cycle unit is composed of an update gate and a reset gate.

3. A method for predicting guidance points using intelligent guidance laws, It is characterized in that The steps include: S1. Use the target maneuver branch prediction model to predict the target maneuver trajectory; S2, according to the predicted target maneuver trajectory data of step S1, using the predicted guidance point calculation model to calculate the predicted guidance point; S3. Construct the state space, action and reward and punishment functions of the intelligent guidance law based on the Markov process to ensure the guidance performance of the interceptor missile; S4, initialize the parameters of the deep neural network model so that the neural network can start selecting actions and controlling the interaction between the agent and the environment; S5. Use the DQN algorithm to train the deep neural network model so that the deep neural network model can learn to predict the changing scenarios and guidance paths of the guidance point maneuvers.

4. A method for predicting guidance points using intelligent guidance law according to claim 3, It is characterized in that The training steps of the target maneuver branch prediction model are: S51, construct 1000 trajectory data by pulling the standard trajectory and adding observation noise; S52, divide the 1000 trajectory data into a training set, a validation set, and a test set in a ratio of 5:3:2; S53, using the trajectory data of the training set to train the target maneuver branch prediction model, the objective function used is: S54. Use the trained target maneuver branch prediction model to predict the target maneuver trajectory and obtain prediction data of the target maneuver trajectory.

5. A method for predicting guidance points using intelligent guidance law according to claim 3, It is characterized in that The calculation steps of the predicted guidance point are: S61. Determine the initial interception height H f , calculate the time t required from the initial moment of interception to the initial interception height based on the known target ballistic equation f And the corresponding target position (x f ,y f ); S62. Assuming that the interceptor missile is guided and intercepted according to the proportional guidance law, find the missile's initial condition (x 0 ,y 0 ) The time required to move to the target position t go , let: Δt = t f -t go ; S63, setting error ε, where: When |Δt|≤ε, it means that the interceptor missile and the target will arrive at the point at the same time, and this point is the predicted hit point; When |Δt|>ε, the interception height is reselected and recalculated and iterated until |Δt| is less than the set error. At this time, the target position at the interception height is the predicted guidance point.

6. A method for predicting guidance points using intelligent guidance law according to claim 3, It is characterized in that The state space is the position coordinates of the predicted guidance point (x f ,y f ); the action selects the guidance bias term γ of the interceptor missile; the reward function should not only encourage the interceptor missile to reach the middle and end handover points within certain constraints, but also punish the interceptor missile for colliding with the boundary.

7. A method for predicting guidance points using intelligent guidance law according to claim 6, It is characterized in that The certain constraint is a proportional guidance law with a variable bias term controlled by a neural network.

8. A method for predicting guidance points using intelligent guidance law according to claim 3, It is characterized in that The deep neural network model is a three-layer fully connected neural network, and the ReLU function is selected as the activation function in the layer, wherein the three-layer fully connected neural network are respectively an input layer, a hidden layer and an output layer.

9. A method for predicting guidance points using intelligent guidance law according to claim 3, It is characterized in that The simulation training steps of the deep neural model are as follows: S91. Assuming that the initial predicted interception point calculated before the interceptor missile is launched remains unchanged, after the target missile is launched, the interceptor missile conducts guided interception based on the initial predicted interception point; S92: When the interceptor missile is about 10 km away from the predicted interception point, the predicted interception point changes randomly within the square with a length of 4 km in the lower right corner due to the maneuvering of the target missile; S93. After the guidance task is changed online, the intelligent agent is required to quickly adjust the interception missile posture in the dynamic adjustment section so that it can meet the handover performance requirements as soon as possible, and finally successfully complete the handover to enable terminal guidance and hit the predicted interception point at a relative terminal speed.

10. A method for predicting guidance points using intelligent guidance law according to claim 9, It is characterized in that During the guidance process, the upper limit of the interceptor missile's overload is 20g.

Citation Information

Cited By

  • Intercepted maneuvering target reinforcement learning guidance law design method and device based on imitation learning

    CN119828462A