Method and device for generating aircraft tracking strategy

Abstract

The invention provides a method and a device for generating an aircraft tracking strategy, which can be applied to the field of artificial intelligence and the field of unmanned aerial vehicles. The aircraft tracking strategy generation method comprises the following steps: inputting flight state information of an aircraft and target state information of a tracked moving target into kinematic models corresponding to the aircraft and the moving target respectively to obtain predicted flight state information of the aircraft and predicted target state information of the moving target; the flight state information and the target state information are input into a relative situation model, relative situation information between the aircraft and the moving target is obtained, and the relative situation model is established according to flight state parameters of the aircraft and target state parameters of the moving target; the relative situation information represents a relative motion relation between the aircraft and the moving target; and inputting the predicted flight state information, the predicted target state information, the relative situation information and the operation action of the aircraft into an action prediction model to obtain a predicted operation action of the aircraft.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and unmanned aerial vehicles, and more specifically, to a method and device for generating an aircraft tracking strategy. Background Art

[0002] With the continuous development of unmanned technology, the research and development and application of UAV (Unmanned Aerial Vehicle) in military and civilian fields are becoming more and more extensive. UAVs rely on advanced sensors and intelligent algorithms and perform well in tracking targets.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: the aircraft tracking algorithm is usually difficult to explain, and the accuracy of the tracking strategy is low. Summary of the invention

[0004] In view of this, the present disclosure provides a method and device for generating an aircraft tracking strategy.

[0005] One aspect of the present disclosure provides a method for generating an aircraft tracking strategy, comprising: inputting the flight state information of the aircraft and the target state information of the tracked moving target into the kinematic models corresponding to the aircraft and the moving target respectively, to obtain the predicted flight state information of the aircraft and the predicted target state information of the moving target; inputting the flight state information and the target state information into a relative situation model, to obtain the relative situation information between the aircraft and the moving target, wherein the relative situation model is established based on the flight state parameters of the aircraft and the target state parameters of the moving target, and the relative situation information characterizes the relative motion relationship between the aircraft and the moving target; inputting the predicted flight state information, the predicted target state information, the relative situation information and the operating action of the aircraft into an action prediction model, to obtain the predicted operating action of the aircraft.

[0006] Another aspect of the present disclosure provides a device for generating an aircraft tracking strategy, including: a first input module, used to input the flight state information of the aircraft and the target state information of the tracked moving target into the kinematic models corresponding to the aircraft and the moving target respectively, to obtain the predicted flight state information of the aircraft and the predicted target state information of the moving target; a second input module, used to input the flight state information and the target state information into a relative situation model, to obtain the relative situation information between the aircraft and the moving target, the relative situation model is established based on the flight state parameters of the aircraft and the target state parameters of the moving target, and the relative situation information characterizes the relative motion relationship between the aircraft and the moving target; a third input module, used to input the predicted flight state information, the predicted target state information, the relative situation information and the operating action of the aircraft into the action prediction model, to obtain the predicted operating action of the aircraft.

[0007] Another aspect of the present disclosure provides an electronic device, comprising:

[0008] one or more processors;

[0009] a memory for storing one or more programs,

[0010] When one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0011] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0012] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and the instructions are used to implement the above method when executed.

[0013] According to an embodiment of the present disclosure, by inputting the flight state information of the aircraft and the target state information of the moving target being tracked into the kinematic models corresponding to the aircraft and the moving target, the theoretical predicted flight state information of the aircraft and the predicted target state information of the moving target are obtained, and richer tracking state information can be obtained. Then the flight state information and the target state information are input into the relative situation model to obtain the relative situation information between the aircraft and the moving target under actual conditions, which has strong explanatory power for the tracking situation. Therefore, the predicted flight state information, the predicted target state information, the relative situation information and the operating action of the aircraft are input into the action prediction model, so that the predicted operating action of the aircraft is more accurate, so as to improve the accuracy of the tracking strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0015] Figure 1 A system architecture diagram schematically illustrates a method for generating an aircraft tracking strategy that can be applied according to an embodiment of the present disclosure;

[0016] Figure 2 A flow chart schematically illustrates a method for generating an aircraft tracking strategy according to an embodiment of the present disclosure;

[0017] Figure 3a A schematic diagram of an action prediction model according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 3b A schematic diagram of an action prediction sub-model according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 3c A schematic diagram of training an action prediction model according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 4a A schematic diagram of a three-dimensional simulation of a flight action corresponding to maximum accelerated level flight according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 4b A schematic diagram of a three-dimensional simulation of a flight action corresponding to maximum deceleration level flight according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 4c A schematic diagram of a three-dimensional simulation of a flight action corresponding to a climb according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 4d A schematic diagram of a three-dimensional simulation of a flight action corresponding to a dive flight according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 4e A schematic diagram of a three-dimensional simulation of a flight action corresponding to a right turn according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 4f A schematic diagram of a three-dimensional simulation of a flight action corresponding to a left turn according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 4g A schematic diagram of a three-dimensional simulation of a flight action corresponding to a straight and level flight according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 5 A schematic diagram schematically shows flight state parameters under a kinematic model according to an embodiment of the present disclosure;

[0028] Figure 6 A block diagram schematically shows a device for generating an aircraft tracking strategy according to an embodiment of the present disclosure; and

[0029] Figure 7 A block diagram of an electronic device suitable for implementing a method for generating an aircraft tracking strategy according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0032] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0033] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.

[0035] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0036] An embodiment of the present disclosure provides a method for generating an aircraft tracking strategy, including: inputting the flight state information of the aircraft and the target state information of the tracked moving target into the kinematic models corresponding to the aircraft and the moving target respectively, to obtain the predicted flight state information of the aircraft and the predicted target state information of the moving target; inputting the flight state information and the target state information into a relative situation model, to obtain the relative situation information between the aircraft and the moving target, the relative situation model is established based on the flight state parameters of the aircraft and the target state parameters of the moving target, and the relative situation information characterizes the relative motion relationship between the aircraft and the moving target; inputting the predicted flight state information, the predicted target state information, the relative situation information and the operating action of the aircraft into the action prediction model, to obtain the predicted operating action of the aircraft.

[0037] Figure 1 The system architecture diagram of the method for generating an aircraft tracking strategy according to an embodiment of the present disclosure is schematically shown.

[0038] It should be noted that Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0039] like Figure 1 As shown, the system architecture 100 according to this embodiment may include an aircraft 101, a tracked moving target 102, a server 103, and a network 104. The network 104 is used to provide a medium for communication links between the aircraft 101, the moving target 102, and the server 103. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0040] The aircraft 101 may be an unmanned aerial vehicle (UAV). For example, the aircraft 101 may be a rotary wing UAV, a fixed wing UAV, a vertical take-off and landing UAV, etc.

[0041] The aircraft 101 may have a variety of flight sensors, and the flight status information collected by the various flight sensors is transmitted to the server 103 through the network 104. For example, the ultrasonic sensor is used to obtain the distance between the aircraft 101 and the moving target 102 being tracked; the gyroscope is used to obtain the angular velocity of the aircraft 101; the accelerometer is used to measure the acceleration of the aircraft 101; the magnetometer is used to measure the flight direction of the aircraft 101; the radar is used to measure the distance between the aircraft 101 and the surrounding objects to avoid obstacles; the GPS (Global Positioning System) sensor is used to locate the position of the aircraft 101; and the camera is used to capture images of the outside of the aircraft 101.

[0042] The moving target 102 may be a moving object driven on the ground, such as a vehicle. The moving target 102 is configured with a variety of target sensors. The target state information collected by the various target sensors is transmitted to the server 103 via the network 104. For example, the GPS sensor is used to locate the position of the moving target 102; the accelerometer is used to detect the motion state and steering angle of the moving target 102; and the camera sensor is used to capture images around the moving target 102.

[0043] The server 103 is used to input the flight status information of the aircraft 101 and the target status information of the tracked moving target 102 into the kinematic models corresponding to the aircraft and the moving target respectively, so as to obtain the predicted flight status information of the aircraft and the predicted target status information of the moving target; input the flight status information and the target status information into the relative situation model, so as to obtain the relative situation information between the aircraft and the moving target, wherein the relative situation model is established based on the flight status parameters of the aircraft and the target status parameters of the moving target, and the relative situation information represents the relative motion relationship between the aircraft and the moving target; input the predicted flight status information, the predicted target status information, the relative situation information and the operating action of the aircraft into the action prediction model, so as to obtain the predicted operating action of the aircraft.

[0044] It should be noted that the method for generating the aircraft tracking strategy provided in the embodiment of the present disclosure can generally be executed by the server 103. Accordingly, the device for generating the aircraft tracking strategy provided in the embodiment of the present disclosure can generally be set in the server 103.

[0045] Figure 2 The flowchart of the method for generating an aircraft tracking strategy according to an embodiment of the present disclosure is schematically shown.

[0046] like Figure 2 As shown, the method includes operations S210 to S230.

[0047] In operation S210, the flight state information of the aircraft and the target state information of the tracked moving target are input into the kinematic models corresponding to the aircraft and the moving target respectively to obtain the predicted flight state information of the aircraft and the predicted target state information of the moving target.

[0048] According to an embodiment of the present disclosure, the aircraft may be a drone. For example, the aircraft may be a rotary-wing drone, a fixed-wing drone, etc. The aircraft is used to track a moving target.

[0049] According to an embodiment of the present disclosure, the moving target may be a device moving on the ground or a device moving in the air. For example, the moving target may be a vehicle moving on the ground or a helicopter moving in the air.

[0050] According to an embodiment of the present disclosure, the flight status information characterizes the flight movement of the aircraft. For example, the flight status information may be that the aircraft is flying north at a speed of 4 kilometers per second at an altitude of 20 kilometers. For example, the flight status information may also be the position and posture of the aircraft.

[0051] According to an embodiment of the present disclosure, the target state information represents the flight movement or driving situation of the moving target. For example, the target state information may include the position of the moving target, the flight speed or driving speed, the position of surrounding obstacles, etc.

[0052] According to the embodiments of the present disclosure, the kinematic model can be inferred based on the position, speed and other state information of the aircraft or the moving target. The flight state information or target state information at the current moment is input into the kinematic model corresponding to the aircraft and the moving target, respectively, to obtain the predicted flight state information and predicted target state information at the next moment.

[0053] In operation S220, the flight state information and the target state information are input into a relative situation model to obtain relative situation information between the aircraft and the moving target.

[0054] According to an embodiment of the present disclosure, the relative situation model is established based on the flight state parameters of the aircraft and the target state parameters of the moving target.

[0055] According to an embodiment of the present disclosure, the relative situation information represents the relative motion relationship between the aircraft and the moving target.

[0056] According to an embodiment of the present disclosure, the flight state parameter may be one or more kinematic parameters that can characterize the motion state of the aircraft. For example, the flight state parameter may be the position, posture, etc. of the aircraft.

[0057] According to an embodiment of the present disclosure, the target state parameter may be one or more kinematic parameters that can characterize the motion state of the moving target. For example, the target state parameter may be the position, posture, etc. of the moving target.

[0058] In operation S230, the predicted flight state information, the predicted target state information, the relative situation information, and the operation action of the aircraft are input into the action prediction model to obtain the predicted operation action of the aircraft.

[0059] According to an embodiment of the present disclosure, the operating action of the aircraft may be level flight, climbing, diving, left turn, etc.

[0060] According to an embodiment of the present disclosure, the action prediction model can be a machine learning algorithm. For example, the action prediction model can be one of the models such as a neural network model, a support vector machine model, a deep learning model, a random forest model, or a combination of multiple models. For example, the moving target and the aircraft are two devices, and the action prediction model can choose to apply a combination model of multiple agents. The agent can be any machine learning algorithm. The predicted target state information and relative situation information of the moving target are input into the first agent to obtain the anti-tracking operation of the moving target at the next moment; the anti-tracking operation of the moving target at the next moment, the predicted flight state information, the relative situation information and the operating action of the aircraft are input into the second agent to obtain the predicted operating action of the aircraft.

[0061] According to an embodiment of the present disclosure, by inputting the flight state information of the aircraft and the target state information of the moving target being tracked into the kinematic models corresponding to the aircraft and the moving target, the theoretical predicted flight state information of the aircraft and the predicted target state information of the moving target are obtained, and richer tracking state information can be obtained. Then the flight state information and the target state information are input into the relative situation model to obtain the relative situation information between the aircraft and the moving target under actual conditions, which has strong explanatory power for the tracking situation. Therefore, the predicted flight state information, the predicted target state information, the relative situation information and the operating action of the aircraft are input into the action prediction model, so that the predicted operating action of the aircraft is more accurate, so as to improve the accuracy of the tracking strategy.

[0062] According to an embodiment of the present disclosure, an action prediction model includes an attack prediction sub-model corresponding to a moving target and an action prediction sub-model corresponding to an aircraft; predicted flight state information, predicted target state information, relative situation information and operating actions of the aircraft are input into the action prediction model to obtain predicted operating actions of the aircraft, including: inputting predicted target state information and relative situation information into the attack prediction sub-model to obtain a predicted attack area of ​​the moving target; inputting the predicted attack area, predicted flight state information, relative situation information and operating actions of the aircraft into the action prediction sub-model to obtain predicted operating actions of the aircraft.

[0063] According to an embodiment of the present disclosure, a moving target may launch an attack object toward an aircraft to prevent the aircraft from continuing to track. The predicted attack area represents the predicted position where the moving target launches an attack object toward the aircraft.

[0064] According to an embodiment of the present disclosure, the action prediction model includes an attack prediction sub-model and an action prediction sub-model, which can predict the predicted attack area of ​​the moving target, and then input the predicted attack area into the action prediction sub-model. The obtained predicted operation action of the aircraft can prevent the moving target from attacking the aircraft, thereby ensuring the safety of the aircraft.

[0065] Figure 3a A schematic diagram of an action prediction model according to an embodiment of the present disclosure is schematically shown.

[0066] like Figure 3a As shown, the action prediction model 300 includes an attack prediction sub-model 310 and an action prediction sub-model 320 .

[0067] The predicted target state information and relative situation information are input into the attack prediction sub-model 310 to obtain the predicted attack area. The attack prediction sub-model 310 can be trained using historical target state information and historical relative situation information. The attack prediction sub-model can be any one of the machine learning algorithms.

[0068] The predicted attack area, operation action, predicted target state information and relative situation information are input into the action prediction sub-model 320 to obtain the predicted operation action of the aircraft. The action prediction sub-model 320 can be the same machine learning algorithm as the attack prediction sub-model 310.

[0069] According to an embodiment of the present disclosure, the action prediction sub-model includes a policy network, a Q-value network and a target Q-value network. The target Q-value network is obtained based on the network parameters of the Q-value network of a preset training time. The action prediction sub-model is trained based on the following method: obtaining historical attack areas, historical flight status information, historical relative situation information and historical operation actions from the experience replay pool; inputting the historical attack areas, historical flight status information, historical relative situation information and historical operation actions into the policy network to obtain the operation actions of the aircraft; inputting the operation actions into the Q-value network to obtain the reward value of the operation actions, and the reward value is calculated based on the reward function; and updating the network parameters of the policy network according to the reward value.

[0070] According to the embodiments of the present disclosure, the role of the experience replay pool is to store historical attack areas, historical flight status information, historical relative situation information and historical operation actions, and randomly extract a batch of experiences from the experience replay pool for learning during training, thereby avoiding the experience data used in training from being too relevant, improving the stability of learning, and enhancing the diversity of samples.

[0071] Figure 3b A schematic diagram of an action prediction sub-model according to an embodiment of the present disclosure is schematically shown.

[0072] like Figure 3b As shown, the action prediction sub-model 320 includes a policy network 321 , a Q-value network 322 , and a target Q-value network 323 .

[0073] According to an embodiment of the present disclosure, the policy network 321 may also be referred to as an actor, and is used to determine an operation action from among a plurality of candidate operation actions of the aircraft. The policy network 321 may output a probability distribution or a deterministic operation action to guide the behavior of the aircraft.

[0074] According to an embodiment of the present disclosure, the Q-value network 322 may also be referred to as a critic, which is used to evaluate whether the operation action selected by the policy network 321 (actor) is effective. The quality of the action is evaluated by calculating the reward value of the operation action and feeding back to the evaluation policy network 321 (actor).

[0075] According to an embodiment of the present disclosure, the target Q value network 323 is a delayed copy of the Q value network 322, and is updated once in a while, and the updated network parameters of the target Q value network 323. Delayed updates can help smooth changes in target values, reduce fluctuations during training, and thus improve stability.

[0076] During the model training process, the action prediction sub-model 320 interacts with the environment at each time step, that is, obtains the previous operation action (a t ) is executed (s t ), the reward value obtained (r t ), and transfer to the next state (s t+1 Status (s) t ) can include the attack area, flight status information and relative situation information of the moving target.

[0077] According to an embodiment of the present disclosure, the decision strategy adopted by the policy network is Gaussian distribution, the policy network includes a linear layer, and the output of the linear layer is the mean and variance.

[0078] According to an embodiment of the present disclosure, the process of finding a target decision strategy in a Markov decision process and finding a target decision strategy in an action prediction sub-model is defined as:

[0079] (1)

[0080] π* represents the target decision strategy, π represents the decision strategy at time t, and st is the flight status information at time t, is the operation action selected according to the decision strategy at time t, According to the state s t and The reward value generated is is the discount factor, α is the temperature parameter, and the temperature parameter is used to control the importance of the entropy regularization term. The larger α is, the stronger the randomness of the strategy is. α can be a fixed value or automatically adjusted. H is the entropy regularization term.

[0081] Figure 3c A schematic diagram of training an action prediction model according to an embodiment of the present disclosure is schematically shown.

[0082] like Figure 3c As shown, the historical flight state information and historical target state information corresponding to the operation action are input into the kinematic models 340 corresponding to the aircraft and the moving target, respectively, to obtain the predicted historical flight state information of the aircraft and the predicted historical target state information of the moving target. The historical flight state information and the historical target state information are input into the relative situation model 350 to obtain the historical relative situation information between the aircraft and the moving target. The state includes the predicted historical target state information and the historical relative situation information.

[0083] The predicted historical target state information and historical relative situation information are input into the attack prediction sub-model 310 to obtain the attack area of ​​the moving target (i.e., the attack area corresponding to the operation action). The attack area, the predicted historical flight state information, the historical relative situation information, and the operation action are input into the strategy network 321 to obtain the predicted operation action at the next moment.

[0084] The above-mentioned operation actions, historical flight status information, historical target status information, historical relative situation information, attack area, and predicted operation actions are stored in the experience replay pool 330. The experience replay pool 330 is used to cache sample experience from previous training. Its function is to reduce the correlation between experiences. During training, a batch of values ​​are randomly selected from the experience replay pool 330, which improves the training effectiveness and popularity of the network.

[0085] When the prediction action sub-model is being trained, the policy network 321 randomly extracts a batch of experiences from the experience replay pool 330 for learning. The policy network 321 acts as a probability density function estimator and can predict the action distribution based on the current state. The policy network 321 includes two fully connected hidden layers, each with 512 units, followed by a ReLU activation function. The policy is a Gaussian distribution. The last layer of the policy network 321 includes two parallel linear layers, each of which outputs the mean and variance of the action.

[0086] At the same time, the Q value network 322 is used to calculate the reward value of the predicted operation action at the next moment. For example, the reward value can be a Q value, and the Q value can represent the expected cumulative reward value of taking a certain action in a given state. The network parameters of the policy network are updated according to the expected cumulative reward value.

[0087] The above operations are repeated until the expected cumulative reward value converges or reaches a preset threshold, and the trained action prediction sub-model is obtained.

[0088] The target Q value network 323 is a delayed copy of the Q value network 322 and is updated once in a while. The target Q value network 323 is associated with the Q value network 322 and outputs the optimal value under continuous interaction. The delayed update can help smooth the change of the target value and reduce the fluctuation during the training process, thereby improving stability.

[0089] Through the above method, the size explosion problem caused by the continuous state space in air combat is solved. And the simulation results show that under the premise of ensuring the convergence of the algorithm, this method effectively enables the aircraft to select appropriate maneuvers according to different state information to facilitate attack or escape. This lays the foundation for simulating the performance of real combat aircraft through vertical take-off and landing aircraft.

[0090] According to an embodiment of the present disclosure, the aircraft is a vertical take-off and landing aircraft, and the above method also includes: determining a target operation instruction from a plurality of operation instructions included in a flight action library according to the predicted operation action; and performing the predicted operation action on the aircraft according to the target operation instruction.

[0091] In the field of vertical take-off and landing UAVs, due to their special configuration, some vertical take-off and landing aircraft can perform fast cruising, hovering and other maneuvers in the air. If necessary, they can avoid enemy pursuit by hovering. The take-off and landing conditions of vertical take-off and landing aircraft are low, and the survival rate is high. However, there are still some problems. During the flight, the complex aircraft structure makes manual operation difficult, and correspondingly, pilots are also difficult to train. How to achieve autonomous flight and tracking is the pain point and urgent problem in the field of vertical take-off and landing aircraft operations.

[0092] Aircraft include but are not limited to vertical take-off and landing aircraft, and moving targets include but are not limited to tracked enemy targets.

[0093] In the field of flight action library research, there is a very important issue in the design of the action library, namely, the exit and conversion of flight actions. That is, when to exit the current action and make the next flight decision. In the decision-making program design, the step length is usually less than 1 second, and most complex actions require continuous execution of several steps to complete. In actual tracking, the air environment changes rapidly. When the aircraft chooses to perform an action according to the current situation, the position and flight parameters of the tracked target will change. In addition, due to the existence of various sensors, the tracked moving target will take corresponding countermeasures after obtaining the flight action information, which will cause a significant change in the relative situation. The continued execution of the original flight action may lose its advantage or even become passive. In this case, the action must be exited to make the next flight decision. In this case, the aircraft's action may not be completed, which requires the design of exits for each action. It is relatively complicated to design the exit in detail, and it is necessary to design dynamic exits to be practical. The general principle is: "Set an intermediate exit at the place where the action is halfway completed and two-thirds completed." Such a setting is relatively rough. From the perspective of being closer to reality, it is reasonable to make flight decisions in each step, but this will inevitably lead to a sharp increase in the amount of calculation, and it may also result in a large number of actions having to be exited as soon as they are executed.

[0094] The expected total reward and policy entropy are maximized through the action prediction sub-model (Flexible Actor-Critic Algorithm (SAC)) in the deep reinforcement learning algorithm. Maximizing policy entropy can promote exploration, prevent premature convergence to local optimality, enhance algorithm robustness, and speed up training, which is very suitable for the complex aerial environment encountered by vertical take-off and landing aircraft. In addition, the flexible actor-critic algorithm is characterized by having few and easy-to-adjust hyperparameters. The above technical solution effectively realizes the autonomous decision-making process of vertical take-off and landing aircraft. Through intelligent decision-making, the unmanned advantage can be fully utilized, and the harsh situation that pilots cannot bear under large overload conditions can be avoided.

[0095] Figure 4a A schematic diagram of a three-dimensional simulation of a flight action corresponding to maximum accelerated level flight according to an embodiment of the present disclosure is schematically shown.

[0096] Figure 4b A three-dimensional simulation diagram of a flight action corresponding to maximum deceleration and level flight according to an embodiment of the present disclosure is schematically shown.

[0097] Figure 4c A schematic diagram of a three-dimensional simulation of a flight action corresponding to a climb according to an embodiment of the present disclosure is schematically shown.

[0098] Figure 4d A schematic diagram of a three-dimensional simulation of a flight action corresponding to a dive flight according to an embodiment of the present disclosure is schematically shown.

[0099] Figure 4e A three-dimensional simulation diagram of a flying action corresponding to a right turn according to an embodiment of the present disclosure is schematically shown.

[0100] Figure 4f A three-dimensional simulation diagram of a flight action corresponding to a left turn according to an embodiment of the present disclosure is schematically shown.

[0101] Figure 4g A schematic diagram of a three-dimensional simulation of a flight action corresponding to straight and level flight according to an embodiment of the present disclosure is schematically shown.

[0102] like Figure 4a-4g The corresponding flight actions are: maximum acceleration level flight, maximum deceleration level flight, climb, dive, right turn, left turn and fixed straight level flight. The three-dimensional simulation schematic diagram can show the position change and speed change of the aircraft. The position of the aircraft can be expressed by (X, Y, Z) three-dimensional coordinates, and the speed V (m\s) can be expressed according to the depth of the color. The use of a basic control action library avoids the contradictions faced by a complex tactical action library. The action library in the technical solution of this application can be used to perform "one step and one judgment" to make flight decision-making and select actions at each step of the program.

[0103] According to an embodiment of the present disclosure, the formula of the kinematic model corresponding to the aircraft is as follows:

[0104] (2)

[0105] x r ,y r , z r represents the flight coordinate value of the aircraft, v represents the aircraft speed, γ is the aircraft's roll angle, θ r represents the flight pitch angle of the aircraft, ψ r Indicates the flight yaw angle of the aircraft, n x and n z They represent the tangential overload and normal overload of the aircraft respectively.

[0106] According to an embodiment of the present disclosure, the relative situation information includes a deviation angle, a breakaway angle, a center of mass distance between the aircraft and the moving target, a height difference between the aircraft and the moving target, and a speed and acceleration of the aircraft.

[0107] In tracking and reconnaissance, the aircraft can chase the target's tail or gain altitude advantage by executing maneuvers to gain the advantage of air tracking and game, while avoiding entering the attack zone of the moving target. The AspectAngle (AA) and Antenna Train Angle (ATA) are introduced to define the current chasing situation of both parties.

[0108] According to an embodiment of the present disclosure, the flight state parameters include the flight coordinate value, the flight pitch angle and the flight yaw angle, the target state parameters include the target coordinate value, the target pitch angle and the target yaw angle of the moving target, and the formula of the relative situation model is as follows:

[0109] (3)

[0110] q r is the deviation angle, q b is the breakaway angle, d is the distance between the center of mass of the aircraft and the moving target, is the height difference between the aircraft and the moving target; v is the speed of the aircraft, is the acceleration of the aircraft, x b ,y b , z b represents the target coordinate value, θ b represents the target pitch angle, ψ b Indicates the target yaw angle.

[0111] According to an embodiment of the present disclosure, the deviation angle may be the angle between the longitudinal axis of the aircraft body and the distance vector of the center of mass.

[0112] According to an embodiment of the present disclosure, the breakaway angle may be the angle between the longitudinal axis of the moving target and the distance vector of the center of mass.

[0113] According to an embodiment of the present disclosure, the deviation angle and the breakaway angle may be obtained by performing image processing on an image captured by a camera.

[0114] For example, a multidimensional state vector s is selected as relative situation information, and the multidimensional state vector s may include: deviation angle, breakaway angle, center of mass distance between the aircraft and the moving target, height difference between the aircraft and the moving target, aircraft speed, and aircraft acceleration.

[0115] Figure 5 A schematic diagram of flight state parameters under a kinematic model according to an embodiment of the present disclosure is schematically shown.

[0116] like Figure 5 As shown, v is the aircraft speed, γ is the aircraft roll angle, θ is the aircraft flight pitch angle, ψ is the aircraft flight yaw angle, n x and n z Represent the tangential overload and normal overload of the aircraft respectively. x, y, z represent the flight coordinates of the aircraft. The state variables of the aircraft are expressed as [x, y, z, v, ψ, θ], which is used to represent the state of the aircraft. The control variables are expressed as [n x , n z , γ], this group of variables is used to represent the control quantity of the aircraft.

[0117] Figure 6 A block diagram of a device for generating an aircraft tracking strategy according to an embodiment of the present disclosure is schematically shown.

[0118] like Figure 6 As shown, the aircraft tracking strategy generation device 600 includes a first input module 610 , a second input module 620 and a third input module 630 .

[0119] The first input module 610 is used to input the flight state information of the aircraft and the target state information of the tracked moving target into the kinematic models corresponding to the aircraft and the moving target, respectively, to obtain the predicted flight state information of the aircraft and the predicted target state information of the moving target. In one embodiment, the first input module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0120] The second input module 620 is used to input the flight state information and the target state information into the relative situation model to obtain the relative situation information between the aircraft and the moving target. The relative situation model is established based on the flight state parameters of the aircraft and the target state parameters of the moving target. The relative situation information represents the relative motion relationship between the aircraft and the moving target. In one embodiment, the second input module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0121] The third input module 630 is used to input the predicted flight state information, the predicted target state information, the relative situation information and the operation action of the aircraft into the action prediction model to obtain the predicted operation action of the aircraft. In one embodiment, the third input module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0122] According to an embodiment of the present disclosure, the action prediction model includes an attack prediction sub-model corresponding to a moving target and an action prediction sub-model corresponding to an aircraft; the third input module 630 includes a first input sub-module and a second input sub-module. The first input sub-module is used to input the predicted target state information and relative situation information into the attack prediction sub-model to obtain the predicted attack area of ​​the moving target; the second input sub-module is used to input the predicted attack area, predicted flight state information, relative situation information and the operation action of the aircraft into the action prediction sub-model to obtain the predicted operation action of the aircraft.

[0123] According to an embodiment of the present disclosure, the action prediction sub-model includes a policy network, a Q-value network and a target Q-value network. The target Q-value network is obtained based on the network parameters of the Q-value network of a preset training time. The action prediction sub-model is trained based on the following method: obtaining historical attack areas, historical flight status information, historical relative situation information and historical operation actions from the experience replay pool; inputting the historical attack areas, historical flight status information, historical relative situation information and historical operation actions into the policy network to obtain the operation actions of the aircraft; inputting the operation actions into the Q-value network to obtain the reward value of the operation actions, and the reward value is calculated based on the reward function; and updating the network parameters of the policy network according to the reward value.

[0124] According to an embodiment of the present disclosure, the decision strategy adopted by the policy network is Gaussian distribution, the policy network includes a linear layer, and the output of the linear layer is the mean and variance.

[0125] According to an embodiment of the present disclosure, the process definition of finding the target decision strategy in the action prediction sub-model is as shown in formula (1).

[0126] According to an embodiment of the present disclosure, the aircraft is a vertical take-off and landing aircraft, and the above-mentioned device also includes a determination module and an execution module. The determination module is used to determine a target operation instruction from multiple operation instructions included in the flight action library according to the predicted operation action; the execution module is used to execute the predicted operation action on the aircraft according to the target operation instruction.

[0127] According to an embodiment of the present disclosure, the formula of the kinematic model corresponding to the aircraft is shown in formula (2).

[0128] According to an embodiment of the present disclosure, the relative situation information includes a deviation angle, a breakaway angle, a center of mass distance between the aircraft and the moving target, a height difference between the aircraft and the moving target, and a speed and acceleration of the aircraft.

[0129] According to an embodiment of the present disclosure, the flight state parameters include flight coordinate values, flight pitch angles and flight yaw angles, and the target state parameters include target coordinate values, target pitch angles and target yaw angles of the moving target. The formula of the relative situation model is shown in Formula 3.

[0130] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0131] For example, any multiple of the first input module 610, the second input module 620, and the third input module 630 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first input module 610, the second input module 620, and the third input module 630 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in a suitable combination of any of them. Alternatively, at least one of the first input module 610 , the second input module 620 , and the third input module 630 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0132] It should be noted that the generation device part of the aircraft tracking strategy in the embodiment of the present disclosure corresponds to the generation method part of the aircraft tracking strategy in the embodiment of the present disclosure. The description of the generation device part of the aircraft tracking strategy specifically refers to the generation method part of the aircraft tracking strategy, which will not be repeated here.

[0133] Figure 7A block diagram of an electronic device suitable for implementing a method for generating an aircraft tracking strategy according to an embodiment of the present disclosure is schematically shown.

[0134] Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0135] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0136] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 705. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0137] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 705. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0138] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0139] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0140] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0141] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0142] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for generating an aircraft tracking strategy provided by the embodiment of the present disclosure.

[0143] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0144] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0145] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0146] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0147] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for generating an aircraft tracking strategy, characterized in that: The method comprises: Inputting the flight state information of the aircraft and the target state information of the tracked moving target into the kinematic models corresponding to the aircraft and the moving target, respectively, to obtain the predicted flight state information of the aircraft and the predicted target state information of the moving target; Inputting the flight state information and the target state information into a relative situation model to obtain relative situation information between the aircraft and the moving target, wherein the relative situation model is established based on the flight state parameters of the aircraft and the target state parameters of the moving target, and the relative situation information represents the relative motion relationship between the aircraft and the moving target; The predicted flight state information, the predicted target state information, the relative situation information and the operating action of the aircraft are input into an action prediction model to obtain the predicted operating action of the aircraft.

2. The method according to claim 1, characterized in that The action prediction model includes an attack prediction sub-model corresponding to the moving target and an action prediction sub-model corresponding to the aircraft; The step of inputting the predicted flight state information, the predicted target state information, the relative situation information, and the operation action of the aircraft into an action prediction model to obtain the predicted operation action of the aircraft comprises: Inputting the predicted target state information and the relative situation information into the attack prediction sub-model to obtain the predicted attack area of ​​the moving target; The predicted attack area, the predicted flight state information, the relative situation information and the operating action of the aircraft are input into an action prediction sub-model to obtain the predicted operating action of the aircraft.

3. The method according to claim 2, characterized in that The action prediction sub-model includes a policy network, a Q value network and a target Q value network, wherein the target Q value network is obtained based on the network parameters of the Q value network at a preset training time, and the action prediction sub-model is obtained by training in the following manner: Obtain historical attack areas, historical flight status information, historical relative situation information, and historical operation actions from the experience replay pool; Inputting the historical attack area, the historical flight state information, the historical relative situation information and the historical operation action into the strategy network to obtain the operation action of the aircraft; Inputting the operation action into the Q value network to obtain a reward value for the operation action, wherein the reward value is calculated based on a reward function; The network parameters of the policy network are updated according to the reward value.

4. The method according to claim 3, characterized in that The decision strategy adopted by the strategy network is Gaussian distribution. The strategy network includes a linear layer, and the output of the linear layer is a mean and a variance.

5. The method according to claim 4, characterized in that The process of finding the target decision strategy in the action prediction sub-model is defined as: π* represents the target decision strategy, π represents the decision strategy at time t, and s t is the flight status information at the time t, is the operation action selected according to the decision strategy at time t, and r is the action selected according to the state s t and The reward value generated is is the discount factor, α is the temperature parameter, and the temperature parameter is used to control the importance of the entropy regularization term. H is the entropy regularization term.

6. The method according to claim 1, characterized in that The aircraft is a vertical take-off and landing aircraft, and the method further comprises: According to the predicted operation action, determining a target operation instruction from a plurality of operation instructions included in a flight action library; According to the target operation instruction, a predicted operation action is performed on the aircraft.

7. The method according to claim 1, characterized in that The formula of the kinematic model corresponding to the aircraft is as follows: x r ,y r , z r represents the flight coordinate value of the aircraft, v represents the speed of the aircraft, γ is the roll angle of the aircraft, θ r represents the flight pitch angle of the aircraft, ψ r represents the flight yaw angle of the aircraft, n x and n z represent the tangential overload and normal overload of the aircraft respectively.

8. The method according to claim 1, characterized in that The relative situation information includes a deviation angle, a breakaway angle, a distance between the center of mass of the aircraft and the moving target, a height difference between the aircraft and the moving target, and a speed and acceleration of the aircraft.

9. The method according to claim 1, characterized in that: The flight state parameters include the flight coordinate value, the flight pitch angle and the flight yaw angle, the target state parameters include the target coordinate value, the target pitch angle and the target yaw angle of the moving target, and the formula of the relative situation model is as follows: q r is the deviation angle, q b is the breakaway angle, d is the distance between the center of mass of the aircraft and the moving target, is the height difference between the aircraft and the moving target; v is the speed of the aircraft, is the acceleration of the aircraft, x b ,y b , z b represents the target coordinate value, θ b represents the target pitch angle, ψ b represents the target yaw angle.

10. A device for generating an aircraft tracking strategy, characterized in that: The device comprises: A first input module is used to input the flight state information of the aircraft and the target state information of the tracked moving target into the kinematic models corresponding to the aircraft and the moving target respectively, so as to obtain the predicted flight state information of the aircraft and the predicted target state information of the moving target; A second input module is used to input the flight state information and the target state information into a relative situation model to obtain relative situation information between the aircraft and the moving target, wherein the relative situation model is established based on the flight state parameters of the aircraft and the target state parameters of the moving target, and the relative situation information represents the relative motion relationship between the aircraft and the moving target; The third input module is used to input the predicted flight state information, the predicted target state information, the relative situation information and the operating action of the aircraft into the action prediction model to obtain the predicted operating action of the aircraft.

Images