Construction method, control method and system of flight control network

Abstract

The invention provides a construction method, a control method and a system of a flight control network, and the method comprises the steps: employing a depth deterministic strategy gradient algorithm, building an attitude control network of an unmanned aerial vehicle in a flight process based on the state information and a target flight value of the unmanned aerial vehicle in the flight process, employing a depth reinforcement learning algorithm, and carrying out the attitude control of the unmanned aerial vehicle in the flight process. Based on the state information and the information of the environment where the unmanned aerial vehicle is located, a path planning network and an intelligent avoidance network of the unmanned aerial vehicle in the flight process are built respectively; based on the attitude control network, the path planning network and the intelligent avoidance network, constructing a hierarchical deep reinforcement learning network of the unmanned aerial vehicle in the flight process; and training the hierarchical deep reinforcement learning network by adopting historical flight data of the unmanned aerial vehicle in different flight environments and a preset flight reward mechanism to obtain a flight control network of the unmanned aerial vehicle. According to the invention, the adaptability and flexibility of the unmanned aerial vehicle in the flight process can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hierarchical deep learning algorithms, and in particular to a flight control network construction method, a control method and a system. Background Art

[0002] Flight control systems are crucial to the safe and efficient operation of aircraft. With the widespread application of drones in civil and military fields, the requirements for adaptability and flexibility of flight control systems have increased. In related technologies, traditional flight control systems are not adaptable enough when facing unknown environments or emergencies. The development of artificial intelligence technology has promoted the application of deep learning algorithms in complex tasks, but it still faces challenges in real-time and environmental uncertainty in the field of flight control. Summary of the invention

[0003] In order to solve the problems of the prior art, the present invention proposes a flight control network construction method, a control method and a system, aiming to improve the adaptability and flexibility of the UAV during flight.

[0004] The purpose of the present invention is achieved by adopting the following technical solutions:

[0005] In one aspect, the present invention provides a method for constructing a flight control network of an unmanned aerial vehicle, the method comprising:

[0006] Using a deep deterministic policy gradient algorithm, based on the state information and target flight value of the UAV during flight, an attitude control network of the UAV during flight is constructed;

[0007] Using a deep reinforcement learning algorithm, based on the state information and the environmental information of the UAV, a path planning network and an intelligent avoidance network are respectively built for the UAV during flight;

[0008] Based on the attitude control network, the path planning network and the intelligent avoidance network, a hierarchical deep reinforcement learning network of the UAV during the flight process is constructed;

[0009] The historical flight data of the UAV in different flight environments and a preset flight reward mechanism are used to train the hierarchical deep reinforcement learning network to obtain the flight control network of the UAV.

[0010] Optionally, before building a posture control network of the UAV during flight based on the state information and target flight value of the UAV during flight using a deep deterministic policy gradient algorithm, the method further includes:

[0011] Establishing a reference model corresponding to the UAV during flight;

[0012] The deep deterministic policy gradient algorithm is used to build a posture control network of the UAV during flight based on the state information and target flight value of the UAV during flight, including:

[0013] The state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV are input into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training to obtain the attitude control network.

[0014] Optionally, the deep reinforcement learning algorithm includes: a double-delayed deep deterministic policy gradient algorithm, a soft actor-critic algorithm, and a unit state sequence-based reinforcement learning algorithm. The deep reinforcement learning algorithm is used to build a path planning network and an intelligent avoidance network for the drone during flight based on the state information and the environmental information of the drone, including:

[0015] The state information and the distance information in the environment information are input into the recursive neural network constructed by the double-delayed deep deterministic policy gradient algorithm and the soft actor-critic algorithm for network training to obtain the path planning network; wherein the distance information at least includes: the distance between the drone and the obstacle identified by the drone during flight;

[0016] The state information and obstacle information in the environment information are input into the neural network constructed by the reinforcement learning algorithm based on the unit state sequence for network training to obtain the intelligent avoidance network.

[0017] Optionally, constructing a hierarchical deep reinforcement learning network of the UAV during the flight process based on the attitude control network, the path planning network and the intelligent avoidance network includes:

[0018] Based on the state information, the executable action information of the drone during flight and the corresponding reward information, an infrastructure of an initial level deep reinforcement learning network is constructed;

[0019] The posture control network, the path planning network and the intelligent avoidance network are sequentially used as the low-level deep reinforcement learning subnetwork, the middle-level deep reinforcement learning subnetwork and the high-level deep reinforcement learning subnetwork in the basic architecture to obtain the hierarchical deep reinforcement learning network; wherein the high-level deep reinforcement learning subnetwork can control the decision of the middle-level deep reinforcement learning subnetwork, and the middle-level deep reinforcement learning subnetwork can control the decision of the low-level deep reinforcement learning subnetwork.

[0020] Optionally, before the historical flight data of the UAV under different flight environments and a preset flight reward mechanism are used to train the hierarchical deep reinforcement learning network to obtain the flight control network of the UAV, the method further includes:

[0021] Based on the flight parameters of the UAV during the flight, constructing a reward signal reflecting the flight quality of the UAV; wherein the flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters;

[0022] Constructing reward strategies corresponding to different flight control stages of the UAV;

[0023] Based on the reward signal and the reward strategy, the preset flight reward mechanism is constructed.

[0024] Optionally, the adopting of historical flight data of the UAV under different flight environments and a preset flight reward mechanism to train the hierarchical deep reinforcement learning network to obtain a flight control network of the UAV includes:

[0025] Annotating and organizing the historical flight data to form a training data set;

[0026] The preset flight reward mechanism is used to build a reward function of the hierarchical deep reinforcement learning network to obtain a network to be adjusted, and the training data set is used to train the network to be adjusted to obtain the flight control network.

[0027] Optionally, after the hierarchical deep reinforcement learning network is trained by using the historical flight data of the drone in different flight environments and a preset flight reward mechanism to obtain the flight control network of the drone, the method further includes:

[0028] Determine the preset network optimization strategy;

[0029] The preset network optimization strategy is adopted to optimize the network structure of the flight control network to obtain an optimized flight control network.

[0030] Correspondingly, the present invention also provides a system for constructing a flight control network of an unmanned aerial vehicle, the system comprising:

[0031] A first network building module is used to build a posture control network of the UAV during flight based on the state information and target flight value of the UAV during flight by using a deep deterministic policy gradient algorithm;

[0032] A second network building module is used to use a deep reinforcement learning algorithm to build a path planning network and an intelligent avoidance network for the UAV during flight based on the state information and the environment information of the UAV;

[0033] A network construction module, used to construct a hierarchical deep reinforcement learning network of the UAV during the flight process based on the attitude control network, the path planning network and the intelligent avoidance network;

[0034] The training module is used to train the hierarchical deep reinforcement learning network using the historical flight data of the UAV in different flight environments and a preset flight reward mechanism to obtain a flight control network of the UAV.

[0035] In another aspect, the present invention provides a flight control method for an unmanned aerial vehicle, the method comprising:

[0036] Obtaining the current flight target and current flight environment of the UAV;

[0037] Inputting the current flight target and the current flight environment into a flight control network constructed by any of the above methods to obtain a flight control instruction for the UAV;

[0038] The flight control instruction is used to control the UAV to fly to the current flight target.

[0039] Correspondingly, the present invention also provides a flight control system for an unmanned aerial vehicle, the system comprising:

[0040] An acquisition module, used to acquire the current flight target and current flight environment of the UAV;

[0041] An input module, used to input the current flight target and the current flight environment into the flight control network constructed by the above-mentioned system to obtain the flight control instructions of the UAV;

[0042] The control module is used to control the UAV to fly to the current flight target using the flight control instruction.

[0043] In another aspect, the present invention further provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0044] The memory is used to store one or more programs;

[0045] When the one or more programs are executed by the at least one processor, the method for constructing a flight control network of a drone as described in any one of the above items and the flight control method of the drone as described in the above items are implemented.

[0046] On the other hand, the present invention also provides a readable storage medium having an execution program stored thereon, which, when executed, implements the method for constructing a flight control network for an unmanned aerial vehicle as described in any one of the above items, and the flight control method for the unmanned aerial vehicle as described above.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The invention provides a method and system for constructing a flight control network of an unmanned aerial vehicle. First, a posture control network of an unmanned aerial vehicle based on the RM-DDPG algorithm is designed in sequence to process time series data and multi-agent learning to optimize the posture control strategy of the unmanned aerial vehicle. A path planning network in a three-dimensional environment based on hierarchical deep reinforcement learning is designed, which can receive the current state information and environmental information of the unmanned aerial vehicle, and plan the optimal flight path of the unmanned aerial vehicle in the three-dimensional environment through a hierarchical decision-making process. At the same time, an intelligent avoidance network based on hierarchical deep reinforcement learning is designed to achieve real-time detection and prediction of potential risks and obstacles of the unmanned aerial vehicle during flight, and generate the optimal avoidance action according to the current state and environmental information of the unmanned aerial vehicle; secondly, a posture control network, a path planning network and an intelligent avoidance network of the unmanned aerial vehicle based on hierarchical deep reinforcement learning are constructed. And based on the flight data of the unmanned aerial vehicle in different environments (such as: the attitude state of the unmanned aerial vehicle, the flight trajectory, the environmental information and the flight mission, etc.) and the preset flight reward mechanism that is conducive to network training and control, the learning and optimization of the hierarchical deep reinforcement learning network architecture are trained and guided. In this way, on the one hand, the adaptability and flexibility of drones during flight can be improved, that is, the flight autonomy of drones can be improved, the ability to adapt to complex environments can be improved, the decision-making process can be optimized, and the ability to avoid obstacles can be enhanced, thereby improving the efficiency of drone missions, effectively reducing human operating errors and reducing costs, thereby improving the reliability and economic benefits of drone inspections. On the other hand, the flight control network built by this solution also has the ability to continuously learn and improve, can respond to emergencies, and promote technological innovation.

[0049] The present invention provides a control method and system for a drone, which outputs corresponding flight control instructions by means of a flight control network that improves the autonomy of the drone, adapts to complex environments, optimizes decision-making processes, enhances obstacle avoidance capabilities, and improves mission execution efficiency, so as to improve the adaptability and flexibility of the drone during flight.

[0050] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions provided by the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0052] Figure 1 A schematic diagram of a flow chart of a method for constructing a flight control network of an unmanned aerial vehicle provided by the present invention;

[0053] Figure 2 A schematic diagram of a strategy-evaluation network architecture provided by the present invention;

[0054] Figure 3 A schematic diagram of the architecture of a posture control network of a drone provided by an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a network architecture of a path planning network provided by an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of a network architecture of a path planning network provided by an embodiment of the present invention;

[0057] Figure 6 A schematic diagram of a flow chart for constructing a flight control strategy using a method for constructing a flight control network of an unmanned aerial vehicle provided by the present invention;

[0058] Figure 7 A schematic diagram of a flow chart of a method for controlling a drone provided by the present invention;

[0059] Figure 8 A schematic diagram of the composition of a system for constructing a flight control network of an unmanned aerial vehicle provided by an embodiment of the present invention;

[0060] Fig. 9 A schematic diagram of the composition of a control system of an unmanned aerial vehicle provided by an embodiment of the present invention;

[0061] Fig.10 A schematic diagram of the composition of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0063] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0064] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0065] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0066] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0067] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0068] The flight control system is crucial to the safe and efficient operation of aircraft. With the widespread application of drones and other technologies in civil and military fields, the requirements for adaptability and flexibility of flight control systems have increased. Traditional flight control systems are not adaptable enough when facing unknown environments or emergencies. In related technologies, with the development of artificial intelligence technology, it has promoted the application of deep learning algorithms in complex tasks, but flight control still faces the challenges of real-time and environmental uncertainty. However, when using deep learning algorithms to process flight control, there are usually the following technical problems: how to design an effective hierarchical structure to adapt to different flight missions and environments; how to balance the information transmission and coordination of control strategies between different levels; how to ensure the efficient operation of the algorithm under limited computing resources and real-time requirements; how to achieve rapid adaptation and decision-making in unknown environments while ensuring flight safety.

[0069] Based on the above problems, the present invention proposes a flight control network construction method, a control method and a system, which can improve the adaptability and flexibility of the UAV during flight.

[0070] Embodiment 1:

[0071] The present invention provides a method for constructing a flight control network of an unmanned aerial vehicle. Figure 1 FIG. 1 is a flow chart of a method for constructing a flight control network of an unmanned aerial vehicle provided by the present invention; wherein, in combination with Figure 1 As shown, the construction method of the UAV flight control network is described as follows:

[0072] Step 101: Using a deep deterministic policy gradient algorithm, based on the state information and target flight value of the UAV during the flight, build an attitude control network of the UAV during the flight.

[0073] In some embodiments of the present invention, the deep deterministic policy gradient algorithm may be: Reinforcement Learning-based Multi-agent-DeepDeterministic Policy Gradient (RM-DDPG) algorithm. Here, the attitude control network of the UAV during flight can be constructed based on the RM-DDPG algorithm. Due to the characteristics of the RM-DDPG algorithm, it can not only eliminate the attitude steady-state error of the UAV during flight, but also enhance the robustness of the network, thereby achieving dynamic stability and rapid response of the UAV during flight. And it can optimize the overshoot phenomenon that may occur in the reinforcement learning controller during target tracking.

[0074] It should be noted that a drone is an unmanned aerial vehicle, which may be a quadcopter, an unmanned fixed-wing aircraft, etc., and the present invention does not impose any limitation on this.

[0075] In some embodiments of the present invention, the RM-DDPG algorithm can be used to build a posture control network for the UAV during flight based on the state information of the UAV during flight, the target flight value corresponding to the UAV (such as: the position information corresponding to the aerial object that the UAV needs to capture) and the environmental information of the UAV.

[0076] In some embodiments of the present invention, before executing step 101, the method for constructing the flight control network of the UAV may perform the following steps A1:

[0077] Step A1: Establish a reference model corresponding to the UAV during flight.

[0078] In some embodiments of the present invention, a reference model corresponding to a UAV control system with high stability and dynamic performance may be established.

[0079] In some embodiments of the present invention, the output state variables corresponding to the UAV during flight, namely, angle, angular velocity, angular acceleration, and angular error integral, etc., can be obtained first; then, based on the angle, angular acceleration, angular acceleration, and angular error integral, etc., a corresponding reference model can be established.

[0080] Correspondingly, the above step 101 can be implemented by the following step A2:

[0081] Step A2: input the state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training to obtain the attitude control network.

[0082] In some embodiments of the present invention, the state information, target flight value and reference state information of the UAV during the flight (obtained by inputting the environmental information and target flight value of the UAV into the reference model) can be input into the strategy-evaluation network architecture built based on the RM-DDPG algorithm for network training to obtain the attitude control network corresponding to the UAV flying to the target flight value.

[0083] Among them, because the RM-DDPG algorithm is based on DDPG, its matching strategy-evaluation network architecture (Actor-Critic network architecture), that is, the AC network structure can be referred to Figure 2 As shown, it involves: state 201, strategy 202, environment 203 and value function 204. The AC network structure is mainly divided into two parts: Actor network and Critic network, where: Actor network and Critic network have one thing in common: input state 201, Actor network outputs strategy 202 based on state 201, and is responsible for selecting actions, and executes actions in environment 203 to enter the next state, and obtains Reward, which is input into value function 204; Critic network, according to input state 201, also inputs the output evaluation value into value function 204, so that value function 204 updates network parameters, and calculates the optimal value after updating to calculate the time difference error (TD-error). Value function 204 then uses TD-error to update the strategy corresponding to the output of Actor network, that is, strategy 202.

[0084] It should be noted that the Actor network is implemented using the policy gradient algorithm, which can select an action based on the current state and use the action to interact with the environment; the goal of the Actor network is to maximize the expectation of the cumulative return, and to improve the way of selecting actions by continuously updating the policy function so that the selected actions can obtain higher returns; the Actor network is usually implemented using a neural network, with the input being the current state and the output being an action. The training goal of the network is to maximize the expectation of the cumulative return. The Critic network is implemented using the temporal difference, which estimates the value function of the current policy; its goal is to evaluate the quality of the actions selected by the Actor network and provide feedback information to the Actor network; the Critic network is also implemented using a neural network, which can estimate the value function of the current policy, that is, it can evaluate the quality of the Actor network.

[0085] Correspondingly, in the reasoning phase of the Actor-Critic architecture, the Actor network and the Critic network collaborate to learn how to better interact with the environment. The Actor network continuously updates the policy function to improve the way of selecting actions, while the Critic network provides feedback information to guide the update direction of the Actor network. This collaboration enables the Actor-Critic algorithm corresponding to the Actor-Critic network architecture to learn and adapt to changes in the environment more effectively.

[0086] It should be noted that since the RM-DDPG algorithm is based on the Deep Deterministic Policy Gradient (DDPG) algorithm, the RM-DDPG algorithm uses the deterministic policy gradient method to optimize the Actor function. The core of the algorithm is to use the action function π u The parameter u in represents the strategy parameter, and the strategy parameter u is continuously optimized to approach the optimal strategy, as shown in formula (1):

[0087]

[0088] Among them, the strategy parameter u can be continuously optimized along the direction of the gradient of formula (1) to approach the optimal strategy. is the gradient of the Actor network, ρ π (s) is the state distribution, To take the gradient of the policy parameters, π μ (a|s) is the distribution of actions, To take the gradient of the action, is the state-action value function, π ais the strategy, E is the expected value of the probability that the agent takes actions according to the strategy under different states, s is the state parameter, a is the action parameter, and A and ρ are parameters representing the strategy.

[0089] Among them, based on the deterministic policy gradient method to optimize the Actor-Critic network architecture, formula (1) defines the state-action value function The core of this is the fixed action selection of the deterministic strategy for a given state s and parameter θ. This means that once the initial state is determined, the behavior trajectory becomes unique. Therefore, it is crucial to choose an appropriate action value function. The Actor-Critic algorithm can be used here, in which the Actor network is used to generate action strategies and the Critic network is used to evaluate the effectiveness of the strategy. By introducing an alternative state-action value function with the parameter to be approximated ω To approximate the original state-action value function Correspondingly, the parameters of the Actor network are optimized and updated according to the deterministic policy gradient. The sampling process of the Actor network is to obtain the policy π from the state s through the parameter θ. μ (s), a certain amount of noise can be added to the sampling process to enhance exploration (to give it a certain exploration ability), and the corresponding reward parameter is Q ω (s t ,a t ), is determined by the Critic network, and the corresponding update process can be shown in formula (2):

[0090]

[0091] Among them, a θ is the learning rate, θ t+1 and θ t are the strategy parameters corresponding to time t+1 and time t respectively, is the gradient of the strategy at time t with respect to state s, is the gradient of the action value function at time t with respect to action a, s t 、a t are the state and action at time t respectively.

[0092] It should be noted that the Actor-Critic algorithm usually uses a neural network to replace the Actor function and the Critic function. In some embodiments of the present invention, in order to improve the efficiency and stability of network training, experience data replay can also be applied and the target network can be designed. For example, a playback buffer D of size M is designed to store experience data e. When the buffer is full, the earliest data is discarded. In addition, the Actor network and the Critic network are updated by random sampling to reduce data correlation. Here, the target network is introduced to improve stability through soft update. Among them, the Critic target network is updated by resampling the experience data and minimizing the loss function. Among them, by using the integral of the angular velocity error as an extended state substitute controller, the steady-state error can be effectively eliminated. In the process of controlling the drone, excessive control amount may cause the drone to roll over, thereby violating the principle of physical motion. Therefore, a reward function can be set in the network. The setting of the reward function is the key to solving this problem. For complex tasks, state quantities such as angle, angular velocity, and angular acceleration need to be considered to achieve a balance between the drone's tracking of the target and the stability of the drone's control system.

[0093] Based on the above description, the attitude control network of the UAV during flight can be built based on the RM-DDPG algorithm, such as Figure 3 As shown, it is a schematic diagram of the architecture of a posture control network of a drone provided by an embodiment of the present invention; wherein the input of the Actor network, that is, 304, is the drone ( Figure 3 The state vector of the drone model 302 in the figure and the state vector of the reference model 301 are compared. The purpose of the Actor network is to simulate an actor function with excellent performance and sufficient stability. The Actor network has two fully connected hidden layers, each of which is a 128-dimensional network with a sigmoid activation function. The output of the Actor network is the control signal of the motor. In order to ensure that the output of the control signal is always within a reasonable range, the sigmoid function is also applied to the output layer. At the same time, the input of the Critic network, i.e., 305, is the drone ( Figure 3 The state of the drone model in the image, the state of the reference model 301, and the control input generated by the Actor network 304, whose hidden layer and activation function are similar to those of the Actor network. In addition, in order to train the Actor network faster and better, the output layer of the Critic is replaced with a linear activation function. Here, Reward is the sum of all state errors, which is input to the system as feedback. In this way, the processing can enable the Actor network to avoid the steady-state error in the system when it is output. These designs enable the control signal to quickly and stably track the reference model while preventing overshoot and steady-state errors.

[0094] After the above processing, the control signal can quickly and stably catch up with the output of the reference model 301, and at the same time, it can avoid the system overshoot caused by too much control amount leading to too fast tracking speed of the target angle, and can effectively compensate for the steady-state error. Figure 3 As shown in the figure, a Critic network is cloned as the Critic target network, i.e., 306, and then the Actor network outputs a control signal to the drone model 302, and the reference model 301 also outputs a target value 303. Here, the state of the drone model 302 and the state of the reference model 301 are processed to obtain the sum of the corresponding errors, and the sum is fed back to the Critic target network 306 as a reward, i.e., 307. At the same time, for each training step, the state of the trained model will be stored in the memory buffer, and training will begin when there are enough samples. Here, the TD error can be calculated using the outputs corresponding to the Critic target network (306) and the Critic network (305) as the loss function, i.e., 308, and the Actor network (304) and the Critic network (305) are trained and adjusted synchronously based on the loss function 308.

[0095] It should be noted that the freedom and scope of exploration of deterministic strategies are much lower than those of random strategies. This is because the actions output by deterministic strategies are deterministic actions and lack exploration of the environment. During the training phase, noise can be added to the actions output by the Actor network to give it a certain exploration capability, that is, to design and add Gaussian noise to construct an exploration strategy. This can improve the freedom and scope of exploration of the Actor-Critic network architecture.

[0096] In some embodiments of the present invention, in the flight control of the UAV, attitude state information (such as roll angle, pitch angle and yaw angle, etc.) is an important indicator reflecting the current flight state of the UAV. These attitude state information are not only affected by the current control instructions, but also by the previous flight state of the UAV. Therefore, it has significant timing characteristics. In order to accurately capture the dynamic changes of the attitude of the UAV, the cyclic connection structure within the strategy-evaluation network architecture matched by the RM-DDPG algorithm can be used to memorize and process these timing data. Specifically, through the cyclic connection, the output of the previous time step is used as one of the inputs of the current time step, thereby realizing the effective processing of the timing information.

[0097] In this way, by processing the drone's time series data, the dynamic changes of the drone's attitude can be captured more accurately and the future attitude state can be predicted. This prediction capability is crucial for drone flight control because it can help perceive the drone's flight trend in advance and make more accurate control decisions.

[0098] At the same time, the network model is combined with the DDPG algorithm in the RM-DDPG algorithm architecture to form a complete drone attitude control strategy. It can not only make control decisions based on current environmental information and reward signals, but also use memory capabilities to use past flight experience to guide current and future control behaviors, thereby achieving high-precision control of the drone attitude.

[0099] In this way, by integrating the strategy-evaluation network architecture matched by the RM-DDPG algorithm and the reference model, an attitude control network is obtained, which can not only quickly balance the rationality between the target tracking and control input of the UAV, thereby avoiding excessive control caused by excessive pursuit of response speed, but also effectively cope with external interference and environmental uncertainty changes to achieve rapid response to the target flight value, while effectively reducing overshoot and steady-state error, so as to improve the flight control performance of the UAV.

[0100] Step 102: Using a deep reinforcement learning algorithm, based on the state information and the environmental information of the UAV, respectively build a path planning network and an intelligent avoidance network for the UAV during flight.

[0101] In some embodiments of the present invention, the deep reinforcement learning algorithm includes at least: Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, Soft Actor-Critic (SAC) algorithm, and Unit State Sequence (USS)-based reinforcement learning algorithm. Correspondingly, an adaptive algorithm can be selected, and based on parameters such as the state information of the drone, a path planning network and an intelligent avoidance network of the drone during flight can be built respectively.

[0102] Continuing from the above description, the above step 102 can be implemented by the following steps 1021 and 1022 (not shown in the figure):

[0103] Step 1021: input the state information and the distance information in the environment information into the recursive neural network constructed by the double-delayed deep deterministic policy gradient algorithm and the soft actor-critic algorithm to perform network training to obtain the path planning network.

[0104] The distance information includes at least: the distance between the drone and an obstacle identified by the drone during flight.

[0105] In some embodiments of the present invention, a hierarchical deep reinforcement learning algorithm can be used based on the state information of the drone during flight and the environmental information in which it is located, and the optimal flight path of the drone in a three-dimensional environment can be planned through a hierarchical decision-making process to build a path planning network.

[0106] In some embodiments of the present invention, a new three-dimensional and mapless navigation strategy network for drones based on hierarchical deep reinforcement learning, i.e., a path planning network, can be constructed; that is, without using an image-based sensing method, a simple learning strategy is proposed, using only a small amount of sparse range data from a distance sensor to train the learning agent of the relevant network. The new hierarchical deep reinforcement learning-based network can be built based on two state-of-the-art dual-critic deep RL models: the TD3 algorithm and the SAC algorithm, which are superior to the DDPG technology and the obstacle avoidance method (BugAlgorithms2, BUG2) algorithm. In addition, the new deep reinforcement learning (RL) structure based on the recurrent neural network (RNN) is also superior to the current structure used to perform mapless navigation of mobile robots.

[0107] Here, the path planning network can learn to navigate from the starting position in space to the target position, and establish its own motion plan in an environment with obstacles through a series of target points. Only two distance sensor data and the relative positioning data of the drone are needed to perform map-free navigation and obstacle avoidance. The corresponding motion equation is shown in formula (3):

[0108] v t =f(x t ,p t ,v t-1 ) formula (3);

[0109] Among them, v t is the speed of the drone at time t, x t is the raw data of the sensor reading, p t is the relative position and angle, v t-1 is the speed of the drone at the last moment of time t. This path planning network allows to obtain the actions that the drone can make, given its current state s t The corresponding ones are all reflected in the path planning network, and the expected result of the path planning network is the action of the current state.

[0110] In some embodiments of the present invention, the network architecture Q of the path planning network may have a structure of 26 inputs and 3 outputs. Figure 4As shown in FIG. 1 , a schematic diagram of the network architecture of the path planning network is shown; among the 26 inputs, 20 represent distance findings from simulated lidar, 3 represent previous actions of the drone, and the other 3 represent information on flight target values. Figure 4 As shown. Here, the lidar can provide 1080 samples within a range of 270°, 20 of which are sampled at 13.5°. The information of the flight target value used is the relative distance of the drone from the flight target value, and the two relative angles to the flight target value. The output of the network architecture Q provides an action for a given step size, which is determined based on 3 outputs: the linear speed of the drone, the altitude speed, and the change in the yaw angle. These actions are then applied to the path planning of the drone.

[0111] It should be noted that the network architecture of the path planning network can also be: Actor-Critic network architecture; Figure 5 As shown, 501 is part of the network layer included in the Actor network, and 502 is part of the network layer included in the Critic network. Here, the input of the Actor network is: the state s of the drone at time t t , which includes: Long Short-Term Memory (LSTM) layer and two linear layers; correspondingly, the LSTM layer has a ReLU activation function, and the two linear layers have a ReLU activation function and a hyperbolic tangent function Tanh. At the same time, the input of the Critic network is: the state s of the drone at time t t and action a t , which in turn includes: four linear layers, and the linear layers have ReLU activation functions.

[0112] Here, the LSTM layer can be a 32-unit LSTM layer for the Actor network activated with a ReLU function. Correspondingly, the hyperbolic tangent function Tanh is used as the activation function in the output of the Actor network. The value of the linear speed of the linear layer in the Actor network is scaled between 0 and 0.25m / s, and the delta yaw is between -2.5 and 2.5rads. The height speed is -0.25 and 0.25m / s. The Q value of the current state and the action performed by the agent is given in the Critic network. In addition, for the SAC-based method, the structure of the Critic network is the same as that of the critic network.

[0113] In this way, based on the state information of the UAV during the flight and the environmental information in which it is located, a path planning network is built using a hierarchical deep reinforcement learning algorithm. This network can comprehensively consider various environmental factors such as wind speed, wind direction, terrain, etc. during the flight of the UAV, and plan the optimal flight path of the UAV in a three-dimensional environment through a hierarchical decision-making process.

[0114] Step 1022: input the state information and obstacle information in the environment information into the neural network constructed by the reinforcement learning algorithm based on the unit state sequence to perform network training, so as to obtain the intelligent avoidance network.

[0115] In some embodiments of the present invention, the USS-based reinforcement learning algorithm may be a USS-based hierarchical deep reinforcement learning algorithm. Thus, the obstacle information in the state information and environmental information of the drone may be input into a neural network constructed by the USS-based hierarchical deep reinforcement learning algorithm for network training, so as to obtain an intelligent avoidance network for the drone during flight.

[0116] It should be noted that reinforcement learning, especially hierarchical deep reinforcement learning, can effectively cope with large-scale state spaces and complex tasks by decomposing tasks, multi-level learning and decision-making, so as to improve learning efficiency and the generalization and robustness of the algorithm.

[0117] Here, the hierarchical deep reinforcement learning algorithm based on USS needs to perform the following operations in sequence: feature encoding, distance priority sorting and USS construction, and extraction of hidden features; the specific execution logic is as follows:

[0118] First, the obstacle information and the UAV status information are feature encoded one-to-one to form feature units. These feature units must be able to fully express the UAV status information and the characteristics of the obstacles.

[0119] Secondly, all the encoded feature units are sorted according to the distance priority between the drone and the obstacle, sorted and combined into USS. Here, the sorting process helps the drone to deal with the closest and most urgent obstacles first when avoiding obstacles. At the same time, the sorted feature units are combined into a sequence, namely USS. This sequence will be used as the input of the deep reinforcement learning algorithm to reflect the state of the drone's surrounding environment at a specific moment.

[0120] Then, the gated recurrent unit (GRU) is used to fuse the feature units in the USS, capture dynamic changes, extract the hidden information (hidden features) as the state information at that moment, and input the state information into the corresponding reinforcement learning neural network. Here, hidden feature extraction is the core link. In addition, CNN can be used to extract deep features (i.e., hidden features) to understand the structure of the environment. Among them, in practical applications, the policy iteration and end-to-end training optimization extraction process in the hierarchical deep reinforcement learning algorithm based on USS can omit manual feature engineering. In this way, multimodal fusion is realized to enhance environmental perception, and obstacle avoidance and navigation are combined to improve task execution capabilities. At the same time, in the design of the neural network constructed based on the hierarchical deep reinforcement learning algorithm of USS, real-time and generalization are emphasized, which can ensure that the drone can effectively avoid obstacles in complex environments.

[0121] In some embodiments of the present invention, USS can be further combined with DDQN and DDPG algorithms to improve the training effect of the UAV in the continuous obstacle avoidance scenario experiment, that is, the reinforcement learning neural network mentioned above can be applied to the double deep Q network (Double Deep Q Network, DDQN) algorithm and the DDPG algorithm, and trained on the UAV obstacle avoidance simulation platform. For example, in the constructed intelligent avoidance network, the environmental observation data (the state information and obstacle information of the UAV) is first converted into USS. Subsequently, GRU processes USS to fuse and refine features to generate a fixed-length feature vector Feature. These feature vectors can be directly input into the reinforcement learning model. Here, the DDQN algorithm can be further combined to realize that the GRU module is responsible for extracting features from USS and updating them together with the evaluation network parameters. During the update process, the gradient will be passed back to the GRU module to achieve synchronous optimization of parameters.

[0122] Here, the network structure corresponding to the USS-based DDPG algorithm can include two GRU units, which serve the policy network and the value network respectively to process the feature features. In the network parameter update phase, the gradient will be passed to the corresponding GRU unit to ensure the synchronous optimization of the feature extraction process.

[0123] It should be noted that during the flight of the drone, a variety of detection devices are used to detect targets or obstacles. When facing multiple obstacles, the number of environmental entities will change dynamically. Traditional neural networks need to input environmental information according to the maximum number of entities, but not all information will affect the decision. Therefore, it is crucial to effectively obtain environmental situation characteristics and filter important state information to improve the training effect. Here, the USS encoding method can map the drone and the target or obstacle into state units by drawing on the word vector encoding in natural language processing, and encode the state units into vectors of equal length to distinguish the differences. The GRU network is used to extract hidden features from the USS to apply to single entities, such as drones against multiple targets, that is, multiple obstacles (variable) scenarios.

[0124] In this way, based on the obstacle information in the state information and environmental information, the network is trained in the neural network constructed by the reinforcement learning algorithm based on the unit state sequence to obtain an intelligent avoidance network, which can not only detect and predict potential risks and obstacles in real time during the flight of the drone, but also generate the optimal avoidance action according to the current state and environmental information of the drone. In other words, the intelligent avoidance network can use the deep learning network to detect obstacles in real time, and provide the drone with the optimal avoidance path through calculation and analysis. This ensures that the drone can respond quickly and accurately when encountering obstacles to avoid collision accidents.

[0125] It should be noted that the intelligent avoidance decision mechanism corresponding to the intelligent avoidance network is a crucial part of the UAV flight control system. It cleverly combines the real-time sensor data of the UAV, detailed map information and advanced prediction models to ensure that the UAV can quickly perceive and accurately avoid potential risks during flight. Among them, real-time sensor data is the key for the UAV to perceive its surrounding environment. These real-time sensor data include measurements from devices such as lidar, infrared cameras, ultrasonic sensors, etc., which can reflect the position, shape, speed and other information of obstacles around the UAV in real time. The intelligent avoidance decision mechanism can process these real-time sensor data in real time and make an instant assessment of the risks around the UAV. Here, in addition to real-time sensor data, map information also provides an important reference for the intelligent avoidance decision mechanism. Map information includes pre-loaded static obstacle information (such as buildings, mountains, etc.), terrain height data, flight restricted areas, etc. This information can provide a macroscopic environmental view for the UAV, helping the UAV to avoid collisions with known obstacles during flight and choose a safer flight path. The prediction model plays a vital role in the intelligent avoidance decision-making mechanism. It can predict the risks that may occur in the future based on historical data, current environmental information and the flight status of the drone. These prediction results can provide early warnings for drones, giving them enough time to react and avoid potential risks.

[0126] Through the above steps, the path planning network constructed based on the hierarchical deep reinforcement learning algorithm can receive the current state information and environmental information of the UAV, and plan the optimal flight path of the UAV in a three-dimensional environment through a hierarchical decision-making process; at the same time, the intelligent avoidance network based on hierarchical deep reinforcement learning can enable the UAV to detect and predict potential risks and obstacles in real time during flight, and generate the optimal avoidance action based on the current state and environmental information of the UAV.

[0127] Step 103: construct a hierarchical deep reinforcement learning network for the UAV during the flight based on the attitude control network, the path planning network and the intelligent avoidance network.

[0128] In some embodiments of the present invention, a hierarchical deep reinforcement learning network structure can be constructed based on the attitude control network, the path planning network and the intelligent avoidance network. Here, a basic network architecture can be built first, and then the attitude control network, the path planning network and the intelligent avoidance network are respectively used as the strategy networks in the basic network architecture for network construction to obtain a hierarchical deep reinforcement learning network of the drone during flight.

[0129] In some embodiments of the present invention, the above step 103 may be implemented by the following steps 1031 and 1032 (not shown in the figure):

[0130] Step 1031: Based on the state information, the executable action information of the drone during flight, and the corresponding reward information, build an initial level deep reinforcement learning network infrastructure.

[0131] In some embodiments of the present invention, the infrastructure of the initial level deep reinforcement learning network can be built based on the state information, executable action information and corresponding reward information of the drone during flight. The infrastructure can be constructed with the following basic parameters:

[0132] 1. State space: can be used to define the state of the drone in a three-dimensional environment, including: the position, speed, attitude, environmental information, etc. of the drone during flight.

[0133] 2. Action space: It can be used to define the actions that the drone can perform, including: attitude adjustment of the drone during flight, steering and speed control in path planning, etc.

[0134] 3. Reward function: It can be used to describe the behavior of the drone in the environment and design a reward mechanism to guide the initial level deep reinforcement learning network to learn and optimize.

[0135] Step 1032: sequentially use the posture control network, the path planning network, and the intelligent avoidance network as a low-level deep reinforcement learning subnetwork, a mid-level deep reinforcement learning subnetwork, and a high-level deep reinforcement learning subnetwork in the infrastructure to obtain the hierarchical deep reinforcement learning network.

[0136] Among them, the high-level deep reinforcement learning sub-network can control the decision of the middle-level deep reinforcement learning sub-network, and the middle-level deep reinforcement learning sub-network can control the decision of the low-level deep reinforcement learning sub-network.

[0137] In some embodiments of the present invention, the posture control network can be used as a low-level deep reinforcement learning subnetwork (Sub-Policy) (low-level policy network) in the infrastructure, the path planning network can be used as a mid-level deep reinforcement learning subnetwork (middle-level policy network) in the infrastructure, and the intelligent avoidance network can be used as a high-level deep reinforcement learning subnetwork (Meta-Policy) (high-level policy network) in the infrastructure, thereby obtaining a hierarchical deep reinforcement learning network.

[0138] It should be noted that the path planning network uses hierarchical deep reinforcement learning technology to design three-dimensional environment path planning. It can receive the current state and environmental information of the drone to output path planning instructions, and the intelligent avoidance network can be responsible for generating avoidance instructions when the drone encounters obstacles during flight, guiding the drone to safely bypass. Correspondingly, the attitude control network uses the RM-DDPG algorithm, combined with RNN to process time series data, to achieve precise control of the drone's attitude. It can accept instructions output by the middle-level strategy network and the high-level strategy network, such as: path planning and avoidance instructions output by the path planning network and the intelligent avoidance network, and output specific attitude control actions.

[0139] Here, the low-level deep reinforcement learning subnetwork, the middle-level deep reinforcement learning subnetwork, and the high-level deep reinforcement learning subnetwork in the hierarchical deep reinforcement learning network can communicate through an effective information exchange mechanism. The middle-level policy network and the high-level policy network pass the output path planning and avoidance instructions to the low-level policy network, respectively, and the low-level policy network feeds back the output posture control state to the middle-level policy network and the high-level policy network.

[0140] Here, the low-level policy network, middle-level policy network, and high-level policy network in the hierarchical deep reinforcement learning network can be jointly trained to jointly optimize the performance of the entire hierarchical deep reinforcement learning network. At the same time, the three policy networks can collaborate with each other by sharing reward signals and state information, thereby achieving more efficient learning.

[0141] In this way, a hierarchical deep reinforcement learning algorithm is introduced, which realizes the hierarchical decision-making process from low-level attitude control to middle-level path planning and then to high-level intelligent avoidance decision-making through a unique network architecture. The core of this hierarchical deep reinforcement learning algorithm lies in its multi-level network structure; first, the low-level policy network focuses on the attitude control of the UAV, which receives data from sensors deployed on the UAV, such as data collected by gyroscopes, accelerometers and GPS, and perceives the current attitude information of the UAV in real time. By processing and analyzing this information, the bottom-level low-level policy network can generate accurate attitude control instructions to ensure that the UAV performs flight actions stably and accurately; then, the middle-level policy network focuses on path planning, which uses deep reinforcement learning technology to learn the optimal path planning strategy based on the target position and current environment information provided by the high-level policy network. The middle-level policy network can consider various complex environmental factors, such as terrain, obstacles, wind direction, etc., so as to plan a safe and efficient flight path; finally, the high-level policy network is responsible for intelligent avoidance decisions. It can make quick and accurate judgments on possible obstacles or emergencies based on the path planning provided by the middle-level policy network and the real-time flight status fed back by the bottom-level policy network. In addition, through deep reinforcement learning, the high-level policy network can learn how to intelligently avoid risks in various complex scenarios to ensure the flight safety of the drone. In this way, the hierarchical decision-making process can not only improve the autonomy and intelligence of drone flight, but also greatly enhance its ability to cope with complex environments and emergencies. Through the collaborative work of sub-networks at all levels, the drone can autonomously complete the entire flight process from takeoff, cruising to landing without human intervention, and make quick and accurate decisions when necessary to adapt to complex and changing flight environments.

[0142] Step 104: Use the historical flight data of the UAV in different flight environments and the preset flight reward mechanism to train the hierarchical deep reinforcement learning network to obtain the flight control network of the UAV.

[0143] In some embodiments of the present invention, historical flight data may be collected first and a preset flight reward mechanism may be constructed, so that a hierarchical deep reinforcement learning network may be trained and constructed based on the obtained historical flight data and the preset flight reward mechanism to obtain a flight control network for the UAV.

[0144] Here, the construction process of the preset flight reward mechanism can refer to the following steps B1 to B3:

[0145] Step B1: constructing a reward signal reflecting the flight quality of the drone based on the flight parameters of the drone during flight.

[0146] Wherein, the flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters;

[0147] In some embodiments of the present invention, a reward signal reflecting the flight quality of the UAV can be constructed based on the flight parameters of the UAV during the flight, such as flight stability parameters, flight corresponding route tracking parameters, flight corresponding energy consumption parameters, etc. Here, the reward signal can be used to reflect the core objectives of flight control, and correspondingly, appropriate weights can be assigned to the reward signals corresponding to these flight parameters to balance their importance in the overall flight control strategy.

[0148] Step B2: construct the reward strategy corresponding to the UAV in different flight control stages.

[0149] In some embodiments of the present invention, the reward strategy of the drone corresponds to different flight control stages, wherein the different flight control stages include at least: take-off, landing, rotation, etc. Here, the reward strategy can be a multi-target and multi-stage reward. In order to take into account the diversity of control tasks, the research will explore how to apply different reward strategies in different control stages and realize dynamic adjustment of rewards, while avoiding potential conflicts between different reward items, and ensuring that the network can learn the global optimal control strategy.

[0150] Step B3: constructing the preset flight reward mechanism based on the reward signal and the reward strategy.

[0151] In some embodiments of the present invention, a preset flight reward mechanism can be constructed based on the reward signal and the reward strategy. Correspondingly, a reward function corresponding to the preset flight reward mechanism can be constructed to facilitate subsequent training and adjustment of related networks.

[0152] It should be noted that the preset flight reward mechanism here may be the same as or different from the reward mechanism involved in the above-mentioned "reward mechanism for describing the behavior design of the drone in the environment", and the present invention does not impose any limitation on this.

[0153] It should be noted that the importance of the preset flight reward mechanism lies in the ability to flexibly design a reward function with clear guidance according to different flight mission requirements. This reward function can not only provide guidance signals for the drone during the flight process, but also ensure that the drone can achieve efficient and safe flight in complex environments.

[0154] In this way, with the help of the corresponding reward signals and reward strategies of the drone during the flight, the preset flight reward mechanism can enable the subsequent guidance network to learn more effective and reasonable flight strategies during the training process.

[0155] In some embodiments of the present invention, the above step 104 may be implemented by the following steps 1041 and 1042 (not shown in the figure):

[0156] Step 1041: annotate and organize the historical flight data to form a training data set.

[0157] In some embodiments of the present invention, the construction process of constructing a training data set based on historical flight data includes three stages: data collection, labeling, and organization.

[0158] The process of collecting historical flight data is as follows: Use the flight platform to test the drone under various environmental conditions, including different weather conditions (such as clear, cloudy, rainy, windy, etc.), different time periods (day and night), and different terrain features (such as plains, mountains, different altitudes, etc.), to ensure that sufficient flight data samples are collected in each typical environmental combination. At the same time, flight data under complex conditions such as sensor failure, airflow disturbance, and abnormal attitude can be collected through artificial interference to obtain historical flight data.

[0159] In some embodiments of the present invention, historical flight data is annotated, such as the collected historical flight data is annotated in detail to obtain annotated information, such as environmental status, aircraft attitude, and human interference, etc. This process can be initially annotated by an automated algorithm, and then manually reviewed and corrected.

[0160] In addition, experts in the field of flight control can be invited to participate in the quality assessment of historical flight data and the correction of erroneous annotations.

[0161] In some embodiments of the present invention, the historical flight data is sorted, and the annotated historical flight data is processed and organized according to the input and output requirements of the control network to form a standardized training data set. Here, because the input data of the network involved in the subsequent operation contains flight status characteristics, the output data is the expected control instruction. Therefore, the annotated and sorted historical flight data should include normal flight data and hardware failure data in complex environments, and the failure conditions should be specially marked to test the robustness of the control instructions output by the subsequent network.

[0162] Here, historical flight data is labeled and organized to form a comprehensive and diverse training data set. Because this training data set covers data corresponding to flight conditions in conventional and complex environments, it can effectively train the reinforcement learning controller, that is, the related network structure that needs to be trained later.

[0163] Step 1042: Use the preset flight reward mechanism to build a reward function of the hierarchical deep reinforcement learning network to obtain the network to be adjusted, and use the training data set to train the network to be adjusted to obtain the flight control network.

[0164] In some embodiments of the present invention, first, a reward function of a hierarchical deep reinforcement learning network is constructed based on a preset flight reward mechanism, thereby obtaining a new network, namely, a network to be adjusted; then, the network to be adjusted is trained using a training data set to obtain a flight control network that outputs an optimal flight control strategy. Here, the network to be adjusted also includes: an attitude control network, a path planning network, and an intelligent avoidance network.

[0165] In some embodiments of the present invention, the constructed training data set can be loaded into the framework corresponding to the network to be adjusted for network training, and regularization technology can also be used to prevent overfitting. In addition, the construction of the training data set involved above is also convenient for subsequent incremental updates. By continuously collecting new flight data and annotating it, the data set can be continuously enriched and the generalization ability of the flight control network can be optimized.

[0166] It should be noted that during the training process, in order to avoid the instability of the control process caused by sudden changes in the reward signal, the reward function of the hierarchical deep reinforcement learning network built with the preset flight reward mechanism can be smoothed, and a delay mechanism can be further introduced so that the network to be adjusted can associate the current action with the future flight state, thereby achieving predictive control.

[0167] In some embodiments of the present invention, in the training phase of the network to be adjusted, an adaptive reward adjustment mechanism may be further involved, that is, in the training process of the network to be adjusted, it is possible to study how to automatically adjust the reward function according to the current performance level of the network to promote the adaptability and efficiency of the training process, that is, the adaptive reward adjustment mechanism. At the same time, it is also necessary to explore methods to avoid the network's dependence on specific rewards. In addition, in order to further improve the real-time and adaptability of the control strategy, the study will design an online reward generation module to dynamically adjust the reward according to the real-time execution effect to guide the network to make better control decisions.

[0168] It should be noted that the reward function of the hierarchical deep reinforcement learning network is constructed using the preset flight reward mechanism. The construction process of this reward function needs to consider multiple aspects:

[0169] First, we must have a deep understanding of the specific requirements of the flight mission, including flight path, flight speed, flight altitude, flight attitude and other parameters. These parameters are the basis for designing the reward function and determine which aspects of the performance of the drone need to focus on during flight.

[0170] Secondly, the reward function needs to be clearly oriented. It should be able to clearly reflect the behavior preferences of the drone in the process of completing the task, so that the drone can spontaneously choose a more efficient and safer flight strategy. For example, in an obstacle avoidance task, the reward function can be designed to give higher rewards to successful obstacle avoidance behaviors and punish behaviors that collide with obstacles.

[0171] In addition, the reward function also needs to consider the characteristics of complex environments. In actual flight, drones may encounter various uncertainties and interference factors, such as weather changes, sensor noise, obstacle movement, etc. Therefore, the reward function should have a certain degree of robustness and adaptability to cope with the challenges of these complex environments.

[0172] Finally, the design of the reward function also needs to consider the convergence and stability of the algorithm. This is because a well-designed reward function should ensure that the drone can quickly converge to the optimal strategy during training and maintain stable performance in actual flight.

[0173] In this way, the rich training data set can not only provide rich learning samples for the network model, but also guide the network model to learn more effective and reasonable flight strategies during the training process based on the preset flight reward mechanism, thereby improving the control strategy output by the flight control network and controlling the adaptability and flexibility of the UAV in complex environments.

[0174] Continuing from the above description, the method for constructing a flight control network of a drone provided in an embodiment of the present invention may further perform the following steps C1 and C2 after executing step 104:

[0175] Step C1: Determine a preset network optimization strategy.

[0176] In some embodiments of the present invention, in order to improve the adaptive flight control strategy of the flight control network, it is possible to start from the optimization of the network structure of the flight control network. Its main research direction, that is, the preset network optimization strategy can be constructed from the following aspects, such as:

[0177] 1. Multi-sensor data fusion: A module can be developed to fuse different types of sensor data (such as inertial navigation measurement data, global positioning system measurement data, meteorological data, etc.) to extract the comprehensive flight state characteristics of the UAV as the input of the flight control network. Here, attention mechanisms and graph neural networks can be used to fuse and correlate data to improve the flight performance of the UAV.

[0178] 2. Control command interpreter: An interpreter module is designed at the output end of the flight control network to convert the control command into specific robot application programming interface (API) instructions to enhance the interpretability and accuracy of the control output. Here, the interpreter module can also integrate the flight dynamics model for simulation prediction and command optimization.

[0179] 3. Automatic search and generation of network structures: Use reinforcement learning, evolutionary algorithms and other technologies to automatically explore the optimal network structure, or generate network structures customized for specific tasks based on meta-learning to improve design efficiency.

[0180] 4. Multi-task and transfer learning: Build sub-task networks for different flight phases and different robot platforms, realize knowledge transfer through transfer learning and multi-task learning, and improve learning efficiency.

[0181] 5. Enhance network generalization: Through data enhancement and network regularization technology, the adaptability of the flight control network to different environments is improved to prevent overfitting and cope with more unknown environments. At the same time, adversarial samples are used to improve the robustness of the flight control network.

[0182] 6. Online network optimization: Continuously optimize the parameter weights of the flight control network during flight to adapt to new environmental changes. When encountering a control emergency, quickly adjust the weights of the key layers to achieve online adaptation.

[0183] 7. Performance indicator verification: Develop a set of indicators to quantitatively evaluate the control performance of the flight control network, including: stability error, route tracking error, etc., and test the robustness of the flight control network in situations such as sensor failure.

[0184] Step C2: using the preset network optimization strategy to optimize the network structure of the flight control network to obtain an optimized flight control network.

[0185] In some embodiments of the present invention, the preset network optimization strategy constructed as described above can be directly adopted to optimize the network structure of the flight control network, that is, to adjust and change it to obtain an optimized flight control network, thereby improving the accuracy and robustness of the network's adaptive control.

[0186] Here, a preset network optimization strategy is used to optimize the network structure of the flight control network. Alternatively, the network structure can be continuously optimized and improved during the training of the flight control network. For example, by increasing the depth of the network, the system can extract more abstract and advanced feature information and improve the expression and generalization capabilities of the network model. Alternatively, by improving the network architecture and introducing the attention mechanism, the system can further improve the network model's attention to and processing capabilities of key information, thereby improving the adaptive control accuracy and robustness of the drone in complex environments.

[0187] In addition, the flight control network can be optimized by introducing advanced neural network structures, and / or adopting efficient optimization algorithms, and / or utilizing large-scale distributed computing resources, etc., so as to improve the training efficiency and performance of the network model.

[0188] In this way, by optimizing the network structure of the flight control network, the accuracy and robustness of the adaptive control of the flight control network can be improved, so that the UAV can achieve high-precision and high-robustness flight control in complex and changing environments.

[0189] The invention provides a method for constructing a flight control network of an unmanned aerial vehicle. First, a posture control network of an unmanned aerial vehicle based on the RM-DDPG algorithm is designed in sequence to process time series data and multi-agent learning to optimize the posture control strategy of the unmanned aerial vehicle. A path planning network in a three-dimensional environment based on hierarchical deep reinforcement learning is designed, which can receive the current state information and environmental information of the unmanned aerial vehicle, and plan the optimal flight path of the unmanned aerial vehicle in the three-dimensional environment through a hierarchical decision-making process. At the same time, an intelligent avoidance network based on hierarchical deep reinforcement learning is designed to achieve real-time detection and prediction of potential risks and obstacles of the unmanned aerial vehicle during flight, and generate the optimal avoidance action according to the current state and environmental information of the unmanned aerial vehicle; secondly, a posture control network, a path planning network and an intelligent avoidance network of the unmanned aerial vehicle based on hierarchical deep reinforcement learning are constructed. And based on the flight data of the unmanned aerial vehicle in different environments (such as: the attitude state of the unmanned aerial vehicle, the flight trajectory, the environmental information and the flight mission, etc.) and the preset flight reward mechanism that is conducive to network training and control, the learning and optimization of the hierarchical deep reinforcement learning network architecture are trained and guided. In this way, on the one hand, the adaptability and flexibility of drones during flight can be improved, that is, the flight autonomy of drones can be improved, the ability to adapt to complex environments can be improved, the decision-making process can be optimized, and the ability to avoid obstacles can be enhanced, thereby improving the efficiency of drone missions, effectively reducing human operating errors and reducing costs, thereby improving the reliability and economic benefits of drone inspections. On the other hand, the flight control network built by this solution also has the ability to continuously learn and improve, can respond to emergencies, and promote technological innovation.

[0190] Correspondingly, such as Figure 6 As shown, it is a flow chart of constructing a flight control strategy by using a method for constructing a flight control network of a UAV provided by the present invention, wherein the flow chart of the adaptive flight control method may include the following steps:

[0191] Step 601: construct a UAV attitude controller strategy based on the RM-DDPG algorithm, that is, design a UAV attitude controller strategy based on the RM-DDPG algorithm, which is used to combine RNN to process time series data and multi-agent learning to optimize the attitude control strategy of the UAV.

[0192] Step 602, a three-dimensional environment path planning strategy based on hierarchical deep reinforcement learning, that is, designing a three-dimensional environment path planning strategy based on hierarchical deep reinforcement learning, which can receive the current state information and environmental information of the UAV, and plan the optimal flight path of the UAV in the three-dimensional environment through a hierarchical decision-making process.

[0193] Step 603, intelligent avoidance decision-making based on hierarchical deep reinforcement learning, that is, designing an intelligent avoidance decision-making mechanism based on hierarchical deep reinforcement learning, which can detect and predict potential risks and obstacles in real time during the flight of the UAV, and generate the optimal avoidance action according to the current state and environmental information of the UAV.

[0194] Step 604: construct a hierarchical deep reinforcement learning network architecture, that is, integrate the drone attitude control, three-dimensional path planning and intelligent obstacle avoidance based on hierarchical deep reinforcement learning to construct a hierarchical deep reinforcement learning network architecture.

[0195] Step 605, collecting and annotating flight data under different environments to generate a training data set, that is, collecting and annotating flight data under different environments, including the attitude state, flight trajectory, environmental information and flight mission of the UAV, to generate a training data set, so that the collected training data set can be used to train a hierarchical deep reinforcement learning network architecture, which includes an RM-DDPG attitude controller network (i.e., the UAV attitude controller strategy obtained in 601), a path planning network (i.e., the three-dimensional environment path planning strategy obtained in 602) and an intelligent avoidance decision network (i.e., the intelligent avoidance decision obtained in 603) to learn the optimal flight control strategy.

[0196] Step 606: Study a reward mechanism design method that is beneficial to network training and control, that is, study a reward mechanism design method that is beneficial to network (hierarchical deep reinforcement learning network architecture) training and control. This method can design a reasonable reward function according to different flight missions and environmental conditions to guide the learning and optimization of the deep reinforcement learning network.

[0197] Step 607: Optimize the network structure to improve the accuracy and robustness of the adaptive control of the network obtained in step 606. Optimizing the network structure includes increasing the network depth, improving the network model, introducing the attention mechanism, etc., so that the UAV can achieve high-precision and high-robustness flight control in a complex and changing environment.

[0198] Example 2

[0199] The present invention provides a method for controlling a drone. Figure 7 FIG. 1 is a flow chart of a method for controlling a drone provided by the present invention; wherein, in combination with Figure 7 As shown, the control method of the drone is described as follows:

[0200] Step 701: Acquire the current flight target and current flight environment of the UAV.

[0201] Step 702: input the current flight target and the current flight environment into a flight control network constructed by any of the above methods to obtain a flight control instruction for the UAV.

[0202] Step 703: Use the flight control instruction to control the UAV to fly to the current flight target.

[0203] In some embodiments of the present invention, the acquired current flight target and current flight environment of the UAV can be directly input into the flight control network constructed by any of the above embodiments. Here, other information such as obstacle information identified by the UAV during the flight can be further input to obtain the flight control instructions corresponding to the UAV flying to the current flight target, so as to control the UAV to fly to the current flight target based on the flight control instructions.

[0204] It should be noted that the description of the flight control network here can refer to the above-mentioned description of the method for constructing the flight control network of the drone, and the present invention will not go into details therein.

[0205] In this way, the present invention provides a control method for a drone, which outputs corresponding flight control instructions by means of a flight control network that improves the autonomy of the drone, adapts to complex environments, optimizes decision-making processes, enhances obstacle avoidance capabilities, and improves mission execution efficiency, so as to improve the adaptability and flexibility of the drone during flight.

[0206] Example 3

[0207] The present invention based on the same inventive concept also provides a construction system 800 for a flight control network of an unmanned aerial vehicle, see Figure 8 FIG. 1 is a schematic diagram of a construction of a flight control network of a drone provided by an embodiment of the present invention, and the system includes:

[0208] The first network building module 801 is used to build a posture control network of the UAV during flight based on the state information and target flight value of the UAV during flight by using a deep deterministic policy gradient algorithm;

[0209] The second network building module 802 is used to use a deep reinforcement learning algorithm to build a path planning network and an intelligent avoidance network for the UAV during flight based on the state information and the environment information of the UAV;

[0210] A network construction module 803 is used to construct a hierarchical deep reinforcement learning network of the UAV during the flight process based on the attitude control network, the path planning network and the intelligent avoidance network;

[0211] The training module 804 is used to train the hierarchical deep reinforcement learning network using the historical flight data of the UAV in different flight environments and a preset flight reward mechanism to obtain a flight control network for the UAV.

[0212] In some embodiments of the present invention, the system also includes: an establishment module, used to establish a reference model corresponding to the UAV during the flight process; correspondingly, the first network construction module 801 is specifically used to input the state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training to obtain the attitude control network.

[0213] In some embodiments of the present invention, the deep reinforcement learning algorithm includes: a double-delay deep deterministic policy gradient algorithm, a soft actor-critic algorithm, and a reinforcement learning algorithm based on a unit state sequence. The second network building module 802 is specifically used to input the state information and the distance information in the environmental information into the recursive neural network built by the double-delay deep deterministic policy gradient algorithm and the soft actor-critic algorithm for network training to obtain the path planning network; wherein the distance information at least includes: the distance between the drone and the obstacle identified by the drone during flight; the obstacle information in the state information and the environmental information is input into the neural network constructed by the reinforcement learning algorithm based on a unit state sequence for network training to obtain the intelligent avoidance network.

[0214] In some embodiments of the present invention, the network construction module 803 is specifically used to build an initial hierarchical deep reinforcement learning network infrastructure based on the state information, executable action information of the drone during flight, and corresponding reward information; the attitude control network, the path planning network, and the intelligent avoidance network are sequentially used as the low-level deep reinforcement learning subnetwork, the middle-level deep reinforcement learning subnetwork, and the high-level deep reinforcement learning subnetwork in the infrastructure to obtain the hierarchical deep reinforcement learning network; wherein the high-level deep reinforcement learning subnetwork can control the decision of the middle-level deep reinforcement learning subnetwork, and the middle-level deep reinforcement learning subnetwork can control the decision of the low-level deep reinforcement learning subnetwork.

[0215] In some embodiments of the present invention, the system also includes: a construction module, which is used to construct a reward signal reflecting the flight quality of the UAV based on the flight parameters of the UAV during the flight; wherein the flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters; construct a reward strategy corresponding to the UAV in different flight control stages; based on the reward signal and the reward strategy, construct the preset flight reward mechanism.

[0216] In some embodiments of the present invention, the training module 804 is specifically used to label and organize the historical flight data to form a training data set; use the preset flight reward mechanism to build the reward function of the hierarchical deep reinforcement learning network to obtain the network to be adjusted, and use the training data set to train the network to be adjusted to obtain the flight control network.

[0217] In some embodiments of the present invention, the system further includes: an optimization module, used to determine a preset network optimization strategy; and optimize the network structure of the flight control network using the preset network optimization strategy to obtain an optimized flight control network.

[0218] It should be noted that the description of the corresponding embodiment of the system for constructing the flight control network of the drone is similar to the description of the embodiment of the method for constructing the flight control network of the drone, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0219] Embodiment 4:

[0220] The present invention based on the same inventive concept also provides a control system 900 for an unmanned aerial vehicle, see Fig. 9 FIG. 1 is a schematic diagram of a control system of an unmanned aerial vehicle provided by an embodiment of the present invention, the system comprising:

[0221] The acquisition module 901 is used to acquire the current flight target and current flight environment of the UAV;

[0222] An input module 902 is used to input the current flight target and the current flight environment into the flight control network constructed by the system as described in the above embodiment to obtain the flight control instruction of the UAV;

[0223] The control module 903 is used to control the UAV to fly to the current flight target using the flight control instruction.

[0224] It should be noted that the description of the control system of the drone is similar to the description of the control method of the drone described above, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0225] Example 5

[0226] Based on the same inventive concept, Fig.10 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include at least one processor 1001, a memory 1002, a transceiver component 1003, etc. The processor 1001, the memory 1002 and the transceiver component 1003 are connected via a bus 1004; the memory 1002 can be used to store an execution program, and an exemplary execution program may include instructions; the processor 1001 is used to execute the instructions stored in the memory. The memory 1002 can also be used to store data, which can be called and / or modified when executing instructions.

[0227] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement corresponding method flows or corresponding functions, so as to implement the steps of a method for constructing a flight control network of an unmanned aerial vehicle in the above-mentioned embodiment, or the steps of a flight control method of an unmanned aerial vehicle in the above-mentioned embodiment.

[0228] Example 6

[0229] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium to implement the steps of a method for constructing a flight control network of an unmanned aerial vehicle in the above embodiment, or the steps of a flight control method of an unmanned aerial vehicle in the above embodiment.

[0230] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0231] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0232] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0233] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a flight control network of an unmanned aerial vehicle, characterized in that: The method comprises: Using a deep deterministic policy gradient algorithm, based on the state information and target flight value of the UAV during flight, an attitude control network of the UAV during flight is constructed; Using a deep reinforcement learning algorithm, based on the state information and the environmental information of the UAV, a path planning network and an intelligent avoidance network are respectively built for the UAV during flight; Based on the attitude control network, the path planning network and the intelligent avoidance network, a hierarchical deep reinforcement learning network of the UAV during the flight process is constructed; The historical flight data of the UAV in different flight environments and a preset flight reward mechanism are used to train the hierarchical deep reinforcement learning network to obtain the flight control network of the UAV.

2. The method according to claim 1, characterized in that Before building a posture control network for the UAV during flight by using a deep deterministic policy gradient algorithm based on the state information and target flight value of the UAV during flight, the method further includes: Establishing a reference model corresponding to the UAV during flight; The deep deterministic policy gradient algorithm is used to build a posture control network of the UAV during flight based on the state information and target flight value of the UAV during flight, including: The state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV are input into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training to obtain the attitude control network.

3. The method according to claim 1 or 2, characterized in that: The deep reinforcement learning algorithm includes: a double-delayed deep deterministic policy gradient algorithm, a soft actor-critic algorithm, and a reinforcement learning algorithm based on a unit state sequence. The deep reinforcement learning algorithm is used to build a path planning network and an intelligent avoidance network for the UAV during flight based on the state information and the environmental information of the UAV, including: The state information and the distance information in the environment information are input into the recursive neural network constructed by the double-delayed deep deterministic policy gradient algorithm and the soft actor-critic algorithm for network training to obtain the path planning network; wherein the distance information at least includes: the distance between the drone and the obstacle identified by the drone during flight; The state information and obstacle information in the environment information are input into the neural network constructed by the reinforcement learning algorithm based on the unit state sequence for network training to obtain the intelligent avoidance network.

4. The method according to any one of claims 1 to 3, characterized in that: The step of constructing a hierarchical deep reinforcement learning network of the UAV during the flight process based on the attitude control network, the path planning network and the intelligent avoidance network includes: Based on the state information, the executable action information of the drone during flight and the corresponding reward information, an infrastructure of an initial level deep reinforcement learning network is constructed; The posture control network, the path planning network and the intelligent avoidance network are sequentially used as the low-level deep reinforcement learning subnetwork, the middle-level deep reinforcement learning subnetwork and the high-level deep reinforcement learning subnetwork in the infrastructure to obtain the hierarchical deep reinforcement learning network; wherein the high-level deep reinforcement learning subnetwork can control the decision of the middle-level deep reinforcement learning subnetwork, and the middle-level deep reinforcement learning subnetwork can control the decision of the low-level deep reinforcement learning subnetwork.

5. The method according to claim 1, characterized in that Before the method uses the historical flight data of the UAV under different flight environments and the preset flight reward mechanism to train the hierarchical deep reinforcement learning network to obtain the flight control network of the UAV, the method further includes: Based on the flight parameters of the UAV during the flight, constructing a reward signal reflecting the flight quality of the UAV; wherein the flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters; Constructing reward strategies corresponding to different flight control stages of the UAV; Based on the reward signal and the reward strategy, the preset flight reward mechanism is constructed.

6. The method according to claim 1 or 5, characterized in that: The method uses the historical flight data of the UAV in different flight environments and a preset flight reward mechanism to train the hierarchical deep reinforcement learning network to obtain the flight control network of the UAV, including: Annotating and organizing the historical flight data to form a training data set; The preset flight reward mechanism is used to build a reward function of the hierarchical deep reinforcement learning network to obtain a network to be adjusted, and the training data set is used to train the network to be adjusted to obtain the flight control network.

7. The method according to claim 1, characterized in that After the hierarchical deep reinforcement learning network is trained using the historical flight data of the drone in different flight environments and a preset flight reward mechanism to obtain the flight control network of the drone, the method further includes: Determine the preset network optimization strategy; The preset network optimization strategy is adopted to optimize the network structure of the flight control network to obtain an optimized flight control network.

8. A system for constructing a flight control network of an unmanned aerial vehicle, characterized in that: The system comprises: A first network building module is used to build a posture control network of the UAV during flight based on the state information and target flight value of the UAV during flight by using a deep deterministic policy gradient algorithm; A second network building module is used to use a deep reinforcement learning algorithm to build a path planning network and an intelligent avoidance network for the UAV during flight based on the state information and the environment information of the UAV; A network construction module, used to construct a hierarchical deep reinforcement learning network of the UAV during the flight process based on the attitude control network, the path planning network and the intelligent avoidance network; The training module is used to train the hierarchical deep reinforcement learning network using the historical flight data of the UAV in different flight environments and a preset flight reward mechanism to obtain a flight control network of the UAV.

9. A flight control method for an unmanned aerial vehicle, characterized in that: The method comprises: Obtaining the current flight target and current flight environment of the UAV; Inputting the current flight target and the current flight environment into a flight control network constructed by the method according to any one of claims 1 to 7 to obtain a flight control instruction for the UAV; The flight control instruction is used to control the UAV to fly to the current flight target.

10. A flight control system for an unmanned aerial vehicle, characterized in that: The system comprises: An acquisition module, used to acquire the current flight target and current flight environment of the UAV; An input module, used to input the current flight target and the current flight environment into the flight control network constructed by the system according to claim 8 to obtain the flight control instructions of the UAV; The control module is used to control the UAV to fly to the current flight target using the flight control instruction.

Images