Flight control network construction method, control method and system

Abstract

This invention provides a method for constructing a flight control network, a control method, and a system. The method includes: employing a deep deterministic policy gradient algorithm to construct an attitude control network for the UAV during flight, based on the UAV's state information and target flight values; employing a deep reinforcement learning algorithm to construct a path planning network and an intelligent avoidance network for the UAV during flight, based on the state information and the UAV's environmental information; constructing a hierarchical deep reinforcement learning network for the UAV during flight based on the attitude control network, the path planning network, and the intelligent avoidance network; and training the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the UAV's flight control network. This invention can improve the adaptability and flexibility of the UAV during flight.

Description

Technical Field

[0001] This invention relates to the field of hierarchical deep learning algorithm technology, specifically to a method for constructing a flight control network, a control method, and a system. Background Technology

[0002] Flight control systems are crucial for the safe and efficient operation of aircraft. With the widespread application of drones and other technologies in both civilian and military fields, the requirements for the adaptability and flexibility of flight control systems have increased. Among related technologies, traditional flight control systems are insufficiently adaptable to unknown environments or unexpected situations. While the development of artificial intelligence technology has driven the application of deep learning algorithms in complex tasks, challenges related to real-time performance and environmental uncertainty still exist in the field of flight control. Summary of the Invention

[0003] To address the problems of existing technologies, this invention proposes a method for constructing a flight control network, a control method, and a system, aiming to improve the adaptability and flexibility of unmanned aerial vehicles (UAVs) during flight.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] On one hand, the present invention provides a method for constructing a flight control network for an unmanned aerial vehicle (UAV), the method comprising:

[0006] A deep deterministic policy gradient algorithm is used to build an attitude control network for the UAV during flight, based on the UAV's state information and target flight values.

[0007] Using deep reinforcement learning algorithms, based on the state information and the environmental information of the UAV, a path planning network and an intelligent avoidance network for the UAV during flight are respectively constructed.

[0008] Based on the attitude control network, the path planning network, and the intelligent avoidance network, a hierarchical deep reinforcement learning network for the UAV during the flight process is constructed.

[0009] The hierarchical deep reinforcement learning network is trained using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the flight control network of the UAV.

[0010] Optionally, before constructing the attitude control network for the UAV during flight based on the UAV's state information and target flight values ​​using the deep deterministic policy gradient algorithm, the method further includes:

[0011] Establish a reference model for the UAV during its flight;

[0012] The method employs a deep deterministic policy gradient algorithm to construct an attitude control network for the UAV during flight, based on the UAV's state information and target flight values. This includes:

[0013] The state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV are input into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training, thereby obtaining the attitude control network.

[0014] Optionally, the deep reinforcement learning algorithm includes: a dual-delay deep deterministic policy gradient algorithm, a soft actor-critic algorithm, and a reinforcement learning algorithm based on unit state sequences. The use of deep reinforcement learning algorithms, based on the state information and the environmental information of the UAV, constructs a path planning network and an intelligent avoidance network for the UAV during flight, respectively, including:

[0015] The state information and the distance information in the environment information are input into a recurrent neural network constructed by the dual-delay deep deterministic policy gradient algorithm and the soft actor-critic algorithm for network training to obtain the path planning network; wherein, the distance information includes at least the distance between the UAV and the obstacles identified by the UAV during flight;

[0016] The state information and obstacle information in the environment are input into the neural network constructed by the reinforcement learning algorithm based on the unit state sequence for network training to obtain the intelligent avoidance network.

[0017] Optionally, the construction of a hierarchical deep reinforcement learning network for the UAV during flight, based on the attitude control network, the path planning network, and the intelligent avoidance network, includes:

[0018] Based on the state information, the executable actions of the UAV during flight, and the corresponding reward information, the basic architecture of the initial layer deep reinforcement learning network is built.

[0019] The attitude control network, the path planning network, and the intelligent avoidance network are sequentially used as low-level deep reinforcement learning sub-networks, mid-level deep reinforcement learning sub-networks, and high-level deep reinforcement learning sub-networks in the basic architecture to obtain the hierarchical deep reinforcement learning network; wherein, the high-level deep reinforcement learning sub-network can control the decision-making of the mid-level deep reinforcement learning sub-network, and the mid-level deep reinforcement learning sub-network can control the decision-making of the low-level deep reinforcement learning sub-network.

[0020] Optionally, before training the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the flight control network of the UAV, the method further includes:

[0021] Based on the flight parameters of the UAV during flight, a reward signal reflecting the flight quality of the UAV is constructed; wherein the flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters;

[0022] Construct reward strategies for the UAV at different flight control phases;

[0023] Based on the reward signal and the reward strategy, the preset flight reward mechanism is constructed.

[0024] Optionally, the step of training the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the flight control network of the UAV includes:

[0025] The historical flight data is labeled and organized to form a training dataset;

[0026] The reward function of the hierarchical deep reinforcement learning network is constructed using the preset flight reward mechanism to obtain the network to be adjusted, and the network to be adjusted is trained using the training dataset to obtain the flight control network.

[0027] Optionally, after training the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the flight control network of the UAV, the method further includes:

[0028] Determine the preset network optimization strategy;

[0029] The flight control network is optimized by adopting the preset network optimization strategy to obtain the optimized flight control network.

[0030] Correspondingly, the present invention also provides a system for constructing a flight control network for an unmanned aerial vehicle (UAV), the system comprising:

[0031] The first network construction module is used to construct the attitude control network of the UAV during flight based on the state information and target flight values ​​of the UAV during flight using a deep deterministic policy gradient algorithm.

[0032] The second network construction module is used to construct the path planning network and the intelligent avoidance network of the UAV during flight, respectively, based on the state information and the environmental information of the UAV, using a deep reinforcement learning algorithm.

[0033] The network construction module is used to construct a hierarchical deep reinforcement learning network for the UAV during the flight process based on the attitude control network, the path planning network, and the intelligent avoidance network.

[0034] The training module is used to train the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism, so as to obtain the flight control network of the UAV.

[0035] On the other hand, the present invention provides a flight control method for an unmanned aerial vehicle (UAV), the method comprising:

[0036] Obtain the current flight target and current flight environment of the UAV;

[0037] The current flight target and the current flight environment are input into the flight control network constructed by the method described in any of the above methods to obtain the flight control commands for the UAV.

[0038] The flight control commands are used to control the UAV to fly to the current flight target.

[0039] Correspondingly, the present invention also provides a flight control system for an unmanned aerial vehicle (UAV), the system comprising:

[0040] The acquisition module is used to acquire the current flight target and current flight environment of the UAV;

[0041] The input module is used to input the current flight target and the current flight environment into the flight control network constructed by the system described above, so as to obtain the flight control commands of the UAV;

[0042] The control module is used to control the UAV to fly to the current flight target using the flight control commands.

[0043] In another aspect, the present invention also provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0044] The memory is used to store one or more programs;

[0045] When the one or more programs are executed by the at least one processor, the method for constructing a flight control network for a drone as described in any of the preceding claims, and the flight control method for the drone as described above, are implemented.

[0046] In another aspect, the present invention also provides a readable storage medium having an executable program stored thereon, which, when executed, implements the method for constructing a flight control network for a UAV as described in any of the preceding claims, and the aforementioned flight control method for a UAV.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] This invention provides a method and system for constructing a flight control network for an unmanned aerial vehicle (UAV). First, an attitude control network for the UAV based on the RM-DDPG algorithm is designed sequentially to process temporal data and optimize the UAV's attitude control strategy through multi-agent learning. A path planning network based on hierarchical deep reinforcement learning in a 3D environment is designed, capable of receiving the UAV's current state and environmental information, and planning the optimal flight path for the UAV in the 3D environment through a hierarchical decision-making process. Simultaneously, an intelligent avoidance network based on hierarchical deep reinforcement learning is designed to detect and predict potential risks and obstacles in real time during flight, and generate optimal avoidance actions based on the UAV's current state and environmental information. Second, a hierarchical deep reinforcement learning network architecture is constructed based on the attitude control network, path planning network, and intelligent avoidance network. Finally, the learning and optimization of the hierarchical deep reinforcement learning network architecture are trained and guided based on flight data of the UAV in different environments (e.g., UAV attitude state, flight trajectory, environmental information, and flight mission) and a preset flight reward mechanism conducive to network training and control. Thus, on the one hand, by enhancing the adaptability and flexibility of UAVs during flight—namely, improving flight autonomy, adapting to complex environments, optimizing decision-making processes, and strengthening obstacle avoidance capabilities—the efficiency of UAV mission execution can be increased, effectively reducing human error and lowering costs, thereby improving the reliability and economic benefits of UAV-based inspections. On the other hand, the flight control network built in this solution also possesses continuous learning and improvement capabilities, enabling it to respond to emergencies and promote technological innovation.

[0049] The present invention provides a control method and system for unmanned aerial vehicles (UAVs). By leveraging a flight control network that enhances the autonomy of UAVs, adapts to complex environments, optimizes decision-making processes, strengthens obstacle avoidance capabilities, and improves mission execution efficiency, the system outputs corresponding flight control commands to improve the adaptability and flexibility of UAVs during flight.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the technical solutions provided in the embodiments of the present invention. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0052] Figure 1 A flowchart illustrating a method for constructing a flight control network for an unmanned aerial vehicle (UAV) provided by the present invention.

[0053] Figure 2 A schematic diagram of a strategy-evaluation network architecture provided by the present invention;

[0054] Figure 3 A schematic diagram of the architecture of an attitude control network for an unmanned aerial vehicle (UAV) provided in an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a network architecture for a path planning network provided in an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of a network architecture for a path planning network provided in an embodiment of the present invention;

[0057] Figure 6 A schematic diagram illustrating the process of constructing a flight control strategy using a method for constructing a flight control network for an unmanned aerial vehicle (UAV) provided by this invention;

[0058] Figure 7 A flowchart illustrating a control method for an unmanned aerial vehicle (UAV) provided by the present invention;

[0059] Figure 8 A schematic diagram illustrating the composition of a flight control network construction system for an unmanned aerial vehicle (UAV) according to an embodiment of the present invention;

[0060] Figure 9 A schematic diagram illustrating the composition of a control system for an unmanned aerial vehicle (UAV) according to an embodiment of the present invention;

[0061] Figure 10 This is a schematic diagram of the composition of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0062] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0063] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0064] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0065] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0066] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0067] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0068] Flight control systems are crucial for the safe and efficient operation of aircraft. With the widespread application of drones and other unmanned aerial vehicles (UAVs) in both civilian and military fields, the requirements for the adaptability and flexibility of flight control systems have increased. Traditional flight control systems are insufficiently adaptable to unknown environments or unexpected situations. Among related technologies, the development of artificial intelligence has driven the application of deep learning algorithms in complex tasks, but flight control still faces challenges related to real-time performance and environmental uncertainty. However, using deep learning algorithms for flight control typically presents the following technical challenges: how to design an effective hierarchical structure to adapt to different flight missions and environments; how to balance information transmission and control strategy coordination between different levels; how to ensure the efficient operation of the algorithm under limited computing resources and real-time requirements; and how to achieve rapid adaptation and decision-making in unknown environments while ensuring flight safety.

[0069] Based on the above problems, this invention proposes a method for constructing a flight control network, a control method, and a system, which can improve the adaptability and flexibility of UAVs during flight.

[0070] Example 1:

[0071] This invention provides a method for constructing a flight control network for an unmanned aerial vehicle (UAV), see [link to relevant documentation]. Figure 1 The diagram shown is a flowchart illustrating a method for constructing a flight control network for an unmanned aerial vehicle (UAV) according to the present invention; wherein, combined with Figure 1 As shown, the architecture of the UAV flight control network is explained below:

[0072] Step 101: Using the deep deterministic policy gradient algorithm, an attitude control network for the UAV during flight is constructed based on the UAV's state information and target flight values.

[0073] In some embodiments of the present invention, the deep deterministic policy gradient algorithm can be a reinforcement learning-based multi-agent deep deterministic policy gradient (RM-DDPG) algorithm. Here, the attitude control network for the UAV during flight can be constructed based on the RM-DDPG algorithm. Due to the characteristics of the RM-DDPG algorithm, it can not only eliminate the steady-state attitude error of the UAV during flight and enhance the robustness of the network, but also achieve dynamic stability and fast response of the UAV during flight. Furthermore, it can optimize for overshoot phenomena that may occur in the reinforcement learning controller during target tracking.

[0074] It should be noted that drones are unmanned aerial vehicles, which can be: quadcopters, unmanned fixed-wing aircraft, etc. This invention does not limit them in any way.

[0075] In some embodiments of the present invention, the RM-DDPG algorithm can be used to build an attitude control network for the UAV during flight based on the UAV's state information during flight, the target flight value corresponding to the UAV (e.g., the position information corresponding to the object the UAV needs to capture in the air), and the environmental information of the UAV.

[0076] In some embodiments of the present invention, before performing step 101, the method for constructing the flight control network of the UAV may perform the following step A1:

[0077] Step A1: Establish a reference model for the UAV during flight.

[0078] In some embodiments of the present invention, a reference model corresponding to an unmanned aerial vehicle (UAV) control system with high stability and dynamic performance can be established.

[0079] In some embodiments of the present invention, the output state variables corresponding to the UAV during flight can be obtained first, namely: angle, angular velocity, angular acceleration and angular error integral, etc.; and then a corresponding reference model can be established based on the angle, angular acceleration, angular acceleration and angular error integral, etc.

[0080] Correspondingly, step 101 above can be implemented by following step A2:

[0081] Step A2: Input the state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training to obtain the attitude control network.

[0082] In some embodiments of the present invention, the state information of the UAV during flight, the target flight value, and the reference state information (obtained by inputting the environmental information of the UAV and the target flight value into the reference model) can be input into a policy-evaluation network architecture based on the RM-DDPG algorithm for network training, so as to obtain the attitude control network corresponding to the UAV flying to the target flight value.

[0083] Since the RM-DDPG algorithm is based on DDPG, its matching policy-evaluation network architecture (Actor-Critic network architecture), i.e., the AC network structure, can be referenced. Figure 2 As shown, this AC network structure involves: state 201, policy 202, environment 203, and value function 204. It mainly consists of two parts: an Actor network and a Critic network. The Actor and Critic networks share a common feature: taking state 201 as input, the Actor network outputs policy 202 based on state 201, selects an action, executes the action in environment 203 to enter the next state, and obtains a reward, which is then input into the value function 204. The Critic network, based on the input state 201, also inputs its evaluation value into the value function 204, causing the value function 204 to update its network parameters. After the update, it calculates the optimal value and the time difference error (TD-error). The value function 204 then uses the TD-error to update the policy corresponding to the Actor network output, i.e., policy 202.

[0084] It's important to note that the Actor network utilizes the policy gradient algorithm. It selects an action based on the current state and uses that action to interact with the environment. The goal of the Actor network is to maximize the expected cumulative reward by continuously updating the policy function to improve action selection and ensure higher returns. Actor networks are typically implemented using neural networks, with the current state as input and an action as output. The training objective is to maximize the expected cumulative reward. The Critic network, implemented using temporal difference, estimates the value function of the current policy. Its goal is to evaluate the quality of the actions selected by the Actor network, providing feedback. The Critic network, also implemented using a neural network, estimates the value function of the current policy, effectively evaluating the performance of the Actor network.

[0085] Correspondingly, in the inference phase of the Actor-Critic architecture, the Actor network and the Critic network collaborate to learn how to better interact with the environment. The Actor network continuously updates its policy function to improve its action selection, while the Critic network provides feedback to guide the Actor network's update direction. This collaboration enables the Actor-Critic algorithm, corresponding to the Actor-Critic network architecture, to learn and adapt to changes in the environment more effectively.

[0086] It should be noted that because the RM-DDPG algorithm is based on the Deep Deterministic Policy Gradient (DDPG) algorithm, it uses a deterministic policy gradient method to optimize the Actor function. The core of this algorithm is to use the action function π... u The parameter u in the formula represents the policy parameter. The policy parameter u is continuously optimized to approximate the optimal policy, as shown in formula (1).

[0087]

[0088] Among them, the policy parameter u can be continuously optimized along the direction of the gradient of formula (1) to approximate the optimal policy. Let ρ be the gradient of the Actor network. π (s) represents the state distribution. To obtain the gradient of the policy parameters, π μ (a|s) represents the distribution of actions. To obtain the gradient of the action, Let π be the state-action value function. aLet E be the policy, E be the expected value of the probability that the agent will take an action according to the policy in different states, s be the state parameter, a be the action parameter, and A and ρ be parameters representing the policy.

[0089] Among them, based on the optimization of the Actor-Critic network architecture using the deterministic policy gradient method, formula (1) defines the state-action value function. The core of this approach lies in the deterministic policy's fixed action selection given a state s and parameters θ. This means that once the initial state is determined, the behavioral trajectory becomes unique. Therefore, choosing an appropriate action-value function is crucial. The Actor-Critic algorithm can be used here, where the Actor network generates the action policy and the Critic network evaluates the policy's effectiveness. This is achieved by introducing an alternative state-action value function for the parameter ω to be approximated. To approximate the original state-action value function Correspondingly, the parameters of the Actor network are optimized and updated based on the gradient of the deterministic policy. The sampling process of the Actor network is that the state s obtains the policy π through the parameters θ. μ (s), the sampling process can incorporate a certain amount of noise to enhance the exploration (giving it a certain exploration capability), and the corresponding reward parameter is Q. ω (s t ,a t The value is determined by the Critic network, and the corresponding update process can be referred to as formula (2):

[0090]

[0091] Among them, a θ Let θ be the learning rate. t+1 and θ t These are the policy parameters at time t+1 and time t, respectively. Let be the gradient of the policy with respect to state s at time t. Let s be the gradient of the action value function at time t with respect to action a. t a t These represent the state and action at time t, respectively.

[0092] It should be noted that the Actor-Critic algorithm typically uses a neural network to replace the Actor and Critic functions. In some embodiments of this invention, to improve the efficiency and stability of network training, empirical data replay can be applied and a target network can be designed. For example, a replay buffer D of size M can be designed to store empirical data e. When the buffer is full, the oldest data is discarded. In addition, the Actor and Critic networks are updated by random sampling to reduce data correlation. Here, the introduction of a target network can improve stability through soft updates. The Critic target network is updated by resampling empirical data and minimizing the loss function. Steady-state errors can be effectively eliminated by using the integral of the angular velocity error as an extended state in the controller. During the control of the UAV, excessive control variables may cause the UAV to roll over, thus violating the principles of physical motion. Therefore, a reward function can be set in the network. The setting of this reward function is key to solving this problem. For complex tasks, state variables such as angle, angular velocity, and angular acceleration need to be considered to achieve a balance between the UAV's target tracking and the stability of the UAV's control system.

[0093] Based on the above description, an attitude control network for the UAV during flight can be built using the RM-DDPG algorithm, such as... Figure 3 The diagram shown is a schematic of the architecture of an attitude control network for a drone provided in an embodiment of the present invention; wherein, the input of the Actor network, i.e., 304, is the drone ( Figure 3 The Actor network (model 302) simulates the state vector of the UAV and the output state vector of the reference model 301. The purpose of this Actor network is to simulate a high-performance and sufficiently stable actor function. The Actor network has two fully connected hidden layers, each a 128-dimensional network with a sigmoid activation function. The output of the Actor network is the motor control signal. To ensure the control signal output remains within a reasonable range, the sigmoid function is also applied to the output layer. Meanwhile, the Critic network (model 305) takes the UAV (UAV) as its input. Figure 3 The Critic network (304) generates control inputs from the drone model (301), the reference model (301), and the Actor network (304), with hidden layers and activation functions similar to the Actor network. Furthermore, to train the Actor network faster and better, the Critic's output layer is replaced with a linear activation function. Here, the Reward is the sum of all state errors (error), which is fed back to the system. This processing allows the Actor network to avoid the system's steady-state errors when outputting. These designs enable the control signal to quickly and stably track the reference model while preventing overshoot and steady-state errors.

[0094] After performing the above processing, the control signal can quickly and stably catch up with the output of the reference model 301. This avoids system overshoot caused by excessively large control inputs leading to excessively fast target angle tracking, and effectively compensates for steady-state errors. Here, it is also possible to... Figure 3 The diagram shows a cloned Critic network as the Critic target network, i.e., 306. The Actor network then outputs control signals to the UAV model 302, and the reference model 301 also outputs a target value 303. Here, the states of the UAV model 302 and the reference model 301 are processed to obtain the sum of their corresponding errors, which is then used as a reward, i.e., 307, and fed back to the Critic target network 306. Simultaneously, for each training step, the state of the trained model is stored in a memory cache, and training begins when a sufficient number of samples are available. Here, the TD error can be calculated using the outputs of the Critic target network (306) and the Critic network (305) as the loss function, i.e., 308. Simultaneously, the Actor network (304) and the Critic network (305) are trained and adjusted based on this loss function 308.

[0095] It should be noted that deterministic policies, compared to stochastic policies, have significantly reduced degrees of freedom and exploration range. This is because deterministic policies output deterministic actions, lacking exploration of the environment. During the training phase, noise can be added to the actions output by the Actor network to give it some exploration capability; that is, Gaussian noise can be added to construct the exploration policy. This improves the degrees of freedom and scope of exploration in the Actor-Critic network architecture.

[0096] In some embodiments of the present invention, attitude state information (such as roll angle, pitch angle, and yaw angle) is an important indicator reflecting the current flight state of the UAV in its flight control. This attitude state information is influenced not only by the current control commands but also by the UAV's previous flight state. Therefore, it exhibits significant temporal characteristics. To accurately capture the dynamic changes in the UAV's attitude, the cyclic connection structure within the policy-evaluation network architecture matched by the RM-DDPG algorithm can be utilized to memorize and process this temporal data. Specifically, through cyclic connections, the output of the previous time step is used as one of the inputs to the current time step, thereby achieving effective processing of temporal information.

[0097] By processing the time-series data of drones, it is possible to more accurately capture the dynamic changes in the drone's attitude and predict its future attitude state. This predictive capability is crucial for drone flight control, as it helps to perceive the drone's flight trend in advance, thereby making more precise control decisions.

[0098] Meanwhile, in the RM-DDPG algorithm architecture, the network model is combined with the DDPG algorithm to form a complete UAV attitude control strategy. It can not only make control decisions based on current environmental information and reward signals, but also utilize past flight experience to guide current and future control behaviors through memory capabilities, thereby achieving high-precision control of the UAV's attitude.

[0099] Thus, by integrating the policy-evaluation network architecture matched by the RM-DDPG algorithm with the reference model, an attitude control network is obtained. This attitude control network can not only quickly balance the rationality between target tracking and control input of the UAV, thereby avoiding excessive control input due to overemphasis on response speed, but also effectively cope with external disturbances and uncertainties in the environment to achieve rapid response to target flight values, while effectively reducing overshoot and steady-state errors, thereby improving the flight control performance of the UAV.

[0100] Step 102: Using a deep reinforcement learning algorithm, based on the state information and the environmental information of the UAV, construct the path planning network and intelligent avoidance network of the UAV during flight.

[0101] In some embodiments of the present invention, the deep reinforcement learning algorithm includes at least: the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, the Soft Actor-Critic (SAC) algorithm, and a reinforcement learning algorithm based on Unit State Sequence (USS). Correspondingly, suitable algorithms can be selected, and based on parameters such as the UAV's state information, a path planning network and an intelligent avoidance network for the UAV during flight can be constructed respectively.

[0102] As described above, step 102 can be implemented through the following steps 1021 and 1022 (not shown in the figure):

[0103] Step 1021: Input the state information and the distance information in the environment information into the recurrent neural network built by the dual-delay deep deterministic policy gradient algorithm and the soft actor-critic algorithm to train the network and obtain the path planning network.

[0104] The distance information includes at least the distance between the drone and the obstacles it identifies during flight.

[0105] In some embodiments of the present invention, based on the state information of the UAV during flight and the environmental information it is in, a hierarchical deep reinforcement learning algorithm can be used to plan the optimal flight path of the UAV in the three-dimensional environment through a hierarchical decision-making process, so as to build a path planning network.

[0106] In some embodiments of the present invention, a novel hierarchical deep reinforcement learning-based 3D mapless navigation strategy network for unmanned aerial vehicles (UAVs), namely a path planning network, can be constructed. This involves proposing a simple learning strategy without using image-based sensing methods, training the learning agent of the relevant network using only a small amount of sparse range data from a distance sensor. This novel hierarchical deep reinforcement learning-based approach can be built upon two state-of-the-art dual-critic deep RL models: the TD3 algorithm and the SAC algorithm, both of which outperform DDPG-based and obstacle avoidance (BugAlgorithms2, BUG2) algorithms. Furthermore, a novel deep reinforcement learning (RL) architecture based on a recurrent neural network (RNN) also outperforms current architectures used for mapless navigation of mobile robots.

[0107] Here, the path planning network can learn its spatial starting position to navigate to the target position and establish its own motion plan in an obstacle-prone environment through a series of target points. Only two distance sensor data points and the UAV's relative positioning data are needed to perform map-free navigation and obstacle avoidance, and the corresponding motion equations are shown in equation (3):

[0108] v t =f(x) t ,p t ,v t-1 ) formula (3);

[0109] Among them, v t x is the speed of the drone at time t. t p is the raw data of the sensor readings. t It refers to relative position and angle, v t-1 is the velocity of the drone at time t in the previous time step. This path planning network allows us to determine the actions the drone can take, given its current state s. t The corresponding actions are all reflected in the path planning network, whose expected outcome is the action taken in the current state.

[0110] In some embodiments of the present invention, the network architecture Q of the path planning network may have a structure with 26 inputs and 3 outputs. For example... Figure 4The diagram shows the network architecture of the path planning network; of the 26 inputs, 20 represent distance detection from the simulated lidar, 3 represent previous actions of the UAV, and the other 3 represent information about the flight target. Figure 4 As shown, the lidar can provide 1080 samples within a 270° range, with 20 samples sampled at 13.5° equidistant intervals. The information used for the flight target value is the relative distance between the UAV and the flight target value, as well as the two relative angles between them. The output of the network architecture Q provides the action for a given step size, determined based on three outputs: the UAV's linear velocity, altitude velocity, and yaw angle changes. These actions are then applied to the UAV's path planning.

[0111] It should be noted that the network architecture of this path planning network can also be an Actor-Critic network architecture; for reference... Figure 5 As shown, 501 represents a portion of the Actor network layers, and 502 represents a portion of the Critic network layers. Here, the input to the Actor network is the state s of the UAV at time t. t It consists of, in sequence, a Long Short-Term Memory (LSTM) network layer and two linear layers; correspondingly, the LSTM layer has a ReLU activation function, and the two linear layers have a ReLU activation function and a hyperbolic tangent function (Tanh), respectively. Meanwhile, the input to the Critic network is the UAV's state s at time t. t and action a t It consists of four linear layers, each with a ReLU activation function.

[0112] Here, the LSTM layer can be a 32-unit LSTM layer used for activation of the Actor network using ReLU functionality. Correspondingly, the hyperbolic tangent function Tanh is used as the activation function in the Actor network output. The linear velocity values ​​of the linear layers in the Actor network are scaled between 0 and 0.25 m / s, and the delta yaw is between -2.5 and 2.5 rad / s. The height velocity is -0.25 and 0.25 m / s. The Q-values ​​of the current state and the operations performed by the agent are given in the Critic network. Furthermore, for SAC-based methods, the structure of the Critic network is the same as that of the Critic network.

[0113] In this way, based on the state information of the UAV during flight and the environmental information it is in, a path planning network is built using a hierarchical deep reinforcement learning algorithm. This network can comprehensively consider various environmental factors such as wind speed, wind direction, and terrain during the flight of the UAV, and plan the optimal flight path of the UAV in the three-dimensional environment through a hierarchical decision-making process.

[0114] Step 1022: Input the state information and obstacle information in the environment information into the neural network constructed by the reinforcement learning algorithm based on the unit state sequence for network training to obtain the intelligent avoidance network.

[0115] In some embodiments of the present invention, the USS-based reinforcement learning algorithm can be a USS-based hierarchical deep reinforcement learning algorithm. Thus, the UAV's state information and obstacle information from the environment can be input into a neural network constructed using the USS-based hierarchical deep reinforcement learning algorithm for network training, thereby obtaining an intelligent avoidance network for the UAV during flight.

[0116] It should be noted that reinforcement learning, especially hierarchical deep reinforcement learning, can effectively cope with large-scale state spaces and complex tasks by decomposing tasks, multi-level learning and decision-making, so as to improve learning efficiency and the generalization and robustness of the algorithm.

[0117] Here, the hierarchical deep reinforcement learning algorithm based on USS needs to perform the following operations in sequence: feature encoding, distance priority sorting and USS construction, and extraction of hidden features; the specific execution logic is as follows:

[0118] First, obstacle information and UAV state information are encoded one-to-one to form feature units. These feature units must be able to fully express the UAV state information and the characteristics of obstacles.

[0119] Secondly, all encoded feature units are sorted according to the distance priority between the UAV and obstacles, and then combined into a USS (Units of Targets). This sorting process helps the UAV prioritize the closest and most pressing obstacles during obstacle avoidance. Simultaneously, the sorted feature units are combined into a sequence, i.e., the USS. This sequence will serve as input to a deep reinforcement learning algorithm to reflect the UAV's surrounding environment state at a specific moment.

[0120] Then, a Gated Recurrent Unit (GRU) is used to fuse features from the feature units in the USS, capturing dynamic changes and extracting hidden information (hidden features) as the state information at that moment. This state information is then input into the corresponding reinforcement learning neural network. Here, hidden feature extraction is the core step. Furthermore, CNNs can be used for deep feature (i.e., hidden features) extraction to understand the environmental structure. In practical applications, the policy iteration and end-to-end training optimization extraction processes in the USS-based hierarchical deep reinforcement learning algorithm can omit manual feature engineering. This achieves multimodal fusion to enhance environmental perception, combining obstacle avoidance and navigation to improve task execution capabilities. Simultaneously, the design of the neural network built using the USS-based hierarchical deep reinforcement learning algorithm emphasizes real-time performance and generalization, ensuring effective obstacle avoidance for the UAV in complex environments.

[0121] In some embodiments of the present invention, the USS can be further combined with DDQN and DDPG algorithms to improve the training effect of UAVs in continuous obstacle avoidance scenarios. Specifically, the reinforcement learning neural network mentioned above can be applied to the Double Deep Q Network (DDQN) algorithm and the DDPG algorithm, and trained on a UAV obstacle avoidance simulation platform. Exemplarily, in the constructed intelligent avoidance network, environmental observation data (UAV state information and obstacle information) is first converted into USS. Subsequently, the GRU processes the USS to fuse and refine features, generating fixed-length feature vectors. These feature vectors can be directly input into the reinforcement learning model. Here, the DDQN algorithm can be further combined to enable the GRU module to extract features from the USS and update them along with the evaluation network parameters. During the update process, gradients are fed back to the GRU module to achieve synchronous parameter optimization.

[0122] Here, the network structure of the USS-based DDPG algorithm can contain two GRU units, serving the policy network and the value network respectively, to process feature data. During the network parameter update phase, gradients are passed to the corresponding GRU units to ensure synchronous optimization of the feature extraction process.

[0123] It's important to note that during drone flight, multiple detection devices are used to detect targets or obstacles. When facing multiple obstacles, the number of environmental entities changes dynamically. Traditional neural networks require inputting environmental information based on the maximum number of entities, but not all information influences decision-making. Therefore, effectively acquiring environmental situational features and filtering important state information is crucial for improving training performance. Here, the USS encoding method can borrow from word vector encoding in natural language processing to map the drone and targets or obstacles into state units, and encode these state units into vectors of equal length to distinguish differences. A GRU network is then used to extract hidden features from the USS, making it suitable for single-entity scenarios, such as drones facing multiple targets, i.e., scenarios with multiple (variable) obstacles.

[0124] In this way, based on obstacle information from state and environmental information, a neural network constructed using a reinforcement learning algorithm based on unit state sequences is trained to obtain an intelligent avoidance network. This network can not only detect and predict potential risks and obstacles in real time during UAV flight, but also generate optimal avoidance actions based on the UAV's current state and environmental information. In other words, this intelligent avoidance network can use deep learning networks to detect obstacles in real time and provide the UAV with the optimal avoidance path through calculation and analysis. This ensures that the UAV can react quickly and accurately when encountering obstacles, avoiding collisions.

[0125] It's important to note that the intelligent avoidance decision-making mechanism corresponding to the intelligent avoidance network is a crucial component of the UAV flight control system. It cleverly combines real-time sensor data from the UAV, detailed map information, and advanced predictive models to ensure the UAV can quickly perceive and accurately avoid potential risks during flight. Real-time sensor data is key to the UAV's perception of its surroundings. This data includes measurements from devices such as LiDAR, infrared cameras, and ultrasonic sensors, reflecting the position, shape, and speed of obstacles around the UAV in real time. The intelligent avoidance decision-making mechanism processes this real-time sensor data, providing an immediate assessment of risks around the UAV. In addition to real-time sensor data, map information also provides important reference for the intelligent avoidance decision-making mechanism. Map information includes pre-loaded static obstacle information (such as buildings and mountains), terrain height data, and flight restriction areas. This information provides the UAV with a macroscopic environmental view, helping it avoid collisions with known obstacles and choose safer flight paths. Furthermore, predictive models play a crucial role in intelligent risk avoidance decision-making mechanisms. Based on historical data, current environmental information, and the drone's flight status, they can predict potential risks over a future period. These predictions provide early warnings to the drone, giving it sufficient time to react and thus avoid potential risks.

[0126] Through the above steps, the path planning network built based on the hierarchical deep reinforcement learning algorithm can receive the current state information and environmental information of the UAV, and plan the optimal flight path of the UAV in the three-dimensional environment through a hierarchical decision-making process; at the same time, the intelligent avoidance network based on hierarchical deep reinforcement learning enables the UAV to detect and predict potential risks and obstacles in real time during flight, and generate the optimal avoidance action according to the current state and environmental information of the UAV.

[0127] Step 103: Based on the attitude control network, the path planning network, and the intelligent avoidance network, construct a hierarchical deep reinforcement learning network for the UAV during the flight process.

[0128] In some embodiments of the present invention, a hierarchical deep reinforcement learning network structure can be constructed by integrating attitude control network, path planning network, and intelligent avoidance network. Here, a basic network architecture can be built first, and then the attitude control network, path planning network, and intelligent avoidance network can be used as policy networks in the basic network architecture to build the network, thereby obtaining the hierarchical deep reinforcement learning network of the UAV during flight.

[0129] In some embodiments of the present invention, step 103 can be implemented by the following steps 1031 and 1032 (not shown in the figure):

[0130] Step 1031: Based on the state information, the executable action information of the UAV during flight, and the corresponding reward information, build the basic architecture of the initial layer deep reinforcement learning network.

[0131] In some embodiments of the present invention, the basic architecture of an initial-level deep reinforcement learning network can be built based on the state information, executable action information, and corresponding reward information of the UAV during flight. This basic architecture can be constructed with the following basic parameters:

[0132] 1. State space: can be used to define the state of a UAV in a three-dimensional environment, including: the UAV's position, speed, attitude, environmental information, etc. during flight.

[0133] 2. Action Space: This can be used to define the actions that the drone can perform, including attitude adjustment, turning and speed control during flight.

[0134] 3. Reward function: can be used to describe the behavior of drones in the environment and design reward mechanisms to guide the learning and optimization of the initial level deep reinforcement learning network.

[0135] Step 1032: Sequentially use the attitude control network, the path planning network, and the intelligent avoidance network as the low-level deep reinforcement learning sub-network, the mid-level deep reinforcement learning sub-network, and the high-level deep reinforcement learning sub-network in the basic architecture to obtain the hierarchical deep reinforcement learning network.

[0136] The high-level deep reinforcement learning subnetwork can control the decision-making of the mid-level deep reinforcement learning subnetwork, and the mid-level deep reinforcement learning subnetwork can control the decision-making of the low-level deep reinforcement learning subnetwork.

[0137] In some embodiments of the present invention, the attitude control network can be used as a low-level deep reinforcement learning sub-network (Sub-Policy) in the infrastructure, the path planning network can be used as a mid-level deep reinforcement learning sub-network (Mid-level Policy) in the infrastructure, and the intelligent avoidance network can be used as a high-level deep reinforcement learning sub-network (Meta-Policy) in the infrastructure, thereby obtaining a hierarchical deep reinforcement learning network.

[0138] It should be noted that the path planning network utilizes hierarchical deep reinforcement learning techniques to design path planning for a 3D environment. It receives the UAV's current state and environmental information to output path planning commands. The intelligent avoidance network generates avoidance commands when the UAV encounters obstacles during flight, guiding it to safely navigate around them. Correspondingly, the attitude control network uses the RM-DDPG algorithm, combined with RNN processing of temporal data, to achieve precise attitude control of the UAV. It can accept commands from mid-level and high-level policy networks, such as the path planning and avoidance commands output by the path planning and intelligent avoidance networks respectively, and outputs specific attitude control actions.

[0139] Here, the low-level, mid-level, and high-level deep reinforcement learning subnetworks in the hierarchical deep reinforcement learning network can communicate with each other through an effective information exchange mechanism. The mid-level and high-level policy networks pass the output path planning and avoidance instructions to the low-level policy networks, while the low-level policy networks feed back the output attitude control states to the mid-level and high-level policy networks.

[0140] Here, the low-level, mid-level, and high-level policy networks within a hierarchical deep reinforcement learning network can be trained jointly to optimize the overall performance of the network. Furthermore, by sharing reward signals and state information, the three policy networks can collaborate, achieving more efficient learning.

[0141] Thus, a hierarchical deep reinforcement learning algorithm is introduced, which realizes a hierarchical decision-making process from low-level attitude control to mid-level path planning and high-level intelligent avoidance decision-making through a unique network architecture. The core of this hierarchical deep reinforcement learning algorithm lies in its multi-layered network structure. First, the low-level policy network focuses on the attitude control of the UAV, receiving data from sensors deployed on the UAV, such as data collected by gyroscopes, accelerometers, and GPS, to perceive the current attitude information of the UAV in real time. Through processing and analysis of this information, the lowest-level policy network can generate precise attitude control commands to ensure that the UAV executes flight maneuvers stably and accurately. Then, the mid-level policy network focuses on path planning, using deep reinforcement learning techniques to learn the optimal path planning strategy based on the target position and current environmental information provided by the high-level policy network. Furthermore, the mid-level policy network can consider various complex environmental factors, such as terrain, obstacles, and wind direction, to plan a safe and efficient flight path. Finally, the high-level policy network is responsible for intelligent avoidance decisions. Based on the path planning provided by the mid-level policy network and the real-time flight status feedback from the lower-level policy network, it can make quick and accurate judgments about potential obstacles or emergencies. In addition, through deep reinforcement learning, the high-level policy network can learn how to intelligently avoid risks in various complex scenarios, ensuring the flight safety of the UAV. Thus, the hierarchical decision-making process not only enhances the autonomy and intelligence of UAV flight but also greatly improves its ability to cope with complex environments and emergencies. Through the collaborative work of various sub-networks, the UAV can autonomously complete the entire flight process from takeoff, cruise to landing without human intervention, and make quick and accurate decisions when necessary to adapt to complex and ever-changing flight environments.

[0142] Step 104: Using the historical flight data of the UAV under different flight environments and the preset flight reward mechanism, train the hierarchical deep reinforcement learning network to obtain the flight control network of the UAV.

[0143] In some embodiments of the present invention, historical flight data and a preset flight reward mechanism can be collected first, and then a hierarchical deep reinforcement learning network can be trained and constructed based on the obtained historical flight data and the preset flight reward mechanism to obtain the flight control network of the UAV.

[0144] Here, the process of constructing the preset flight reward mechanism can be referenced from steps B1 to B3 as follows:

[0145] Step B1: Based on the flight parameters of the UAV during flight, construct a reward signal that reflects the flight quality of the UAV.

[0146] The flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters;

[0147] In some embodiments of the present invention, reward signals reflecting the flight quality of the UAV can be constructed based on flight parameters of the UAV during flight, such as flight stability parameters, flight path tracking parameters, and flight energy consumption parameters. Here, reward signals can be used to reflect the core objectives of flight control. Correspondingly, appropriate weights can be assigned to the reward signals corresponding to these flight parameters to balance their importance in the overall flight control strategy.

[0148] Step B2: Construct the reward strategy for the UAV in different flight control phases.

[0149] In some embodiments of the present invention, the reward strategy for the UAV corresponds to different flight control phases, wherein the different flight control phases include at least: takeoff, landing, and rotation. Here, the reward strategy can be multi-objective and multi-stage reward. In order to take into account the diversity of control tasks, the research will explore how to apply different reward strategies in different control phases and realize dynamic adjustment of rewards, while avoiding potential conflicts between different reward items, and ensuring that the network can learn the globally optimal control strategy.

[0150] Step B3: Based on the reward signal and the reward strategy, construct the preset flight reward mechanism.

[0151] In some embodiments of the present invention, a preset flight reward mechanism can be constructed based on the reward signal and reward strategy. Correspondingly, a reward function corresponding to the preset flight reward mechanism can be built so that the relevant network can be trained and adjusted in the future.

[0152] It should be noted that the preset flight reward mechanism here may be the same as or different from the reward mechanism involved in the "Reward Mechanism for Describing the Behavior of UAVs in the Environment" mentioned above, and this invention does not impose any limitations on it.

[0153] It is important to note that the significance of a pre-set flight reward mechanism lies in its ability to flexibly design reward functions with clear guidance based on different flight mission requirements. This reward function not only provides guidance signals for the UAV during flight but also ensures that the UAV can achieve efficient and safe flight in complex environments.

[0154] In this way, by leveraging the reward signals and reward strategies corresponding to the drone during flight, a preset flight reward mechanism can be established, enabling the subsequent guidance network to learn more effective and reasonable flight strategies during training.

[0155] In some embodiments of the present invention, step 104 above can be implemented by the following steps 1041 and 1042 (not shown in the figure):

[0156] Step 1041: Label and organize the historical flight data to form a training dataset.

[0157] In some embodiments of the present invention, the construction process of building a training dataset based on historical flight data includes three stages: data collection, annotation, and organization.

[0158] The historical flight data collection process is as follows: The UAV is test-flown using a flight platform under diverse environmental conditions, including different weather conditions (e.g., sunny, cloudy, rainy, windy), different time periods (day and night), and different terrain features (e.g., plains, mountains, different altitudes), to ensure sufficient flight data samples are collected in each typical environmental combination. Simultaneously, flight data can be collected under complex conditions such as sensor malfunctions, airflow disturbances, and abnormal attitude by artificially creating interference, in order to obtain historical flight data.

[0159] In some embodiments of the present invention, historical flight data is annotated, such as by performing detailed annotations on the collected historical flight data to obtain annotation information, such as environmental conditions, aircraft attitude, and human interference. This process can be initially annotated using automated algorithms, and can then be manually reviewed and corrected.

[0160] In addition, experts in the field of flight control can be invited to participate in the quality assessment of historical flight data and the correction of mislabeling.

[0161] In some embodiments of the present invention, organizing historical flight data can be achieved by processing and organizing labeled historical flight data according to the input and output requirements of the control network to form a standardized training dataset. Here, since the input data of the subsequent networks includes flight state characteristics and the output data is the desired control commands, the labeled and organized historical flight data should include normal flight data and hardware failure data under complex environments, with failure conditions specifically labeled to test the robustness of the control commands output by the subsequent networks.

[0162] Here, historical flight data is labeled and organized to form a comprehensive and diverse training dataset. Because this training dataset covers data corresponding to flight conditions in both normal and complex environments, it can effectively train the reinforcement learning controller, i.e., the relevant network structure that needs to be trained subsequently.

[0163] Step 1042: Construct the reward function of the hierarchical deep reinforcement learning network using the preset flight reward mechanism to obtain the network to be adjusted, and train the network to be adjusted using the training dataset to obtain the flight control network.

[0164] In some embodiments of the present invention, firstly, a reward function for a hierarchical deep reinforcement learning network is constructed based on a preset flight reward mechanism, thereby obtaining a new network, namely the network to be adjusted; then, the network to be adjusted is trained using a training dataset to obtain a flight control network that outputs the optimal flight control strategy. Here, the network to be adjusted also includes: an attitude control network, a path planning network, and an intelligent avoidance network.

[0165] In some embodiments of the present invention, the constructed training dataset can be loaded into the framework corresponding to the network to be adjusted for network training, and regularization techniques can be used to prevent overfitting. Furthermore, the construction of the training dataset mentioned above facilitates subsequent incremental updates. By continuously collecting and labeling new flight data, the dataset can be continuously enriched, and the generalization ability of the flight control network can be optimized.

[0166] It should be noted that during training, in order to avoid the instability of the control process caused by abrupt changes in the reward signal, the reward function of the hierarchical deep reinforcement learning network built by the preset flight reward mechanism can be smoothed. Furthermore, a time delay mechanism can be introduced so that the network to be adjusted can associate the current action with the future flight state, thereby achieving predictive control.

[0167] In some embodiments of the present invention, an adaptive reward adjustment mechanism may be further incorporated into the training phase of the network to be adjusted. This involves investigating how to automatically adjust the reward function based on the network's current performance level during training to promote the adaptability and efficiency of the training process. Simultaneously, methods to prevent the network from becoming dependent on specific rewards need to be explored. Furthermore, to further improve the real-time performance and adaptability of the control strategy, an online reward generation module will be designed to dynamically adjust rewards based on real-time execution results, guiding the network to make better control decisions.

[0168] It should be noted that the reward function for building a hierarchical deep reinforcement learning network using a pre-defined flight reward mechanism requires consideration of several aspects during the construction process:

[0169] First, it is essential to thoroughly understand the specific requirements of the flight mission, including parameters such as flight path, flight speed, flight altitude, and flight attitude. These parameters form the basis for designing the reward function and determine which performance aspects of the UAV need to be monitored during flight.

[0170] Secondly, the reward function needs to be clearly directional. It should clearly reflect the drone's behavioral preferences during task completion, enabling the drone to spontaneously choose more efficient and safer flight strategies. For example, in obstacle avoidance tasks, the reward function can be designed to give higher rewards for successful obstacle avoidance and punish collisions with obstacles.

[0171] Furthermore, the reward function needs to consider the characteristics of complex environments. During actual flight, drones may encounter various uncertainties and interference factors, such as weather changes, sensor noise, and obstacle movement. Therefore, the reward function should possess a certain degree of robustness and adaptability to cope with the challenges of these complex environments.

[0172] Finally, the design of the reward function also needs to consider the convergence and stability of the algorithm. This is because a well-designed reward function should ensure that the drone can quickly converge to the optimal policy during training and maintain stable performance in actual flight.

[0173] In this way, the rich training dataset not only provides abundant learning samples for the network model, but also guides the network model to learn more effective and reasonable flight strategies during the training process based on the preset flight reward mechanism. This can improve the control strategy output by the flight control network and control the UAV's adaptability and flexibility in complex environments.

[0174] Following the description above, the method for constructing a flight control network for a drone provided in this embodiment of the invention may further execute the following steps C1 and C2 after executing step 104:

[0175] Step C1: Determine the preset network optimization strategy.

[0176] In some embodiments of the present invention, in order to improve the adaptive flight control strategy of the flight control network, one can start by optimizing the network structure of the flight control network. The main research direction, namely the preset network optimization strategy, can be constructed from the following aspects, such as:

[0177] 1. Multi-sensor data fusion: A module can be developed to fuse data from different types of sensors (e.g., inertial navigation measurement data, GPS measurement data, meteorological data, etc.) to extract comprehensive flight state characteristics of the UAV, which can then be used as input to the flight control network. This can be achieved by utilizing attention mechanisms and graph neural networks for data fusion and correlation analysis, thereby improving the flight performance of the controlled UAV.

[0178] 2. Control Command Interpreter: An interpreter module is designed at the output of the flight control network to translate control commands into specific robot application programming interface (API) instructions, thereby enhancing the interpretability and accuracy of the control output. This interpreter module can also integrate flight dynamics models for simulation prediction and command optimization.

[0179] 3. Automatic search and generation of network structures: Utilize techniques such as reinforcement learning and evolutionary algorithms to automatically explore optimal network structures, or generate task-specific customized network structures based on meta-learning to improve design efficiency.

[0180] 4. Multi-task and transfer learning: Construct sub-task networks for different flight phases and different robot platforms, and improve learning efficiency by transferring knowledge through transfer learning and multi-task learning.

[0181] 5. Enhance network generalization: By employing data augmentation and network regularization techniques, the adaptability of the flight control network to different environments is improved, preventing overfitting and enabling it to cope with more unknown environments. Simultaneously, adversarial examples are used to enhance the robustness of the flight control network.

[0182] 6. Online Network Optimization: The parameter weights of the flight control network are continuously optimized during flight to adapt to changes in the environment. In case of control emergencies, the weights of key layers are quickly adjusted to achieve online self-adaptation.

[0183] 7. Performance index verification: Develop a set of quantitative evaluation indexes for the control performance of the flight control network, including smoothness error, flight path tracking error, etc., and test the robustness of the flight control network under conditions such as sensor failure.

[0184] Step C2: Using the preset network optimization strategy, optimize the network structure of the flight control network to obtain the optimized flight control network.

[0185] In some embodiments of the present invention, the preset network optimization strategy constructed above can be directly adopted to optimize the network structure of the flight control network, that is, to adjust and change it, so as to obtain an optimized flight control network, thereby improving the accuracy and robustness of the network's adaptive control.

[0186] Here, a pre-defined network optimization strategy is used to optimize the network structure of the flight control network. Alternatively, the network structure can be continuously optimized and improved during the training process of the flight control network. For example, by increasing the network depth, the system can extract more abstract and high-level feature information, improving the expressive power and generalization ability of the network model. Or, by improving the network architecture and introducing an attention mechanism, the system can further improve the network model's attention to and processing ability of key information, thereby improving the adaptive control accuracy and robustness of the UAV in complex environments.

[0187] In addition, the flight control network structure can be optimized by introducing advanced neural network structures, and / or by adopting efficient optimization algorithms, and / or by utilizing large-scale distributed computing resources, so as to improve the training efficiency and performance of the network model.

[0188] In this way, by optimizing the network structure of the flight control network, the accuracy and robustness of the adaptive control of the flight control network can be improved, thereby enabling UAVs to achieve high-precision and high-robustness flight control in complex and ever-changing environments.

[0189] This invention provides a method for constructing a flight control network for a drone. First, an attitude control network for the drone based on the RM-DDPG algorithm is designed sequentially to process temporal data and optimize the drone's attitude control strategy through multi-agent learning. A path planning network based on hierarchical deep reinforcement learning in a 3D environment is designed, capable of receiving the drone's current state and environmental information, and planning the optimal flight path for the drone in the 3D environment through a hierarchical decision-making process. Simultaneously, an intelligent avoidance network based on hierarchical deep reinforcement learning is designed to detect and predict potential risks and obstacles in real time during the drone's flight, and generate optimal avoidance actions based on the drone's current state and environmental information. Second, a hierarchical deep reinforcement learning network architecture is constructed based on the drone's attitude control network, path planning network, and intelligent avoidance network. Finally, the hierarchical deep reinforcement learning network architecture is trained and guided to learn and optimize based on the drone's flight data in different environments (e.g., drone attitude state, flight trajectory, environmental information, and flight mission) and a preset flight reward mechanism conducive to network training and control. Thus, on the one hand, by enhancing the adaptability and flexibility of UAVs during flight—namely, improving flight autonomy, adapting to complex environments, optimizing decision-making processes, and strengthening obstacle avoidance capabilities—the efficiency of UAV mission execution can be increased, effectively reducing human error and lowering costs, thereby improving the reliability and economic benefits of UAV-based inspections. On the other hand, the flight control network built in this solution also possesses continuous learning and improvement capabilities, enabling it to respond to emergencies and promote technological innovation.

[0190] Correspondingly, such as Figure 6 The diagram shown illustrates a process for constructing a flight control strategy using a method for building a flight control network for an unmanned aerial vehicle (UAV) provided by this invention. The flowchart of this adaptive flight control method may include the following steps:

[0191] Step 601: Construct a UAV attitude control strategy based on the RM-DDPG algorithm. That is, design a UAV attitude control strategy based on the RM-DDPG algorithm, which is used to combine RNN processing of time series data and multi-agent learning to optimize the attitude control strategy of the UAV.

[0192] Step 602: A 3D environment path planning strategy based on hierarchical deep reinforcement learning, that is, designing a 3D environment path planning strategy based on hierarchical deep reinforcement learning, which can receive the current state information and environmental information of the UAV, and plan the optimal flight path of the UAV in the 3D environment through a hierarchical decision-making process.

[0193] Step 603: Intelligent avoidance decision based on hierarchical deep reinforcement learning, that is, designing an intelligent avoidance decision mechanism based on hierarchical deep reinforcement learning. This mechanism can detect and predict potential risks and obstacles in real time during the flight of the UAV, and generate the optimal avoidance action according to the current state of the UAV and environmental information.

[0194] Step 604: Construct a hierarchical deep reinforcement learning network architecture, that is, integrate UAV attitude control, 3D path planning and intelligent obstacle avoidance based on hierarchical deep reinforcement learning to construct a hierarchical deep reinforcement learning network architecture.

[0195] Step 605: Collect and label flight data under different environments to generate a training dataset. This involves collecting and labeling flight data under different environments, including the UAV's attitude state, flight trajectory, environmental information, and flight mission, to generate a training dataset. The collected training dataset can then be used to train a hierarchical deep reinforcement learning network architecture. This hierarchical deep reinforcement learning network architecture includes an RM-DDPG attitude controller network (i.e., the UAV attitude controller policy obtained in step 601), a path planning network (i.e., the 3D environment path planning policy obtained in step 602), and an intelligent avoidance decision network (i.e., the intelligent avoidance decision obtained in step 603) to learn the optimal flight control strategy.

[0196] Step 606: Research a reward mechanism design method that is beneficial to network training and control. Specifically, research a reward mechanism design method that is beneficial to the training and control of the network (hierarchical deep reinforcement learning network architecture). This method can design a reasonable reward function according to different flight missions and environmental conditions to guide the learning and optimization of the deep reinforcement learning network.

[0197] Step 607: Optimize the network structure to improve the accuracy and robustness of the adaptive control of the network obtained in step 606. Optimizing the network structure includes increasing network depth, improving the network model, and introducing an attention mechanism. This enables the UAV to achieve high-precision and highly robust flight control in complex and ever-changing environments.

[0198] Example 2

[0199] This invention provides a control method for an unmanned aerial vehicle (UAV), see [link to relevant documentation]. Figure 7 The diagram shown is a flowchart illustrating a control method for a drone provided by the present invention; wherein, in conjunction with Figure 7 The control method for the UAV is explained below:

[0200] Step 701: Obtain the current flight target and current flight environment of the UAV.

[0201] Step 702: Input the current flight target and the current flight environment into the flight control network constructed by any of the methods described above to obtain the flight control commands of the UAV.

[0202] Step 703: Use the flight control command to control the UAV to fly to the current flight target.

[0203] In some embodiments of the present invention, the current flight target and current flight environment of the UAV can be directly input into the flight control network constructed in any of the above embodiments. Here, other information such as obstacle information identified by the UAV during flight can also be input to obtain the flight control command corresponding to the UAV flying to the current flight target, thereby controlling the UAV to fly to the current flight target based on the flight control command.

[0204] It should be noted that the description of the flight control network here can be found in the above description of the construction method of the flight control network of the UAV, and will not be repeated here in this invention.

[0205] Thus, the UAV control method provided by this invention, by leveraging a flight control network that enhances the autonomy of the UAV, adapts to complex environments, optimizes the decision-making process, strengthens obstacle avoidance capabilities, and improves mission execution efficiency, outputs corresponding flight control commands to improve the adaptability and flexibility of the UAV during flight.

[0206] Example 3

[0207] Based on the same inventive concept, this invention also provides a system 800 for constructing a flight control network for unmanned aerial vehicles (UAVs), see reference. Figure 8 The diagram shown illustrates the composition of a flight control network for an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. The system includes:

[0208] The first network construction module 801 is used to construct the attitude control network of the UAV during flight based on the state information and target flight value of the UAV during flight using a deep deterministic policy gradient algorithm.

[0209] The second network construction module 802 is used to construct the path planning network and the intelligent avoidance network of the UAV during flight, respectively, based on the state information and the environmental information of the UAV, using a deep reinforcement learning algorithm.

[0210] The network construction module 803 is used to construct a hierarchical deep reinforcement learning network for the UAV during the flight process based on the attitude control network, the path planning network and the intelligent avoidance network.

[0211] Training module 804 is used to train the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism, so as to obtain the flight control network of the UAV.

[0212] In some embodiments of the present invention, the system further includes: a modeling module for establishing a reference model corresponding to the UAV during flight; correspondingly, the first network building module 801 is specifically used to input the state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training to obtain the attitude control network.

[0213] In some embodiments of the present invention, the deep reinforcement learning algorithm includes: a dual-delay deep deterministic policy gradient algorithm, a soft actor-commentator algorithm, and a reinforcement learning algorithm based on unit state sequences. The second network construction module 802 is specifically used to input the state information and the distance information in the environment information into a recurrent neural network constructed by the dual-delay deep deterministic policy gradient algorithm and the soft actor-commentator algorithm for network training to obtain the path planning network. The distance information includes at least the distance between the UAV and obstacles identified by the UAV during flight. The state information and the obstacle information in the environment information are input into the neural network constructed by the reinforcement learning algorithm based on unit state sequences for network training to obtain the intelligent avoidance network.

[0214] In some embodiments of the present invention, the network construction module 803 is specifically used to build the basic architecture of an initial hierarchical deep reinforcement learning network based on the state information, the executable action information of the UAV during flight, and the corresponding reward information; and sequentially using the attitude control network, the path planning network, and the intelligent avoidance network as the low-level deep reinforcement learning sub-network, the mid-level deep reinforcement learning sub-network, and the high-level deep reinforcement learning sub-network in the basic architecture to obtain the hierarchical deep reinforcement learning network; wherein, the high-level deep reinforcement learning sub-network can control the decision of the mid-level deep reinforcement learning sub-network, and the mid-level deep reinforcement learning sub-network can control the decision of the low-level deep reinforcement learning sub-network.

[0215] In some embodiments of the present invention, the system further includes: a construction module, configured to construct a reward signal reflecting the flight quality of the UAV based on the flight parameters of the UAV during flight; wherein the flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters; construct a reward strategy corresponding to the UAV in different flight control phases; and construct the preset flight reward mechanism based on the reward signal and the reward strategy.

[0216] In some embodiments of the present invention, the training module 804 is specifically used to label and organize the historical flight data to form a training dataset; to build the reward function of the hierarchical deep reinforcement learning network using the preset flight reward mechanism to obtain the network to be adjusted; and to train the network to be adjusted using the training dataset to obtain the flight control network.

[0217] In some embodiments of the present invention, the system further includes: an optimization module, configured to determine a preset network optimization strategy; and to optimize the network structure of the flight control network using the preset network optimization strategy to obtain an optimized flight control network.

[0218] It should be noted that the description of the system for constructing the flight control network of this UAV in the corresponding embodiment is similar to the description of the method for constructing the flight control network of the UAV described above, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0219] Example 4:

[0220] Based on the same inventive concept, the present invention also provides a control system 900 for an unmanned aerial vehicle (UAV), see reference. Figure 9 The diagram shown is a schematic representation of the composition of a control system for an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. The system includes:

[0221] The acquisition module 901 is used to acquire the current flight target and current flight environment of the UAV;

[0222] The input module 902 is used to input the current flight target and the current flight environment into the flight control network constructed by the system as described in the above embodiment, so as to obtain the flight control command of the UAV;

[0223] The control module 903 is used to control the UAV to fly to the current flight target using the flight control commands.

[0224] It should be noted that the description of the control system of this UAV in the corresponding embodiment is similar to the description of the control method embodiment of the UAV described above, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0225] Example 5

[0226] Based on the same inventive concept, such as Figure 10 As shown, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include at least one processor 1001, a memory 1002, a transceiver component 1003, etc. The processor 1001, memory 1002, and transceiver component 1003 are connected via a bus 1004; the memory 1002 can be used to store executable programs, and an exemplary executable program may include instructions; the processor 1001 is used to execute the instructions stored in the memory. The memory 1002 can also be used to store data, which can be accessed and / or modified when instructions are executed.

[0227] The processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and it is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to realize the steps of the method for constructing a flight control network for a UAV in the above embodiments, or the steps of the flight control method for a UAV in the above embodiments.

[0228] Example 6

[0229] Based on the same inventive concept, this invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory). This readable storage medium is a memory device within an electronic device used to store programs and data. It is understood that the storage medium here can include both built-in storage media within the electronic device and extended storage media supported by the electronic device. The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor loads and executes one or more instructions stored in the storage medium to implement the steps of the method for constructing a flight control network for a UAV in the above embodiments, or the steps of the flight control method for a UAV in the above embodiments.

[0230] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0231] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0232] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0233] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a flight control network for an unmanned aerial vehicle (UAV), characterized in that, The method includes: A deep deterministic policy gradient algorithm is used to build an attitude control network for the UAV during flight, based on the UAV's state information and target flight values. The state information and the distance information from the environment in which the UAV is located are input into a recurrent neural network built with a dual-delay deep deterministic policy gradient algorithm and a soft actor-commentator algorithm to train the network and obtain a path planning network. The state information and obstacle information in the environment information are input into a neural network constructed by a reinforcement learning algorithm based on unit state sequences for network training to obtain an intelligent avoidance network. Based on the state information, the executable action information of the UAV during flight, and the corresponding reward information, the basic architecture of the initial hierarchical deep reinforcement learning network is constructed; the attitude control network, the path planning network, and the intelligent avoidance network are sequentially used as the low-level deep reinforcement learning sub-network, the mid-level deep reinforcement learning sub-network, and the high-level deep reinforcement learning sub-network in the basic architecture to obtain the hierarchical deep reinforcement learning network of the UAV during flight. Using the historical flight data of the UAV under different flight environments and a preset flight reward mechanism, the hierarchical deep reinforcement learning network is trained to obtain the flight control network of the UAV; The distance information includes at least the distance between the UAV and obstacles identified by the UAV during flight; the high-level deep reinforcement learning subnetwork can control the decision-making of the mid-level deep reinforcement learning subnetwork, and the mid-level deep reinforcement learning subnetwork can control the decision-making of the low-level deep reinforcement learning subnetwork; the execution process of the reinforcement learning algorithm based on unit state sequence is as follows: the obstacle information and the UAV state information are encoded one-to-one to form feature units; all encoded feature units are sorted and combined into a unit state sequence according to the distance priority between the UAV and the obstacle; the feature units in the unit state sequence are fused using a gated recurrent unit to extract hidden information as the state information at that moment, and the state information is input into the corresponding reinforcement learning neural network.

2. The method according to claim 1, characterized in that, Before constructing the attitude control network for the UAV during flight based on the UAV's state information and target flight values ​​using the deep deterministic policy gradient algorithm, the method further includes: Establish a reference model for the UAV during its flight; The method employs a deep deterministic policy gradient algorithm to construct an attitude control network for the UAV during flight, based on the UAV's state information and target flight values. This includes: The state information, the target flight value, and the reference state information output by the reference model based on the target flight value and the environmental information of the UAV are input into the policy-evaluation network architecture matched by the deep deterministic policy gradient algorithm for network training, thereby obtaining the attitude control network.

3. The method according to claim 1, characterized in that, Before training the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the flight control network of the UAV, the method further includes: Based on the flight parameters of the UAV during flight, a reward signal reflecting the flight quality of the UAV is constructed; wherein the flight parameters include at least: flight stability parameters, route tracking parameters, and energy consumption parameters; Construct reward strategies for the UAV at different flight control phases; Based on the reward signal and the reward strategy, the preset flight reward mechanism is constructed.

4. The method according to claim 1 or 3, characterized in that, The process of training the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the flight control network of the UAV includes: The historical flight data is labeled and organized to form a training dataset; The reward function of the hierarchical deep reinforcement learning network is constructed using the preset flight reward mechanism to obtain the network to be adjusted, and the network to be adjusted is trained using the training dataset to obtain the flight control network.

5. The method according to claim 1, characterized in that, After training the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism to obtain the flight control network of the UAV, the method further includes: Determine the preset network optimization strategy; The flight control network is optimized by adopting the preset network optimization strategy to obtain the optimized flight control network.

6. A system for constructing a flight control network for an unmanned aerial vehicle (UAV), characterized in that, The system includes: The first network construction module is used to construct the attitude control network of the UAV during flight based on the state information and target flight values ​​of the UAV during flight using a deep deterministic policy gradient algorithm. The second network construction module is used to input the state information and the distance information in the environment where the UAV is located into a recurrent neural network built by a dual-delay deep deterministic policy gradient algorithm and a soft actor-commentator algorithm for network training to obtain a path planning network; and to input the state information and the obstacle information in the environment into a neural network built by a reinforcement learning algorithm based on unit state sequences for network training to obtain an intelligent avoidance network. The network construction module is used to build the basic architecture of the initial hierarchical deep reinforcement learning network based on the state information, the executable action information of the UAV during flight and the corresponding reward information; the attitude control network, the path planning network and the intelligent avoidance network are sequentially used as the low-level deep reinforcement learning sub-network, the mid-level deep reinforcement learning sub-network and the high-level deep reinforcement learning sub-network in the basic architecture to obtain the hierarchical deep reinforcement learning network of the UAV during flight. The training module is used to train the hierarchical deep reinforcement learning network using historical flight data of the UAV under different flight environments and a preset flight reward mechanism, so as to obtain the flight control network of the UAV. The distance information includes at least the distance between the UAV and obstacles identified by the UAV during flight; the high-level deep reinforcement learning subnetwork can control the decision-making of the mid-level deep reinforcement learning subnetwork, and the mid-level deep reinforcement learning subnetwork can control the decision-making of the low-level deep reinforcement learning subnetwork; the execution process of the reinforcement learning algorithm based on unit state sequence is as follows: the obstacle information and the UAV state information are encoded one-to-one to form feature units; all encoded feature units are sorted and combined into a unit state sequence according to the distance priority between the UAV and the obstacle; the feature units in the unit state sequence are fused using a gated recurrent unit to extract hidden information as the state information at that moment, and the state information is input into the corresponding reinforcement learning neural network.

7. A flight control method for an unmanned aerial vehicle (UAV), characterized in that, The method includes: Obtain the current flight target and current flight environment of the UAV; The current flight target and the current flight environment are input into the flight control network constructed by the method as described in any one of claims 1 to 5 to obtain the flight control commands for the UAV; The flight control commands are used to control the UAV to fly to the current flight target.

8. A flight control system for an unmanned aerial vehicle (UAV), characterized in that, The system includes: The acquisition module is used to acquire the current flight target and current flight environment of the UAV; An input module is used to input the current flight target and the current flight environment into the flight control network constructed by the system as described in claim 6, so as to obtain the flight control command of the UAV; The control module is used to control the UAV to fly to the current flight target using the flight control commands.

Images