Unmanned aerial vehicle dynamic hovering anti-disturbance control method based on neural network

Abstract

The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to a dynamic hovering anti-disturbance control method for unmanned aerial vehicles based on a neural network, comprising: constructing a time series convolution network as a disturbance observer to estimate total disturbance according to unmanned aerial vehicle state quantities and wind speed measurement values within a historical time window; constructing a deep deterministic policy gradient network as a controller to generate motor speed instructions according to state errors and total disturbance; jointly training the two networks through domain randomization in a simulation environment; deploying the trained networks to a flight controller, estimating total disturbance by the time series convolution network and generating speed instructions by the deep deterministic policy gradient network to drive the motor in real-time flight; and setting a safety constraint layer to perform amplitude limiting on the speed instructions or switch to a backup controller when state parameters exceed a threshold. The present application realizes high-precision hovering control under strong wind disturbance and guarantees safety and real-time performance.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a method for dynamic hovering and anti-disturbance control of UAVs based on neural networks. Background Technology

[0002] Multi-rotor drones, due to their simple structure, maneuverability, and vertical takeoff and landing capabilities, have been widely used in industrial inspection, precision agriculture, aerial logistics, and film and television production. In these applications, drones often need to perform high-precision hovering tasks in complex wind conditions, such as inspecting high-voltage power lines, spraying pesticides at a constant height above crops, and landing precisely on rooftop helipads. However, drones are susceptible to external wind disturbances while hovering. Especially under the influence of nonlinear, high-frequency disturbances such as narrow-channel winds in urban canyons, stern airflow on ship decks at sea, and turbulence caused by rotating wind turbine blades, the hovering accuracy of drones can significantly decrease, and may even induce attitude instability.

[0003] In existing technologies, mainstream flight control systems typically employ proportional-integral-derivative (PI-DI) control or model predictive control, combined with extended state observers (ESOs) to achieve disturbance rejection control. These methods often rely on the accuracy of the UAV dynamics modeling, estimating the total disturbance using an extended state observer and compensating for it in the control law.

[0004] However, the above methods still have technical bottlenecks in practical applications: First, the estimation performance of the extended state observer is limited by the observer bandwidth. For high-frequency and nonlinear disturbances, the estimation will have phase lag and amplitude attenuation, making it difficult to achieve stable centimeter-level hovering accuracy in strong wind environments. Second, model predictive control is sensitive to the accuracy of the aerodynamic model. When the UAV causes model mismatch due to load changes, center of gravity shifts, or aerodynamic parameter drift, the control performance will degrade significantly and it will be difficult to effectively adapt to complex and variable nonlinear disturbances.

[0005] If a neural network observer and a learning controller are directly used for closed-loop disturbance rejection control, a subtle contradiction in coupling stability arises: the observer's disturbance estimation error is fed forward into the controller as compensation, affecting the direction and scale of policy gradient updates during policy learning; simultaneously, the stability and predictability of the controller's actions, in turn, determine the effectiveness of the observer's estimation. If the two are trained independently, collaborative adaptation is difficult to achieve; if end-to-end joint training is used without stabilizing the observation-policy coupling error, oscillations are easily introduced in the early stages of training, making convergence to a stable control policy difficult. Therefore, how to achieve collaborative training and online fusion of the two while ensuring observation accuracy and control performance has become a key challenge that urgently needs to be addressed in this field. Summary of the Invention

[0006] To overcome the above shortcomings, this invention provides a neural network-based dynamic hovering disturbance rejection control method for unmanned aerial vehicles (UAVs). This method aims to improve the existing UAV disturbance rejection control methods, which suffer from insufficient accuracy in estimating high-frequency nonlinear disturbances due to limited bandwidth of disturbance observers, and excessive dependence on aerodynamic models, resulting in low hovering control accuracy and poor adaptability in strong wind environments.

[0007] This invention provides the following technical solution: a method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks, comprising the following steps:

[0008] S1. Construct a temporal convolutional network as a perturbation observer to estimate the total perturbation at the current moment based on the UAV state variables and wind speed measurements within a historical time window. The temporal convolutional network comprises a stacked structure of M residual blocks, each residual block including two dilated causal convolutional layers. The causal convolution of the dilated causal convolutional layers uses zero-padding on the left side of the convolution kernel to ensure that the output depends only on historical input and does not introduce future information. The dilated convolution of the dilated causal convolutional layers expands the receptive field by inserting d-1 zeros between the convolution kernel elements, where d is the dilation coefficient. The output of the temporal convolutional network is mapped to the total perturbation estimate via a fully connected layer. This represents the equivalent acceleration disturbance experienced by the UAV in the three axes of the body coordinate system at the current moment;

[0009] S2. Construct a deep deterministic policy gradient network as a controller to generate speed control commands for each motor based on the current state error and the total disturbance. The estimated total disturbance is used as a feedforward compensation input, which, together with the state error, constitutes the input vector of the controller, forming a feedforward-feedback composite control structure. The feedforward compensation is configured to respond to instantaneous disturbances caused by sudden changes in wind speed, and the deep deterministic policy gradient network is configured to compensate for the residual error after the feedforward compensation based on a reinforcement learning strategy.

[0010] S3. In a simulation environment, the temporal convolutional network and the deep deterministic policy gradient network are jointly trained by domain randomization until the convergence condition is met.

[0011] S4. Deploy the trained temporal convolutional network and deep deterministic policy gradient network to the flight controller. During real-time flight, the temporal convolutional network estimates the total disturbance, and the deep deterministic policy gradient network generates speed control commands to drive the motor.

[0012] S5. A safety constraint layer is set up to monitor the UAV's operating status and limit the rotation speed control command when the status parameters exceed a safety threshold. When the status parameters exceed a degradation threshold, the system switches to a backup controller. The limiting process and the degradation switching are triggered independently, and the limiting process has a preset priority and is always effective in any control mode. In response to the triggering of the degradation switching, the rotation speed control command is gradually transitioned to the initial rotation speed control command output by the backup controller. In response to meeting a preset recovery condition, the backup controller switches back to the deep deterministic policy gradient network, where the rotation speed control command is gradually transitioned.

[0013] Preferably, in step S1, the step of constructing a temporal convolutional network as a perturbation observer specifically includes:

[0014] Collect state parameters and wind speed measurements of the drone during its flight;

[0015] Based on the current moment, the state variables and wind speed measurements within a preset historical time window are extracted to form a time-series input matrix;

[0016] The temporal input matrix is ​​input into a temporal convolutional network, which employs a causal convolutional structure to ensure that the output depends only on historical information, and uses an expanded convolutional structure to increase the receptive field.

[0017] After passing through multiple layers of convolution and nonlinear activation operations of the temporal convolutional network, the total perturbation estimate at the current time is output.

[0018] Preferably, in step S2, the step of constructing a deep deterministic policy gradient network as a controller specifically includes:

[0019] Obtain the expected state and actual state of the UAV at the current moment, and calculate the state error vector;

[0020] The state error vector is concatenated with the total disturbance estimate to form the controller input vector;

[0021] The controller input vector is input to a deep deterministic policy gradient network, which includes an action network and an evaluation network. The action network is used to generate action instructions based on the input, and the evaluation network is used to evaluate the value of the action to guide the action network to update.

[0022] The motion network, after passing through multiple fully connected layers and nonlinear activation operations, outputs speed control commands for each motor. The dimension of the speed control commands matches the number of motors in the UAV.

[0023] Preferably, the step of inputting the controller input vector into the deep deterministic policy gradient network specifically includes:

[0024] The controller input vector is simultaneously input into the action network and the evaluation network;

[0025] The action network extracts features from the controller input vector through multiple fully connected layers, and after processing by a nonlinear activation function, outputs speed control commands for each motor as action commands.

[0026] The evaluation network performs joint feature extraction on the controller input vector and the action command output by the action network through multiple fully connected layers, and outputs the state-action value function corresponding to the current action command.

[0027] The temporal difference error is calculated based on the state-action value function. The network parameters of the evaluation network are updated using the gradient descent method, and the network parameters of the action network are updated using the policy gradient method, so that the action commands output by the action network gradually approach the optimal control policy.

[0028] Preferably, in step S3, the step of jointly training the temporal convolutional network and the deep deterministic policy gradient network in a simulation environment through domain randomization specifically includes:

[0029] Build a drone simulation environment;

[0030] Set a range of domain randomization parameters, which include wind field parameters and UAV dynamics parameters. At the beginning of each training round, randomly sample a set of parameter configurations from the range of parameters to generate diverse training scenarios.

[0031] In each training round, the temporal convolutional network and the deep deterministic policy gradient network are deployed in the simulation environment to drive the UAV to perform dynamic hovering tasks and collect state data, disturbance estimates, control commands and reward signals.

[0032] Based on the collected data, the network parameters of the temporal convolutional network are updated by minimizing the temporal difference error, and the network parameters of the deep deterministic policy gradient network are updated by the policy gradient method.

[0033] Repeat the above steps until the hovering position error of the drone meets the preset convergence threshold under the preset verification scenario, at which point the joint training is considered complete.

[0034] Preferably, the step of setting the range of domain randomization parameters specifically includes:

[0035] Set the randomization range for wind field parameters;

[0036] Set the randomization range for the UAV dynamic parameters;

[0037] At the beginning of each training round, a set of parameter values ​​are independently and randomly sampled from the randomization range of the wind field parameters and the randomization range of the UAV dynamic parameters to form the simulation environment configuration for the current round.

[0038] The simulation environment configuration is loaded into the simulation environment so that the UAV is disturbed by the corresponding wind field parameters during the current round of flight and has the motion characteristics of the corresponding dynamic parameters.

[0039] Preferably, in step S4, the step of deploying the trained temporal convolutional network and deep deterministic policy gradient network to the flight controller specifically includes:

[0040] Export the trained temporal convolutional network and deep deterministic policy gradient network as model files in a preset format;

[0041] The model file is loaded into the non-volatile memory of the flight controller, and the inference engines of the temporal convolutional network and the deep deterministic policy gradient network are initialized on the processor of the flight controller.

[0042] During real-time flight, the state variables and wind speed measurements of the UAV are collected at a preset sampling frequency. The state variables and wind speed measurements within a preset historical time window before the current moment are input into a temporal convolutional network, and the total disturbance estimate at the current moment is obtained through forward inference calculation.

[0043] The desired state and actual state of the UAV at the current moment are obtained, the state error vector is calculated, the state error vector is concatenated with the total disturbance estimate and then input into the deep deterministic policy gradient network, and the speed control command of each motor is obtained through forward inference calculation.

[0044] The speed control command is output to the motor drive module of the flight controller, and the motor drive module generates a pulse width modulation signal to drive the UAV motor to operate.

[0045] Preferably, in step S5, the step of limiting the speed control command when the state parameter exceeds the safety threshold specifically includes:

[0046] Set a set of safety thresholds, which includes an upper limit for motor speed, a lower limit for motor speed, an upper limit for attitude angle, and an upper limit for attitude angular rate;

[0047] During real-time flight, the operating status parameters of the UAV are collected at a preset monitoring frequency. The operating status parameters include at least the current speed of each motor, the current attitude angle of the UAV, and the current attitude angular rate.

[0048] When the current speed exceeds the upper limit of the motor speed or falls below the lower limit of the motor speed, the speed control command is subjected to a limiting process to restrict the speed control command between the upper limit of the motor speed and the lower limit of the motor speed.

[0049] When the current attitude angle exceeds the upper limit of the attitude angle, the speed control command is subjected to a limiting process to restrict the further increase of the attitude angle;

[0050] When the current attitude angular rate exceeds the upper limit of the attitude angular rate, the speed control command is subjected to a limiting process to restrict further increase in the attitude angular rate.

[0051] Preferably, in step S5, the step of switching to the standby controller when the state parameters exceed the degradation threshold specifically includes:

[0052] Set a set of degradation thresholds, which includes a threshold for the duration of attitude angle exceeding the limit and a threshold for the attitude angle rate exceeding the limit;

[0053] During real-time flight, the operating status parameters of the UAV are collected at a preset monitoring frequency. The operating status parameters include at least the current attitude angle and the current attitude angular rate of the UAV.

[0054] When the duration for which the current attitude angle exceeds the upper limit of the attitude angle reaches the attitude angle over-limit duration threshold, the current controller will be switched from the deep deterministic policy gradient network to the backup controller.

[0055] When the current attitude angular rate exceeds the attitude angular rate over-limit threshold, the current controller will be switched from the deep deterministic policy gradient network to the backup controller.

[0056] After switching to the backup controller, the operating status parameters of the UAV are continuously monitored. When the operating status parameters are within the range of the degradation threshold set for a continuous preset time, the current controller is switched from the backup controller back to the deep deterministic policy gradient network.

[0057] The present invention has the following beneficial effects:

[0058] 1. This invention uses a temporal convolutional network as a perturbation observer. The causal convolutional structure ensures that the output depends only on historical information. By expanding the convolutional structure, the receptive field is increased without significantly increasing the number of parameters. This enables accurate estimation of perturbation features and unmodeled dynamics within the historical time window, thus improving the estimation accuracy of high-frequency nonlinear perturbations.

[0059] 2. This invention employs a deep deterministic policy gradient network as the controller, using both state error and disturbance estimation as inputs to form a composite control structure of feedforward compensation and reinforcement learning feedback. This structure does not rely on a precise UAV dynamics model; the control strategy is autonomously learned in a simulation environment through reinforcement learning. When the model becomes inaccurate due to load changes, center of gravity shifts, or aerodynamic parameter drift, the controller still maintains good control performance, improving adaptability to different operating conditions and environments. In this composite control structure, feedforward compensation provides a rapid response to instantaneous disturbances caused by sudden wind speed changes, while the feedback strategy finely corrects the residual error after feedforward compensation. The complementary effect of these two approaches makes the joint training process more likely to converge to a stable control strategy, achieving centimeter-level hovering accuracy even in complex disturbance scenarios.

[0060] 3. This invention establishes a multi-level safety constraint layer, including hard constraints such as motor speed limiting, attitude angle limiting, and attitude angular rate limiting, as well as a degradation switching mechanism based on the duration of attitude angle exceeding the limit and the threshold of attitude angular rate exceeding the limit. When the system state exceeds the safety boundary, it automatically switches to the backup controller and automatically recovers to the intelligent controller after the state returns to normal, ensuring the flight safety of the UAV under extreme conditions. In this safety constraint layer, the limiting processing is always effective according to the preset priority, the degradation switching is triggered independently, the limiting processing does not fail due to the degradation switching, and the degradation switching does not replace the protection function of the limiting processing; together with the gradual transition during the switching process and the automatic back-switch mechanism after the state recovers, multi-level safety protection is achieved, ensuring the reliable operation of the neural network controller in complex wind disturbance environments. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating the neural network-based dynamic hovering anti-disturbance control method for unmanned aerial vehicles proposed in this invention.

[0062] Figure 2 This is a schematic diagram of the causal extended residual structure of the temporal convolutional network in the neural network-based UAV dynamic hovering disturbance resistance control method proposed in this invention.

[0063] Figure 3 This is a schematic diagram of the feedforward-feedback composite control structure of the UAV dynamic hovering disturbance resistance control method based on neural networks proposed in this invention. Detailed Implementation

[0064] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] This invention provides a neural network-based dynamic hovering disturbance rejection control method for unmanned aerial vehicles (UAVs), such as... Figure 1 As shown, it includes the following steps:

[0066] S1. Construct a temporal convolutional network as a disturbance observer to estimate the total disturbance at the current moment based on the UAV state variables and wind speed measurements within the historical time window.

[0067] Furthermore, in S1, the specific steps for constructing a temporal convolutional network as a perturbation observer include:

[0068] Collect state parameters and wind speed measurements of the drone during its flight;

[0069] Based on the current moment, extract the state variables and wind speed measurements within a preset historical time window to form a time-series input matrix;

[0070] The temporal input matrix is ​​fed into a temporal convolutional network. The temporal convolutional network uses a causal convolutional structure to ensure that the output depends only on historical information, and uses an expanded convolutional structure to increase the receptive field.

[0071] After passing through multiple layers of convolution and nonlinear activation operations in a temporal convolutional network, the total perturbation estimate at the current time is output.

[0072] Specifically, the system collects the drone's state variables and wind speed measurements during flight at a preset sampling frequency. State variables include the drone's three-axis position, three-axis axial velocity, three-axis attitude angle, and three-axis angular velocity, obtained through fusion of data from the onboard inertial measurement unit and the global positioning system. Wind speed measurements are collected by an onboard ultrasonic anemometer, installed at the end of the drone's arm to reduce interference from the rotor downwash.

[0073] Using the current time t as a reference, extract the state variable sequence and wind speed measurement value sequence within a preset historical time window length L. Let the sampling period be Ts, then the historical time window contains N sampling times, N = L / Ts. The state variable vectors at the N times are... ,..., Compared with wind speed measurement value ,..., Concatenate along the feature dimensions to form the temporal input matrix. , where D is the sum of the dimension of the state variables and the dimension of the wind speed measurement.

[0074] Temporal convolutional networks employ a stacked structure of causal convolutions and dilated convolutions. Let the temporal input matrix be... The network consists of M stacked residual blocks, each containing two dilated convolutional layers. The dilation coefficient d of each dilated convolutional layer is... , where m is the residual block index. The causal convolution of the dilated causal convolutional layer is achieved by zero-padding to the left of the convolutional kernel, ensuring that the output at time i depends only on the input at time i and previous times, without introducing future information. The dilated convolution of the dilated causal convolutional layer expands the receptive field by inserting d-1 zeros between convolutional kernel elements, enabling the network to capture long-term temporal dependencies without significantly increasing the number of parameters. Each dilated convolutional layer is followed by a batch normalization layer and a ReLU activation function. Skip connections are used within the residual block, adding the input to the feature map after two layers of dilated convolution and activation. The size of the network's receptive field is determined by the convolutional kernel size, dilation coefficient, and number of network layers, and is set to cover at least 2 seconds of historical information to fully capture low-frequency perturbation features above 0.5Hz.

[0075] Input the timing matrix The input is fed into a temporal convolutional network. After forward propagation through M residual blocks, the output feature map is mapped to the total perturbation estimate through a fully connected layer. This represents the equivalent acceleration disturbance experienced by the UAV along the three axes of the body coordinate system at the current moment, including the combined effects of external wind disturbance and unmodeled dynamics. The forward propagation calculation process is represented as follows:

[0076] ;

[0077] in, These are the training parameters for the temporal convolutional network, including the weight matrix and bias terms of each convolutional layer, and the scaling factor and translation parameters of the batch normalization layer.

[0078] The temporal convolutional network and the subsequent deep deterministic policy gradient network are jointly trained in a simulation environment. During training, the mean square error between the perturbation estimate output by the temporal convolutional network and the actual perturbation value in the simulation environment is used as part of the loss function. By minimizing the temporal difference error and co-updating with the policy gradient method, the network parameters are updated, enabling the perturbation observer to converge to the optimal estimate of the total perturbation.

[0079] S2. Construct a deep deterministic policy gradient network as a controller to generate speed control commands for each motor based on the current state error and total disturbance.

[0080] Furthermore, in S2, the specific steps of constructing a deep deterministic policy gradient network as a controller include:

[0081] Obtain the expected state and actual state of the UAV at the current moment, and calculate the state error vector;

[0082] The state error vector is concatenated with the total disturbance estimate to form the controller input vector; wherein the total disturbance estimate is used as the feedforward compensation input, and together with the state error, they constitute the controller input vector, forming a feedforward-feedback composite control structure.

[0083] The controller input vector is fed into a deep deterministic policy gradient network, which includes an action network and an evaluation network. The action network is used to generate action instructions based on the input, and the evaluation network is used to evaluate the value of the action to guide the action network to update.

[0084] After passing through multiple fully connected layers and nonlinear activation operations, the motion network outputs speed control commands for each motor, and the dimension of the speed control commands matches the number of motors in the drone.

[0085] Furthermore, the specific steps of inputting the controller input vector into the deep deterministic policy gradient network include:

[0086] The controller input vector is simultaneously fed into the action network and the evaluation network;

[0087] The action network extracts features from the controller input vector through multiple fully connected layers, and after processing by a nonlinear activation function, outputs speed control commands for each motor as action commands.

[0088] The evaluation network performs joint feature extraction on the controller input vector and the action command output by the action network through multiple fully connected layers, and outputs the state-action value function corresponding to the current action command.

[0089] The temporal difference error is calculated based on the state-action value function. The network parameters of the evaluation network are updated using the gradient descent method, and the network parameters of the action network are updated using the policy gradient method, so that the action commands output by the action network gradually approach the optimal control policy.

[0090] Specifically, the desired state and actual state of the UAV at the current time t are obtained. The desired state is provided by the flight mission planning module, including the desired position. Expected speed Desired attitude angle and desired angular velocity The actual state is obtained by fusing data from airborne sensors, including the actual position. Actual speed Actual attitude angle and actual angular velocity .

[0091] Calculate the state error vector :

[0092] ;

[0093] The state error vector Compared with the total disturbance estimate output in step S1 The vectors are concatenated to form the controller input vector. :

[0094] ;

[0095] The Deep Deterministic Policy Gradient Network adopts an Actor-Critic architecture, comprising two sub-networks: an action network and an evaluation network. The action network... Used to transmit controller input vectors The mapping is to action commands. The action network consists of three stacked fully connected layers, each containing 128 neurons. The first two layers are followed by a ReLU activation function, and the output layer is followed by a Tanh activation function to constrain the output value between -1 and 1. This is then linearly transformed and mapped to the actual physical range of the motor speeds. The action network outputs speed control commands for each motor. Where K is the number of motors in the drone. Evaluation Network Used to assess the state Next action The value of the network is evaluated by taking its input as a state vector. With action vectors The concatenation consists of three stacked fully connected layers, each containing 128 neurons. The first two layers are followed by a ReLU activation function, and the output layer is a single neuron that outputs the state-action value function. , representing the expected cumulative reward for the current state action pair.

[0096] Controller input vector The calculation process via forward propagation through the action network is as follows:

[0097] ;

[0098] ;

[0099] ;

[0100] ;

[0101] in, , and Here is the weight matrix of the action network. , and For bias terms, and This is the speed conversion coefficient, which maps the normalized action to the actual range of motor speed. The final output is the speed control command for each motor. .

[0102] controller input vector Action commands output by the action network After concatenation, the data is input into the evaluation network to calculate the state-action value function.

[0103] ;

[0104] ;

[0105] ;

[0106] in, , and To evaluate the weight matrix of the network, , and For the bias term, output It is a scalar value representing the value of the current state-action pair.

[0107] The training of the deep deterministic policy gradient network employs an experience replay mechanism. During flight mission execution in a simulation environment, samples from each step are used... Stored in the experience replay buffer, where For immediate rewards, mini-batches of samples are randomly sampled from the buffer during training to update parameters. Network updates are evaluated by minimizing temporal difference error. Target value. The calculation is as follows:

[0108] ;

[0109] in, As a discount factor, and For the target evaluation network and the target action network, their parameters and The network parameters are slowly replicated from the current parameters using a soft update method. The network loss function is evaluated as mean squared error.

[0110] ;

[0111] Updating evaluation network parameters using gradient descent .

[0112] The action network is updated by maximizing the evaluation network output using the policy gradient method.

[0113] ;

[0114] Updating Action Network Parameters Using Gradient Ascent .

[0115] The target network parameters are updated using a soft update method:

[0116] ;

[0117] ;

[0118] in This is the soft update coefficient, typically set to 0.001. Through the above update mechanism, the speed control command output by the action network gradually approaches the optimal control strategy.

[0119] S3. In the simulation environment, the temporal convolutional network and the deep deterministic policy gradient network are jointly trained by domain randomization until the convergence condition is met.

[0120] Furthermore, in S3, the specific steps for jointly training the temporal convolutional network and the deep deterministic policy gradient network in a simulation environment through domain randomization include:

[0121] Build a drone simulation environment;

[0122] Set the range of domain randomization parameters, which include wind field parameters and UAV dynamics parameters. At the beginning of each training round, randomly sample a set of parameter configurations from the parameter range to generate diverse training scenarios.

[0123] In each training round, a temporal convolutional network and a deep deterministic policy gradient network are deployed in the simulation environment to drive the UAV to perform dynamic hovering tasks and collect state data, disturbance estimates, control commands and reward signals.

[0124] Based on the collected data, the network parameters of the temporal convolutional network are updated by minimizing the temporal difference error, and the network parameters of the deep deterministic policy gradient network are updated by the policy gradient method.

[0125] Repeat the above steps until the hovering position error of the drone meets the preset convergence threshold under the preset verification scenario, at which point the joint training is considered complete.

[0126] Furthermore, the steps for setting the range of domain randomization parameters specifically include:

[0127] Set the randomization range for wind field parameters;

[0128] Set the randomization range for the UAV dynamic parameters;

[0129] At the beginning of each training round, a set of parameter values ​​are independently and randomly sampled from the randomization range of wind field parameters and the randomization range of UAV dynamic parameters to form the simulation environment configuration for the current round.

[0130] The simulation environment configuration is loaded into the simulation environment so that the UAV is disturbed by the corresponding wind field parameters during the current round of flight and has the motion characteristics of the corresponding dynamic parameters.

[0131] Specifically, a physics engine-based UAV simulation environment is built, which includes a UAV dynamics model and a wind field model. The UAV dynamics model uses six-degree-of-freedom rigid body dynamics equations to describe the UAV's motion response under rotor thrust, aerodynamic forces, gravity, and external disturbances. The wind field model uses the Dryden atmospheric turbulence spectrum model to generate a three-dimensional wind speed field. This model can generate wind speed sequences that conform to the statistical characteristics of atmospheric turbulence, and its power spectral density function is:

[0132] ;

[0133] in For turbulence intensity, Where V is the turbulent scale and V is the flight velocity. ω is the angular frequency. By adjusting the turbulence intensity and average wind speed, wind field disturbances of different intensities can be generated.

[0134] Set the randomization range for wind field parameters. Wind field parameters include average wind speed, wind direction angle, and turbulence intensity. The randomization range for average wind speed is 0 to 15 m / s, for wind direction angle it is 0 to 360 degrees, and for turbulence intensity it is 5% to 30%. These parameters are independently and randomly sampled at the beginning of each training round, and each parameter follows a uniform distribution. Set the randomization range for UAV dynamic parameters. UAV dynamic parameters include UAV mass, moment of inertia, motor time constant, and thrust coefficient. The randomization range for UAV mass is ±20% of the nominal value, for moment of inertia it is ±30% of the nominal value, for motor time constant it is ±15% of the nominal value, and for thrust coefficient it is ±10% of the nominal value. These parameters are independently and randomly sampled at the beginning of each training round, and each parameter follows a uniform distribution.

[0135] At the start of each training round, a set of parameter values ​​are independently and randomly sampled from the randomized ranges of wind field parameters and UAV dynamic parameters to form the simulation environment configuration for the current round. This configuration is then loaded into the simulation environment, causing the UAV to be disturbed by the corresponding wind field parameters and to exhibit the motion characteristics of the corresponding dynamic parameters during the flight of the current round.

[0136] In each training round, the temporal convolutional network constructed in step S1 and the deep deterministic policy gradient network constructed in step S2 are deployed in the simulation environment to drive the UAV to perform a dynamic hovering task. At the start of the task, the UAV takes off to a preset hovering altitude and maintains its position under wind disturbance. Each training round contains T time steps, and each time step performs the following operations: First, obtain the UAV's state variables and wind speed measurements from the simulation environment. State variables include position, velocity, attitude angle, and angular velocity; wind speed measurements are provided by a virtual wind speed sensor in the simulation environment. Second, input the state variables and wind speed measurements from the historical time window into the temporal convolutional network, and obtain the total disturbance estimate for the current time step through forward inference. Then, obtain the expected state and actual state of the UAV at the current time step, calculate the state error vector, concatenate the state error vector with the total disturbance estimate, and input it into the deep deterministic policy gradient network, obtaining the speed control commands for each motor through forward inference. Finally, send the speed control commands to the motor model in the simulation environment to drive the UAV dynamics model to update to the state of the next time step, and calculate the instantaneous reward signal. Reward function. Defined as:

[0137] ;

[0138] in For positional error, For speed error, This is the speed control command for the current moment. The previous speed control command, weighted by a coefficient. =1.0、 =0.5、 =0.01、 =0.005 is used to balance hovering accuracy, energy consumption, and control smoothness, respectively.

[0139] The sample data for each step is stored in the experience replay buffer, and the samples include the current state. ,action ,award Next state And the intermediate features required for training temporal convolutional networks.

[0140] After collecting N samples, a small batch of samples is randomly sampled from the experience replay buffer to update the network parameters. The update of the temporal convolutional network is achieved by minimizing the perturbation estimation error. The loss function is defined as the mean squared error between the perturbation estimate output by the temporal convolutional network and the actual perturbation value in the simulation environment:

[0141] ;

[0142] Where M is the number of samples in the mini-batch. This represents the perturbation estimate output by the temporal convolutional network. The values ​​represent the actual perturbations in the simulation environment. The parameters of the temporal convolutional network are updated using gradient descent. .

[0143] The deep deterministic policy gradient network updates using the temporal difference error minimization and policy gradient method described in step S2. The loss function for evaluating the network is:

[0144] ;

[0145] Where the target value The action network updates via policy gradients:

[0146] ;

[0147] The temporal convolutional network and the deep deterministic policy gradient network adopt an alternating update strategy, and both perform multiple parameter updates in each training round.

[0148] After each training round, the drone's hovering performance is evaluated in a preset validation scenario. The validation scenario includes multiple sets of fixed wind field parameters and dynamic parameters, which are not involved in the domain randomization training. When the validation hovering position error is less than a preset convergence threshold of 5 cm for 50 consecutive training rounds, the joint training is considered complete, and the current network parameters are saved as the final model.

[0149] S4. Deploy the trained temporal convolutional network and deep deterministic policy gradient network to the flight controller. During real-time flight, the temporal convolutional network estimates the total disturbance, and the deep deterministic policy gradient network generates speed control commands to drive the motor.

[0150] Furthermore, in S4, the specific steps for deploying the trained temporal convolutional network and deep deterministic policy gradient network to the flight controller include:

[0151] Export the trained temporal convolutional network and deep deterministic policy gradient network as model files in a preset format;

[0152] The model file is loaded into the non-volatile memory of the flight controller, and the inference engines of the temporal convolutional network and the deep deterministic policy gradient network are initialized on the processor of the flight controller.

[0153] During real-time flight, the state variables and wind speed measurements of the UAV are collected at a preset sampling frequency. The state variables and wind speed measurements within a preset historical time window before the current moment are input into a temporal convolutional network, and the total disturbance estimate at the current moment is obtained through forward inference calculation.

[0154] The desired state and actual state of the UAV at the current moment are obtained, the state error vector is calculated, and the state error vector and the total disturbance estimate are concatenated and input into the deep deterministic policy gradient network. The speed control commands of each motor are obtained through forward inference calculation.

[0155] The speed control command is output to the motor drive module of the flight controller, which then generates a pulse width modulation signal to drive the UAV motor.

[0156] Specifically, the temporal convolutional network and deep deterministic policy gradient network trained in step S3 are exported as model files in a lightweight inference format. The temporal convolutional network contains M residual blocks, each containing two dilated convolutional layers and batch normalization layer parameters. The deep deterministic policy gradient network contains an action network and an evaluation network. During deployment, only the action network is exported for real-time control; the evaluation network does not participate in real-time inference. The model is exported using ONNX or TensorRT Lite format, which supports efficient forward inference on embedded platforms and optimizes the network structure through operator fusion and memory reuse.

[0157] The exported model file is burned into the non-volatile memory of the flight controller. The flight controller adopts an architecture based on an ARM Cortex-M7 or ARM Cortex-A series processor with a clock speed of no less than 400MHz and a memory capacity of no less than 2MB. After the flight controller is powered on, it reads the model file from the non-volatile memory and initializes the inference engine of the temporal convolutional network and the deep deterministic policy gradient network on the processor. The inference engine loads the network weight parameters, allocates input and output tensor memory, and creates the network execution graph. The inference engine uses fixed-point quantization or floating-point arithmetic, selecting the optimal computation backend based on the processor hardware characteristics.

[0158] During real-time flight, the flight controller collects the UAV's state variables and wind speed measurements at a preset sampling frequency. The sampling frequency is consistent with the flight controller's main loop frequency, set to 500 Hz. The state variables are obtained by fusing data from the inertial measurement unit, the global positioning system, and the magnetometer, including three-axis position, three-axis linear velocity, three-axis attitude angle, and three-axis angular velocity. Wind speed measurements are collected by an onboard ultrasonic anemometer, with the sampling frequency synchronized with the state variables. Using the current time t as a reference, a circular buffer of length N is maintained in memory, storing the state variables and wind speed measurements from the most recent N sampling times, where N equals the ratio of the preset historical time window length L to the sampling period Ts, N = L / Ts, and L is set to 2 seconds. The data in the buffer is organized into a time-series input matrix. Where D is the sum of the dimension of the state variables and the dimension of the wind speed measurements. The time series input matrix is ​​then used. The input is fed into the temporal convolutional network inference engine for forward inference computation. The inference engine performs convolution operations, batch normalization operations, and nonlinear activation operations in the order of network layers. Convolution operations are implemented using im2col combined with general matrix multiplication or the Winograd fast convolution algorithm. After forward inference, the total perturbation estimate at the current time step is output. , representing the equivalent acceleration disturbance experienced by the UAV along the three axes of the body coordinate system. The calculation process for the total disturbance estimate is expressed as follows:

[0159] ;

[0160] in This represents a temporal convolutional network inference engine deployed on the flight controller. The end-to-end latency from input data acquisition to disturbance estimate output is no more than 2 milliseconds.

[0161] Obtain the expected state and actual state of the UAV at the current moment, and calculate the state error vector. The desired state is generated by the flight mission planning module via remote control commands or autonomous flight path, while the actual state is provided by the sensor fusion module. The state error vector is then... The total perturbation estimate of the output of the temporal convolutional network The vectors are concatenated to form the controller input vector. :

[0162] ;

[0163] controller input vector The input is fed into the action network inference engine of the deep deterministic policy gradient network to perform forward inference computation. The action network consists of three stacked fully connected layers, each containing 128 neurons. The forward inference process sequentially performs fully connected layer operations, batch normalization operations, and ReLU activation function operations. The output layer, after passing through the Tanh activation function, is mapped to the actual range of motor speed via a linear transformation. The forward inference computation process is represented as follows:

[0164] ;

[0165] in This refers to the action network inference engine deployed on the flight controller. The output speed control command is given by K, where K is the number of motors in the drone; for a quadcopter drone, K is 4. The end-to-end delay from the controller input vector to the output speed control command does not exceed 3 milliseconds.

[0166] The speed control command output by the motion network The output is sent to the motor drive module of the flight controller. The motor drive module converts the speed control command into a pulse-width modulation (PWM) signal, the duty cycle of which is linearly related to the target speed. The PWM signal drives the brushless DC motor via an electronic speed controller, which in turn rotates the rotor to generate lift and torque, enabling attitude control and position holding for the UAV. The output frequency of the speed control command is consistent with the main loop frequency of the flight controller, ensuring real-time updates of the control commands.

[0167] The temporal convolutional neural network inference engine and the deep deterministic policy gradient network inference engine operate in a pipelined parallel manner, sharing the same input sampling period. When new sensor data arrives, the temporal convolutional network inference is triggered, and the deep deterministic policy gradient network inference is triggered immediately after inference is completed, ensuring that control commands are generated before the start of the next control cycle. The total latency of the entire control chain includes sensor sampling latency, data fusion latency, neural network inference latency, and motor response latency. The total latency of neural network inference is less than 5 milliseconds, meeting the real-time requirement of 500 Hz for the flight control main loop.

[0168] S5. Set up a safety constraint layer to monitor the UAV's operating status and limit the speed control command when the status parameters exceed the safety threshold, and switch to the backup controller when the status parameters exceed the degradation threshold.

[0169] Furthermore, in S5, the step of limiting the speed control command when the state parameters exceed the safety threshold specifically includes:

[0170] Set a set of safety thresholds, which includes the upper limit of motor speed, the lower limit of motor speed, the upper limit of attitude angle, and the upper limit of attitude angular rate;

[0171] During real-time flight, the operating status parameters of the UAV are collected at a preset monitoring frequency. The operating status parameters include at least the current speed of each motor, the current attitude angle of the UAV, and the current attitude angular rate.

[0172] When the current speed exceeds the upper limit of the motor speed or falls below the lower limit of the motor speed, the speed control command is limited to between the upper limit and the lower limit of the motor speed.

[0173] When the current attitude angle exceeds the upper limit of the attitude angle, the speed control command is limited to restrict the further increase of the attitude angle;

[0174] When the current attitude angular rate exceeds the upper limit of the attitude angular rate, the speed control command is limited to restrict further increase of the attitude angular rate.

[0175] Furthermore, in S5, the specific steps for switching to the standby controller when the state parameters exceed the degradation threshold include:

[0176] Set a set of degradation thresholds, which includes the attitude angle over-limit duration threshold and the attitude angle rate over-limit threshold;

[0177] During real-time flight, the operating status parameters of the UAV are collected at a preset monitoring frequency. The operating status parameters include at least the current attitude angle and the current attitude angular rate of the UAV.

[0178] When the duration for which the current attitude angle exceeds the upper limit of the attitude angle reaches the attitude angle over-limit duration threshold, the current controller will be switched from the deep deterministic policy gradient network to the backup controller.

[0179] When the current attitude angular rate exceeds the attitude angular rate over-limit threshold, the current controller will be switched from the deep deterministic policy gradient network to the backup controller.

[0180] After switching to the backup controller, the operating status parameters of the UAV are continuously monitored. When the operating status parameters are within the set of degradation thresholds for a continuous preset time, the current controller is switched back from the backup controller to the deep deterministic policy gradient network.

[0181] Specifically, the safety threshold set includes upper and lower limits for motor speed, attitude angle, and attitude angular rate. The upper limit for motor speed is set based on the rated parameters of the UAV motor and electronic speed controller. For a typical quadcopter UAV, the upper limit is set to 8000 rpm, and the lower limit is set to 1000 rpm. The upper limit for attitude angle is set based on the UAV's flight stability boundaries. The upper limits for roll and pitch angles are both set to 45 degrees, and the upper limit for yaw angle is set to 180 degrees. The upper limit for attitude angular rate is set based on the range of the UAV's angular velocity sensor and flight stability requirements. The upper limits for roll and pitch rates are both set to 200 degrees per second, and the upper limit for yaw rate is set to 150 degrees per second.

[0182] During real-time flight, the UAV's operational status parameters are collected at a preset monitoring frequency. The monitoring frequency is consistent with the flight controller's main loop frequency, set to 500 Hz. The operational status parameters include at least the current speed of each motor, the current attitude angle of the UAV, and the current attitude angular rate. The current speed of each motor is obtained through the speed signal fed back by the electronic speed controller, the current attitude angle is obtained through attitude calculation by the inertial measurement unit, and the current attitude angular rate is obtained directly through measurement by the gyroscope of the inertial measurement unit.

[0183] When the current speed of any motor exceeds the upper limit or falls below the lower limit, a speed limiting process is performed on the speed control command output in step S4. Specifically, the speed control command exceeding the upper limit is forcibly set to the upper limit, and the speed control command falling below the lower limit is forcibly set to the lower limit. The limited speed control command then replaces the original command and is output to the motor drive module. This speed limiting process uses an independent speed limiting method for each motor; the speed control command for each motor is checked and processed independently.

[0184] When the current attitude angle of the UAV exceeds the upper limit of the attitude angle, a limiting process is performed on the rotation speed control command output in step S4 to restrict further increase in the attitude angle. The limiting process is as follows: the difference between the current attitude angle and the upper limit of the attitude angle is calculated, and a correction amount for the rotation speed control command is generated based on this difference. The correction direction is to induce an attitude recovery torque in the opposite direction for the UAV. The correction amount is calculated using proportional control, and the correction coefficient is set to 0.5 to ensure the smoothness of the limiting process. The corrected rotation speed control command is gradually withdrawn after the attitude angle recovers to within the upper limit range.

[0185] When the current attitude angular rate of the UAV exceeds the upper limit of the attitude angular rate, a limiting process is performed on the rotational speed control command output in step S4 to restrict further increase in the attitude angular rate. The specific method of the limiting process is as follows: calculate the difference between the current attitude angular rate and the upper limit of the attitude angular rate, and generate a correction amount for the rotational speed control command based on this difference. The correction direction is to cause the UAV to generate a damping torque opposite to the direction of the current angular rate. The correction amount is calculated using proportional control, and the correction coefficient is set to 0.3 to ensure that the angular rate quickly decays to a safe range.

[0186] The degradation threshold set includes the attitude angle overrun duration threshold and the attitude angular rate overrun threshold. The attitude angle overrun duration threshold is set to 0.3 seconds, based on the UAV's attitude recovery capability and safety redundancy requirements. The attitude angular rate overrun threshold is set based on the UAV's angular velocity sensor range and flight stability boundaries; for roll and pitch rates, the threshold is set to 250 degrees per second, and for yaw rate, it is set to 200 degrees per second. During real-time flight, the UAV's operational status parameters, including the current attitude angle and current attitude angular rate, are collected at the same monitoring frequency as in the clipping process. The data collection method is consistent with that used in the clipping process.

[0187] When the UAV's current attitude angle exceeds the upper limit, and the duration of this excess state reaches the attitude angle over-limit duration threshold of 0.3 seconds, a degradation switch is triggered. The specific method of degradation switch is as follows: the current controller is switched from a deep deterministic policy gradient network to a backup controller. The backup controller uses a proportional-integral-derivative controller, and its control parameters are pre-obtained through system identification and parameter tuning. During the switchover process, the current rotational speed control command is smoothly transitioned to the initial command output by the backup controller to avoid any shock during the switchover.

[0188] When the current attitude angular rate of the UAV exceeds the attitude angular rate over-limit threshold, a degradation switch is triggered immediately without waiting for the duration to accumulate. The degradation switch method is the same as the attitude angular rate over-limit duration trigger method, switching the current controller to the backup controller.

[0189] After switching to the backup controller, the drone's operational status parameters are continuously monitored. If these parameters remain within a set of degradation thresholds for a preset duration, the current controller is switched back to the deep deterministic policy gradient network. The preset duration is 2 seconds to ensure the drone's state is fully stable before reverting to the intelligent controller. During the switchback to the deep deterministic policy gradient network, the current rotation speed control command is smoothly transitioned to the command output by the network, and the recovery process also avoids any disruptions.

[0190] The safety constraint layer executes according to priority: first, motor speed limiting is performed, and this limiting is always in effect; second, attitude angle limiting and attitude angular rate limiting are performed, both in parallel; finally, degradation switching judgment is performed. Limiting processing and degradation switching are independent of each other. Limiting processing does not prevent degradation switching from being triggered, nor does degradation switching replace the function of limiting processing. When degradation switching is triggered, the speed control command output by the backup controller is also subject to the safety constraints of limiting processing, ensuring that the speed control command does not exceed the physical safety boundary under any control mode.

[0191] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks, characterized in that, Includes the following steps: S1. Construct a temporal convolutional network as a perturbation observer to estimate the total perturbation at the current moment based on the UAV state variables and wind speed measurements within a historical time window. The temporal convolutional network comprises a stacked structure of M residual blocks, each residual block including two dilated causal convolutional layers. The causal convolution of the dilated causal convolutional layers uses zero-padding on the left side of the convolution kernel to ensure that the output depends only on historical input and does not introduce future information. The dilated convolution of the dilated causal convolutional layers expands the receptive field by inserting d-1 zeros between the convolution kernel elements, where d is the dilation coefficient. The output of the temporal convolutional network is mapped to the total perturbation estimate via a fully connected layer. This represents the equivalent acceleration disturbance experienced by the UAV in the three axes of the body coordinate system at the current moment; S2. Construct a deep deterministic policy gradient network as a controller to generate speed control commands for each motor based on the current state error and the total disturbance. The estimated total disturbance is used as a feedforward compensation input, which, together with the state error, constitutes the input vector of the controller, forming a feedforward-feedback composite control structure. The feedforward compensation is configured to respond to instantaneous disturbances caused by sudden changes in wind speed, and the deep deterministic policy gradient network is configured to compensate for the residual error after the feedforward compensation based on a reinforcement learning strategy. S3. In a simulation environment, the temporal convolutional network and the deep deterministic policy gradient network are jointly trained by domain randomization until the convergence condition is met. S4. Deploy the trained temporal convolutional network and deep deterministic policy gradient network to the flight controller. During real-time flight, the temporal convolutional network estimates the total disturbance, and the deep deterministic policy gradient network generates speed control commands to drive the motor. S5. A safety constraint layer is set up to monitor the UAV's operating status and limit the rotation speed control command when the status parameters exceed a safety threshold. When the status parameters exceed a degradation threshold, the system switches to a backup controller. The limiting process and the degradation switching are triggered independently, and the limiting process has a preset priority and is always effective in any control mode. In response to the triggering of the degradation switching, the rotation speed control command is gradually transitioned to the initial rotation speed control command output by the backup controller. In response to meeting a preset recovery condition, the backup controller switches back to the deep deterministic policy gradient network, where the rotation speed control command is gradually transitioned.

2. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 1, characterized in that, In S1, the step of constructing a temporal convolutional network as a perturbation observer specifically includes: Collect state parameters and wind speed measurements of the drone during its flight; Based on the current moment, the state variables and wind speed measurements within a preset historical time window are extracted to form a time-series input matrix; The temporal input matrix is ​​input into a temporal convolutional network, which employs a causal convolutional structure to ensure that the output depends only on historical information, and uses an expanded convolutional structure to increase the receptive field. After passing through multiple layers of convolution and nonlinear activation operations of the temporal convolutional network, the total perturbation estimate at the current time is output.

3. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 2, characterized in that, In S2, the step of constructing a deep deterministic policy gradient network as a controller specifically includes: Obtain the expected state and actual state of the UAV at the current moment, and calculate the state error vector; The state error vector is concatenated with the total disturbance estimate to form the controller input vector; The controller input vector is input to a deep deterministic policy gradient network, which includes an action network and an evaluation network. The action network is used to generate action instructions based on the input, and the evaluation network is used to evaluate the value of the action to guide the action network to update. The motion network, after passing through multiple fully connected layers and nonlinear activation operations, outputs speed control commands for each motor. The dimension of the speed control commands matches the number of motors in the UAV.

4. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 3, characterized in that, The step of inputting the controller input vector into the deep deterministic policy gradient network specifically includes: The controller input vector is simultaneously input into the action network and the evaluation network; The action network extracts features from the controller input vector through multiple fully connected layers, and after processing by a nonlinear activation function, outputs speed control commands for each motor as action commands. The evaluation network performs joint feature extraction on the controller input vector and the action command output by the action network through multiple fully connected layers, and outputs the state-action value function corresponding to the current action command. The temporal difference error is calculated based on the state-action value function. The network parameters of the evaluation network are updated using the gradient descent method, and the network parameters of the action network are updated using the policy gradient method, so that the action commands output by the action network gradually approach the optimal control policy.

5. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 1, characterized in that, In step S3, the step of jointly training the temporal convolutional network and the deep deterministic policy gradient network in a simulation environment through domain randomization specifically includes: Build a drone simulation environment; Set a range of domain randomization parameters, which include wind field parameters and UAV dynamics parameters. At the beginning of each training round, randomly sample a set of parameter configurations from the range of parameters to generate diverse training scenarios. In each training round, the temporal convolutional network and the deep deterministic policy gradient network are deployed in the simulation environment to drive the UAV to perform dynamic hovering tasks and collect state data, disturbance estimates, control commands and reward signals. Based on the collected data, the network parameters of the temporal convolutional network are updated by minimizing the temporal difference error, and the network parameters of the deep deterministic policy gradient network are updated by the policy gradient method. Repeat the above steps until the hovering position error of the drone meets the preset convergence threshold under the preset verification scenario, at which point the joint training is considered complete.

6. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 5, characterized in that, The specific steps for setting the range of domain randomization parameters include: Set the randomization range for wind field parameters; Set the randomization range for the UAV dynamic parameters; At the beginning of each training round, a set of parameter values ​​are independently and randomly sampled from the randomization range of the wind field parameters and the randomization range of the UAV dynamic parameters to form the simulation environment configuration for the current round. The simulation environment configuration is loaded into the simulation environment so that the UAV is disturbed by the corresponding wind field parameters during the current round of flight and has the motion characteristics of the corresponding dynamic parameters.

7. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 1, characterized in that, In step S4, the step of deploying the trained temporal convolutional network and deep deterministic policy gradient network to the flight controller specifically includes: Export the trained temporal convolutional network and deep deterministic policy gradient network as model files in a preset format; The model file is loaded into the non-volatile memory of the flight controller, and the inference engines of the temporal convolutional network and the deep deterministic policy gradient network are initialized on the processor of the flight controller. During real-time flight, the state variables and wind speed measurements of the UAV are collected at a preset sampling frequency. The state variables and wind speed measurements within a preset historical time window before the current moment are input into a temporal convolutional network, and the total disturbance estimate at the current moment is obtained through forward inference calculation. The desired state and actual state of the UAV at the current moment are obtained, the state error vector is calculated, the state error vector is concatenated with the total disturbance estimate and then input into the deep deterministic policy gradient network, and the speed control command of each motor is obtained through forward inference calculation. The speed control command is output to the motor drive module of the flight controller, and the motor drive module generates a pulse width modulation signal to drive the UAV motor to operate.

8. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 1, characterized in that, In S5, the step of limiting the speed control command when the state parameter exceeds the safety threshold specifically includes: Set a set of safety thresholds, which includes an upper limit for motor speed, a lower limit for motor speed, an upper limit for attitude angle, and an upper limit for attitude angular rate; During real-time flight, the operating status parameters of the UAV are collected at a preset monitoring frequency. The operating status parameters include at least the current speed of each motor, the current attitude angle of the UAV, and the current attitude angular rate. When the current speed exceeds the upper limit of the motor speed or falls below the lower limit of the motor speed, the speed control command is subjected to a limiting process to restrict the speed control command between the upper limit of the motor speed and the lower limit of the motor speed. When the current attitude angle exceeds the upper limit of the attitude angle, the speed control command is subjected to a limiting process to restrict the further increase of the attitude angle; When the current attitude angular rate exceeds the upper limit of the attitude angular rate, the speed control command is subjected to a limiting process to restrict further increase in the attitude angular rate.

9. The method for dynamic hovering and disturbance rejection control of unmanned aerial vehicles based on neural networks according to claim 8, characterized in that, In step S5, the step of switching to the standby controller when the state parameters exceed the degradation threshold specifically includes: Set a set of degradation thresholds, which includes a threshold for the duration of attitude angle exceeding the limit and a threshold for the attitude angle rate exceeding the limit; During real-time flight, the operating status parameters of the UAV are collected at a preset monitoring frequency. The operating status parameters include at least the current attitude angle and the current attitude angular rate of the UAV. When the duration for which the current attitude angle exceeds the upper limit of the attitude angle reaches the attitude angle over-limit duration threshold, the current controller will be switched from the deep deterministic policy gradient network to the backup controller. When the current attitude angular rate exceeds the attitude angular rate over-limit threshold, the current controller will be switched from the deep deterministic policy gradient network to the backup controller. After switching to the backup controller, the operating status parameters of the UAV are continuously monitored. When the operating status parameters are within the range of the degradation threshold set for a continuous preset time, the current controller is switched from the backup controller back to the deep deterministic policy gradient network.

Images