Control method, system, device and medium for strong anti-interference hovering of tethered unmanned aerial vehicle

Abstract

The application discloses a kind of tethered unmanned aerial vehicle strong anti-interference hovering control method, system, equipment and medium, comprising: obtaining the real-time state data and desired state data of tethered unmanned aerial vehicle;Position data, velocity data, attitude angle data and desired position data are input to position controller, and desired pitch angle data, desired roll angle data and desired lift data are output;Desired pitch angle data, desired roll angle data, attitude angle data, angular velocity data and desired yaw angle are input to attitude controller, and desired moment data are output;According to desired lift data and desired moment data, determine current control instruction to control.The application introduces non-singular terminal sliding mode control strategy in position outer loop, to improve the rapidity and robustness of system, then combined with position loop intelligent agent model, realize the adaptive adjustment ability of control parameter under the condition of multi-source disturbance;Attitude inner loop introduces non-singular terminal sliding mode control strategy to improve accurate hovering capability.

Description

Technical Field

[0001] This invention relates to the field of aviation unmanned aerial vehicle (UAV) technology, and in particular to a control method, system, device, and medium for tethered UAVs with strong anti-interference hovering. Background Technology

[0002] Tethered drones are flight systems that are connected to a ground mobile platform via flexible cables. They have significant advantages such as continuous power supply and long-term flight time, and have been widely used in missions such as underwater mapping, disaster search and rescue, communication interruption, and emergency monitoring.

[0003] Unmanned aerial vehicle (UAV) systems typically operate at low altitudes of tens to hundreds of meters during missions. Airflow within this altitude range is significantly influenced by terrain conditions and atmospheric boundary layer effects, exhibiting complexity and time-varying characteristics. Tethered cables can sway or become excessively taut during flight, introducing various types of coupled disturbances, thus placing higher demands on the UAV system's anti-interference capabilities.

[0004] Current hovering control methods mostly employ PID or linear control strategies. While these can achieve basic hovering control under weak disturbances or rational environments, they often suffer from problems such as response hysteresis, large steady-state errors, and insufficient robustness under multi-source strong disturbances. Sliding mode control, due to its strong robustness to uncertainties and external disturbances, is gradually being applied to the field of UAV control. However, traditional sliding mode control suffers from chattering and has limited convergence speed. Although non-singular terminal sliding mode control can achieve finite-time convergence, its control parameters are usually determined empirically, making it difficult to adapt to complex and variable environments.

[0005] Therefore, there is an urgent need for control methods that can simultaneously improve the anti-interference capability and hovering capability of tethered UAVs in complex disturbance environments. Summary of the Invention

[0006] The technical problem to be solved by this invention is to overcome the shortcomings of existing tethered drones in terms of poor multi-source anti-disturbance capability and poor stable hovering capability in complex environments, and to provide a control method, system, device and medium for tethered drones with strong anti-disturbance hovering.

[0007] The present invention solves the above-mentioned technical problems through the following technical solution: In a first aspect, the present invention provides a control method for tethered unmanned aerial vehicles (UAVs) with strong anti-interference hovering capability, the control method comprising: Acquire real-time status data and desired status data of the tethered drone; the real-time status data includes position data, velocity data, attitude angle data and angular velocity data, and the desired status data includes desired position data and desired yaw angle data; The position data, velocity data, attitude angle data, and desired position data are input into a pre-built position controller, which outputs desired pitch angle data, desired roll angle data, and desired lift data. The position controller includes a position loop non-singular terminal sliding mode controller and a position loop agent model. The position loop agent model adopts the maximum entropy reinforcement learning algorithm suitable for the continuous action space MDP problem, with the optimization objective of maximizing cumulative reward while increasing policy entropy. In reinforcement learning training, wind disturbance, cable tension change, and external random disturbance are used as time-varying disturbance inputs. Random parameters are introduced to generate disturbance scenarios of different intensities and frequencies to enhance the generalization ability of the training task. The desired pitch angle data, the desired roll angle data, the attitude angle data, the angular velocity data, and the desired yaw angle are input to a pre-built attitude controller, and the desired torque data is output; the attitude controller includes an attitude loop non-singular terminal sliding mode controller; The current control command is determined based on the desired lift data and the desired torque data, so as to perform strong anti-disturbance hovering control on the tethered UAV using the current control command.

[0008] Preferably, the deep reinforcement learning network structure of the position loop agent model includes an action value network and a policy network, wherein the action value network is used to calculate the agent's state vector at time t. Execute agent actions at time The expected cumulative reward that can be obtained afterward; the policy network is used to determine the state vector at time t. Matching the corresponding agent actions .

[0009] Preferably, the position loop non-singular terminal sliding mode controller takes the desired three-axis hovering position, the current position of the tethered UAV, and the agent's action as inputs, and the desired three-axis acceleration as output; the attitude loop non-singular terminal sliding mode controller takes the desired attitude angle as input and the desired rotational torque as output.

[0010] Preferably, the step of inputting the position data, the velocity data, the attitude angle data, and the desired position data into a pre-built position controller, and outputting the desired pitch angle data, desired roll angle data, and desired lift data, includes: The position data, velocity data, and attitude angle data are input into the position loop agent model, and the agent action data is output. The agent's motion data, position data, velocity data, and desired position data are input to the position loop non-singular terminal sliding mode controller, which outputs the desired pitch angle data, desired roll angle data, and desired lift data.

[0011] Preferably, the mathematical expression for the position loop non-singular terminal sliding mode controller is: ; in, This indicates the three-axis output of the position loop controller; This indicates the error between the drone's current position and its desired position. This represents a three-axis non-singular terminal sliding surface; Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the action output of the position-loop agent model; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function; It represents the acceleration due to gravity.

[0012] Preferably, the mathematical expression for the attitude loop non-singular terminal sliding mode controller is: ; in, This indicates that the attitude loop controller is in the roll angle channel. Pitch angle channel and yaw angle channel The three-channel output; This represents the moment of inertia of the three axes of the drone; This indicates the error between the current attitude and the desired attitude of the drone; This represents the sliding mode surface of a three-channel non-singular terminal. Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function.

[0013] Preferably, the control method further includes: A dynamic model of the tethered UAV is constructed; the dynamic model includes the pose dynamic model of the tethered UAV and the dynamic model of the tethered cable; the pose dynamic model is used to determine the relationship between the three-axis position, velocity and attitude angle in the inertial coordinate system; the tethered cable dynamic model is used to discretize the cable into multiple micro-elements, and determine the motion relationship of each micro-element based on the mass of each micro-element, air resistance and tension of adjacent micro-elements; Based on the posture dynamics model and the tethered cable dynamics model, a position loop non-singular terminal sliding mode controller and an attitude loop non-singular terminal sliding mode controller are constructed.

[0014] Secondly, the present invention provides a control system for tethered unmanned aerial vehicles (UAVs) with strong anti-interference hovering capability, the control system comprising: The acquisition module is used to acquire real-time status data and expected status data of the tethered UAV; the real-time status data includes position data, velocity data, attitude angle data and angular velocity data, and the expected status data includes expected position data and expected yaw angle data; The first input module is used to input the position data, velocity data, attitude angle data, and desired position data into a pre-constructed position controller, and output desired pitch angle data, desired roll angle data, and desired lift data. The position controller includes a position loop non-singular terminal sliding mode controller and a position loop agent model. The position loop agent model adopts the maximum entropy reinforcement learning algorithm suitable for the continuous action space MDP problem, with the optimization objective of maximizing cumulative reward while improving policy entropy. In the reinforcement learning training, wind disturbance, cable tension change, and external random disturbance are used as time-varying disturbance inputs. Random parameters are introduced to generate disturbance scenarios of different intensities and frequencies to enhance the generalization ability of the training task. The second input module is used to input the desired pitch angle data, the desired roll angle data, the attitude angle data, the angular velocity data, and the desired yaw angle to a pre-built attitude controller and output the desired torque data; the attitude controller includes an attitude loop non-singular terminal sliding mode controller; The control module is used to determine the current control command based on the desired lift data and the desired torque data, so as to use the current control command to perform strong anti-disturbance hovering control on the tethered UAV.

[0015] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein the processor executes the computer program to implement the control method for strong anti-disturbance hovering of the tethered unmanned aerial vehicle described in the first aspect.

[0016] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the control method for tethered unmanned aerial vehicle with strong anti-disturbance hovering as described in the first aspect.

[0017] The positive and progressive effects of this invention are as follows: a non-singular terminal sliding mode control strategy is introduced into the position loop to improve the speed and robustness of the system. Combined with a deep reinforcement learning algorithm, the control parameters are adaptively adjusted under multi-source disturbance conditions through interactive learning between the agent and the environment. The desired attitude angle output by the outer position loop is used as the reference input for the inner loop. The inner loop uses a non-singular terminal sliding mode control method to generate attitude control quantities, driving the UAV to adjust its attitude to achieve precise hovering. This results in good adaptive capability and anti-interference performance, and can significantly improve the stable hovering effect of tethered UAVs in complex environments. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the first process of the control method for strong anti-disturbance hovering of a tethered unmanned aerial vehicle according to Embodiment 1 of the present invention.

[0019] Figure 2 This is a schematic diagram of the position loop agent model in the control method for strong anti-disturbance hovering of a tethered unmanned aerial vehicle according to Embodiment 1 of the present invention.

[0020] Figure 3 This is a schematic diagram of the second process of the control method for tethered unmanned aerial vehicle with strong anti-disturbance hovering in Embodiment 1 of the present invention.

[0021] Figure 4 This is a schematic diagram illustrating the application of the control method for tethered unmanned aerial vehicle with strong anti-disturbance hovering in Embodiment 1 of the present invention.

[0022] Figure 5 This is a schematic diagram of the first module of the control system for strong anti-disturbance hovering of a tethered unmanned aerial vehicle according to Embodiment 2 of the present invention.

[0023] Figure 6 This is a schematic diagram of the second module of the control system for strong anti-disturbance hovering of a tethered unmanned aerial vehicle according to Embodiment 2 of the present invention.

[0024] Figure 7 This is a schematic diagram of the electronic device according to Embodiment 3 of the present invention. Detailed Implementation

[0025] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.

[0026] Example 1: This embodiment provides a control method for tethered unmanned aerial vehicles (UAVs) with strong anti-disturbance hovering, such as... Figure 1 As shown, the control method includes: S11. Acquire real-time status data and desired status data of the tethered UAV; the real-time status data includes position data, velocity data, attitude angle data and angular velocity data, and the desired status data includes desired position data and desired yaw angle data; S12. Input the position data, velocity data, attitude angle data, and desired position data into the pre-built position controller, and output the desired pitch angle data, desired roll angle data, and desired lift data. The position controller includes a position loop non-singular terminal sliding mode controller and a position loop agent model. The position loop agent model adopts the maximum entropy reinforcement learning algorithm suitable for the continuous action space MDP problem, with the optimization objective of maximizing the cumulative reward while improving the policy entropy. In the reinforcement learning training, wind disturbance, cable tension change, and external random disturbance are used as time-varying disturbance inputs. Random parameters are introduced to generate disturbance scenarios with different intensities and frequencies to enhance the generalization ability of the training task. S13. Input the desired pitch angle data, desired roll angle data, attitude angle data, angular velocity data, and desired yaw angle into the pre-built attitude controller, and output the desired torque data; the attitude controller includes an attitude loop non-singular terminal sliding mode controller; S14. Determine the current control command based on the desired lift data and desired torque data, so as to use the current control command to perform strong anti-disturbance hovering control on the tethered UAV.

[0027] In this embodiment, the real-time state data includes the three-axis position, three-axis velocity, attitude angle, and angular velocity of the tethered UAV in the inertial coordinate system. The tethered UAV is a UAV connected to the ground platform via a flexible cable, capable of hovering for extended periods or flying at low speeds. A position-loop agent model is designed, defining its state space, action space, and reward function, and constructing a policy network and a value network structure.

[0028] A deeply enhanced position loop agent model outputs actions to a position loop non-singular terminal sliding mode controller to achieve online optimization and adaptive tuning of control parameters. An attitude controller based on the non-singular terminal sliding mode algorithm is designed. It receives the calculated desired attitude angle as input and outputs the three-axis rotational torque of the tethered UAV. Combining the position loop's calculated total lift and the attitude loop's output three-axis rotational torque, and using an allocation matrix to calculate the motor speed, a closed-loop control system for the tethered UAV is achieved.

[0029] In one embodiment, the deep reinforcement learning network structure of the location loop agent model includes an action value network and a policy network. The action value network is used to calculate the agent's state vector at time t. Execute agent actions at time The expected cumulative reward that can be obtained afterward; the policy network is used to determine the state vector at time t. Matching the corresponding agent actions .

[0030] In this embodiment, the position loop agent model employs the Soft Actor-Critic (SAC) algorithm, which is an off-policy deep reinforcement learning method within the maximum entropy reinforcement learning framework and is applicable to Markov Decision Processes (MDPs) in continuous action spaces. SAC introduces a maximum entropy policy optimization mechanism on top of the traditional Actor-Critic structure, maximizing cumulative rewards while encouraging sufficient randomness in the policy, thereby improving exploration capabilities and training stability.

[0031] The MDP problem is typically modeled as a quintuple. state space For state A set of objects; action space For action The set of states; state transition probabilities Indicates the state The agent performs actions At that time, the state at the next moment is The probability; instant reward function Indicates the state The agent performs actions The return function given by the environment; discount factor. Used to balance long-term and short-term gains.

[0032] The SAC algorithm maximizes both reward and policy entropy, and its optimization objective is: ; in, For strategy The optimization objective function; For expectation operation, it represents the average result under the state and action distribution; State-action pair follows policy Induced access distribution; For instant reward functions; For policy entropy; This is the temperature coefficient (entropy weight).

[0033] The SAC algorithm employs a four-core network structure, including a random policy network. The parameters are Input status The probability distribution of output actions; double network To alleviate the problem of Q-value overestimation; objective network Used for stable training. Critic is updated by minimizing the mean square Bellman error: ; in, The loss function for the action value network; For experience replay pool; The objective value for time difference is: ; Temperature parameters Adaptive adjustment: ; in, The objective function for optimizing the temperature parameters is... Indicates action From the current policy network Obtained by sampling.

[0034] The state space vector of an agent is defined as follows: ,in This refers to the positional error; ; For attitude angle; For integration weights. The action space vector is defined as... ,in, This is the adjustment amount for the gain coefficient of the three-axis sliding surface.

[0035] For hovering control of tethered drones, the environmental reward function can be divided into three parts: tracking accuracy, flight stability, and flight safety. Tracking accuracy directly propels the tethered drone toward the target point, achieving the core mission objective; flight stability ensures stable attitude and smooth movements, contributing to the feasibility and robustness of the control system; flight safety prevents dangerous behaviors such as excessive attitude angles or straying from the flight area, ensuring physical safety during training and real-world deployment.

[0036] Tracking accuracy is determined by pose, tracking accuracy, and velocity direction. For position tracking, the linear relationship between the reward value and the Euclidean distance is transformed into a nonlinear relationship, i.e., the relevant terms are designed using an exponential function, as follows: ; in, Rewards are given for accurate location tracking. To determine the positional error of the three axes of the tethered drone.

[0037] The pose tracking reward function is defined as follows: ; in, Rewards for attitude tracking accuracy; To determine the desired attitude angles of the three axes of a tethered drone; Real-time attitude angles of the three axes of a tethered drone.

[0038] Flight stability parameters prevent the agent from performing actions such as rapid, large-angle tilting or drastic acceleration / deceleration of the drone to quickly approach the target. They are defined as follows: ; in, Awarded for flight stability; The three-axis attitude angular velocity of the tethered drone.

[0039] Flight safety measures restrict extreme actions by the agent, as defined below: ; in, Penalties for flight safety; To mitigate the three-axis positional error of the tethered drone; Location of the ground mooring point; To track the real-time location of the drone; This refers to the length of the mooring rope.

[0040] Therefore, the reward function Represented as: ; In the SAC algorithm, it is necessary to compare the value functions of two actions with the same structure. (The parameters are respectively) ) and a policy function (Parameters are) Modeling is performed using two methods. To ensure training stability and numerical comparability, two... The network maintains the same input dimension, number of hidden layers, and activation function, but its parameters are independent. The policy network outputs the mean and variance of a Gaussian distribution based on the action space. For example... Figure 2 As shown, the input to the action value network in the position-loop agent model is a state vector. With action vectors The concatenation process passes through two fully connected hidden layers, each with 256 nodes and using the ReLU activation function, and finally outputs a scalar. The policy network takes a state vector as input, uses two fully connected hidden layers (128 nodes each) and employs the ReLU activation function, and outputs an action mean vector. Sum of logarithmic and standard deviation vectors The motion range is mapped to the Gaussian sampling method using the tanh function. Secondly, during reinforcement learning training, the position-loop agent model constructs a simulation environment containing multi-source disturbances. Wind disturbances, cable tension changes, and external random disturbances are modeled as time-varying disturbance inputs, and randomized parameters are introduced to generate disturbance scenarios of different intensities and frequencies to enhance the diversity and generalization ability of the training task. Simultaneously, by randomly sampling the disturbance type, amplitude, and duration during training, the position-loop agent model learns the adaptive adjustment rules of control parameters through continuous interaction with the environment.

[0041] In one embodiment, the position loop nonsingular terminal sliding mode controller takes the desired three-axis hovering position, the current position of the tethered UAV, and the agent's action as inputs, and the desired three-axis acceleration as output; the attitude loop nonsingular terminal sliding mode controller takes the desired attitude angle as input and the desired rotational torque as output.

[0042] In one embodiment, such as Figure 3 As shown, the control method also includes: S101. Construct a dynamic model of the tethered UAV; the dynamic model includes the pose dynamic model of the tethered UAV and the dynamic model of the tethered cable; the pose dynamic model is used to determine the relationship between the three-axis position, velocity and attitude angle in the inertial coordinate system; the tethered cable dynamic model is used to discretize the cable into multiple micro-elements, and determine the motion relationship of each micro-element based on the mass of each micro-element, air resistance and tension of adjacent micro-elements.

[0043] S102. Construct a position loop non-singular terminal sliding mode controller and an attitude loop non-singular terminal sliding mode controller based on the pose dynamics model and the tethered cable dynamics model.

[0044] In this embodiment, the data expression for the pose dynamics model of the tethered UAV is: ; in, These represent the three-axis positions and velocities of the UAV in the ground inertial coordinate system, respectively. For drone quality; This represents the magnitude of gravitational acceleration. ; This represents the rotation matrix of the tethered unmanned aerial vehicle (UAV) system to the ground inertial coordinate system, expressed as the roll angle. Pitch angle and yaw angle Characterization; This indicates the total lift generated by the drone's rotors. This indicates the tension at the connection point between the tethering cable and the drone; This represents the unit vector indicating the direction of cable tension in the ground-based inertial coordinate system. This indicates the angular velocity of the drone's rotation; express The cross product matrix; This represents the moment of inertia of the drone. This represents the rotational torque generated by the rotor. It represents the disturbance force and torque.

[0045] The mathematical expression for the dynamic model of the tethered cable is: ; in, Indicates the first The mass of a segment of cable micro-element; Indicates the first The second derivative of the position vector of the infinitesimal element of the cable segment; Represents the gravitational acceleration vector; Indicates the first The air resistance experienced by a segment of the cable; Indicates the first The tensile force on a small element of a rope segment; This indicates the number of segments into which the mooring rope is divided.

[0046] In one embodiment, such as Figure 3 As shown, step S12 includes: S121. Input the position data, velocity data and attitude angle data into the position loop agent model, and output the agent action data; S122. Input the agent's motion data, position data, velocity data and desired position data into the position loop non-singular terminal sliding mode controller, and output the desired pitch angle data, desired roll angle data and desired lift data.

[0047] In this embodiment, the internal parameters of the position loop non-singular terminal sliding mode controller are adaptively tuned online through a position loop agent model based on deep reinforcement learning, so as to adjust the agent's action data and achieve rapid convergence of position error and dynamic performance optimization of tethered UAV under multi-source disturbances.

[0048] Secondly, a non-singular terminal sliding mode control strategy is introduced into the position loop to improve the system's speed and robustness. Combined with deep reinforcement learning algorithms, the system achieves adjustment capability and anti-interference capability under multi-source disturbance conditions through interactive learning between the position loop agent model and the environment.

[0049] In one embodiment, the mathematical expression for the position loop non-singular terminal sliding mode controller is: ; in, This indicates the three-axis output of the position loop controller; This indicates the error between the drone's current position and its desired position. This represents a three-axis non-singular terminal sliding surface; Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the action output of the position-loop agent model; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function; It represents the acceleration due to gravity.

[0050] The mathematical expression for the attitude loop non-singular terminal sliding mode controller is: ; in, This indicates that the attitude loop controller is in the roll angle channel. Pitch angle channel and yaw angle channel The three-channel output; This represents the moment of inertia of the three axes of the drone; This indicates the error between the current attitude and the desired attitude of the drone; This represents the sliding mode surface of a three-channel non-singular terminal. Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function.

[0051] In one embodiment, such as Figure 4 As shown, the adaptive position control loop is composed of the SAC agent and the adaptive NTSMC controller, and the attitude control loop is composed of the NTSMC controller. The adaptive position control loop and the attitude control loop work together to improve the tethered UAV's multi-source anti-disturbance capability and stable hovering capability in complex environments.

[0052] In this embodiment, a control method for tethered UAV with strong anti-disturbance hovering is provided. A non-singular terminal sliding mode control strategy is introduced in the position outer loop to improve the system's speed and robustness. Combined with the position loop intelligent agent model, the adaptive adjustment capability of control parameters under multi-source disturbance conditions is realized. A non-singular terminal sliding mode control strategy is introduced in the attitude inner loop to improve the precise hovering capability.

[0053] Example 2: This embodiment provides a control system for tethered unmanned aerial vehicles (UAVs) with strong anti-interference hovering capability, such as... Figure 5 As shown, the control system includes: The acquisition module 310 is used to acquire real-time status data and expected status data of the tethered UAV; the real-time status data includes position data, velocity data, attitude angle data and angular velocity data, and the expected status data includes expected position data and expected yaw angle data. The first input module 320 is used to input position data, velocity data, attitude angle data and desired position data into the pre-built position controller, and output desired pitch angle data, desired roll angle data and desired lift data. The position controller includes a position loop non-singular terminal sliding mode controller and a position loop agent model. The position loop agent model adopts the maximum entropy reinforcement learning algorithm suitable for the continuous action space MDP problem, with the optimization objective of maximizing the cumulative reward while improving the policy entropy. In the reinforcement learning training, wind disturbance, cable tension change and external random disturbance are used as time-varying disturbance inputs. Random parameters are introduced to generate disturbance scenarios with different intensities and frequencies to enhance the generalization ability of the training task. The second input module 330 is used to input the desired pitch angle data, desired roll angle data, attitude angle data, angular velocity data and desired yaw angle to the pre-built attitude controller and output the desired torque data; the attitude controller includes an attitude loop non-singular terminal sliding mode controller; The control module 340 is used to determine the current control command based on the desired lift data and desired torque data, so as to use the current control command to perform strong anti-disturbance hovering control on the tethered UAV.

[0054] In this embodiment, the real-time state data acquired by the acquisition module 310 includes the three-axis position, three-axis velocity, attitude angle, and angular velocity of the tethered drone in the inertial coordinate system. The tethered drone is a drone connected to the ground platform via a flexible cable that can hover for a long time or fly at low speed. A position loop agent model is designed to define its state space, action space, and reward function, and a policy network and value network structure are constructed.

[0055] A deeply enhanced position loop agent model outputs actions to a position loop non-singular terminal sliding mode controller to achieve online optimization and adaptive tuning of control parameters. An attitude controller based on a non-singular terminal sliding mode algorithm is designed. It receives the calculated desired attitude angle as input and outputs the three-axis rotational torque of the tethered UAV. The control module 340 combines the position loop's calculated total lift and the attitude loop's output three-axis rotational torque, and calculates the motor speed using an allocation matrix to achieve a closed-loop control of the tethered UAV.

[0056] In one embodiment, the deep reinforcement learning network structure of the location loop agent model includes an action value network and a policy network. The action value network is used to calculate the agent's state vector at time t. Execute agent actions at time The expected cumulative reward that can be obtained afterward; the policy network is used to determine the state vector at time t. Matching the corresponding agent actions .

[0057] In this embodiment, the position loop agent model employs the Deep Soft Actor-Critic (SAC) algorithm, which is an off-policy deep reinforcement learning method within the maximum entropy reinforcement learning framework, suitable for Markov Decision Processes (MDPs) in continuous action spaces. SAC introduces a maximum entropy policy optimization mechanism on top of the traditional Actor-Critic structure, maximizing cumulative reward while encouraging sufficient randomness in the policy, thereby improving exploration ability and training stability. The MDP problem is typically modeled as a quintuple. state space For state A set of objects; action space For action The set of states; state transition probabilities Indicates the state The agent performs actions At that time, the state at the next moment is The probability; instant reward function Indicates the state The agent performs actions The return function given by the environment; discount factor. Used to balance long-term and short-term gains.

[0058] The SAC algorithm maximizes both reward and policy entropy, and its optimization objective is: ; in, For strategy The optimization objective function; For expectation operation, it represents the average result under the state and action distribution; State-action pair follows policy Induced access distribution; For instant reward functions; For policy entropy; This is the temperature coefficient (entropy weight).

[0059] The SAC algorithm employs a four-core network structure, including a random policy network. The parameters are Input status The probability distribution of output actions; double network To alleviate the problem of Q-value overestimation; objective network Used for stable training. Critic is updated by minimizing the mean square Bellman error: ; in, The loss function for the action value network; For experience replay pool; This is the time difference objective value. Its objective value is: ; Temperature parameters Adaptive adjustment: ; in, The objective function for optimizing the temperature parameters is... Indicates action From the current policy network Obtained by sampling.

[0060] The state space vector of an agent is defined as follows: ,in This refers to the positional error; ; For attitude angle; For integration weights. The action space vector is defined as... ,in, This is the adjustment amount for the gain coefficient of the three-axis sliding surface.

[0061] For hovering control of tethered drones, the environmental reward function can be divided into three parts: tracking accuracy, flight stability, and flight safety. Tracking accuracy directly propels the tethered drone toward the target point, achieving the core mission objective; flight stability ensures stable attitude and smooth movements, contributing to the feasibility and robustness of the control system; flight safety prevents dangerous behaviors such as excessive attitude angles or straying from the flight area, ensuring physical safety during training and real-world deployment.

[0062] Tracking accuracy is determined by pose, tracking accuracy, and velocity direction. For position tracking, the linear relationship between the reward value and the Euclidean distance is transformed into a nonlinear relationship, i.e., the relevant terms are designed using an exponential function, as follows: ; in, Rewards are given for accurate location tracking. To determine the positional error of the three axes of the tethered drone.

[0063] The pose tracking reward function is defined as follows: ; in, Rewards for attitude tracking accuracy; To determine the desired attitude angles of the three axes of a tethered drone; Real-time attitude angles of the three axes of a tethered drone.

[0064] Flight stability parameters prevent agents from performing actions such as rapid, large-angle tilting or drastic acceleration / deceleration of the drone to quickly approach the target. They are defined as follows: ; in, As a reward for flight stability; The three-axis attitude angular velocity of the tethered drone.

[0065] Flight safety measures restrict extreme actions by the agent, as defined below: ; in, Penalties for flight safety; To mitigate the three-axis positional error of the tethered drone; Location of the ground mooring point; To track the real-time location of the drone; This refers to the length of the mooring rope.

[0066] Therefore, the reward function Represented as: ; In the SAC algorithm, it is necessary to compare the value functions of two actions with the same structure. (The parameters are respectively) ) and a policy function (Parameters are) Modeling is performed using two methods. To ensure training stability and numerical comparability, two... The network maintains the same input dimension, number of hidden layers, and activation function, but its parameters are independent. The policy network outputs the mean and variance of a Gaussian distribution based on the action space. In the position-loop agent model, the action-value network takes a state vector as its input. With action vectors The concatenation process passes through two fully connected hidden layers, each with 256 nodes and using the ReLU activation function, and finally outputs a scalar. The policy network takes a state vector as input, uses two fully connected hidden layers (128 nodes each) and employs the ReLU activation function, and outputs an action mean vector. Sum of logarithmic and standard deviation vectors The motion range is mapped to the Gaussian sampling method using the tanh function. Secondly, during reinforcement learning training, the position-loop agent model constructs a simulation environment containing multi-source disturbances. Wind disturbances, cable tension changes, and external random disturbances are modeled as time-varying disturbance inputs, and randomized parameters are introduced to generate disturbance scenarios of different intensities and frequencies to enhance the diversity and generalization ability of the training task. Simultaneously, by randomly sampling the disturbance type, amplitude, and duration during training, the position-loop agent model learns the adaptive adjustment rules of control parameters through continuous interaction with the environment.

[0067] In one embodiment, the position loop nonsingular terminal sliding mode controller takes the desired three-axis hovering position, the current position of the tethered UAV, and the agent's action as inputs, and the desired three-axis acceleration as output; the attitude loop nonsingular terminal sliding mode controller takes the desired attitude angle as input and the desired rotational torque as output.

[0068] In one embodiment, such as Figure 6 As shown, the control system also includes: The first construction module 301 is used to construct the dynamic model of the tethered UAV. The dynamic model includes the pose dynamic model of the tethered UAV and the dynamic model of the tethered cable. The pose dynamic model is used to determine the relationship between the three-axis position, velocity and attitude angle in the inertial coordinate system. The tethered cable dynamic model is used to discretize the cable into multiple micro-elements and determine the motion relationship of each micro-element based on the mass of each micro-element, air resistance and tension of adjacent micro-elements.

[0069] The second construction module 302 is used to construct a position loop non-singular terminal sliding mode controller and an attitude loop non-singular terminal sliding mode controller based on the pose dynamics model and the tethered cable dynamics model.

[0070] In this embodiment, the data expression of the pose dynamics model of the tethered UAV in the first construction module 301 is as follows: ; in, These represent the three-axis positions and velocities of the UAV in the ground inertial coordinate system, respectively. For drone quality; Indicates the magnitude of gravitational acceleration; ; This represents the rotation matrix of the tethered unmanned aerial vehicle (UAV) system to the ground inertial coordinate system, expressed as the roll angle. Pitch angle and yaw angle Characterization; This indicates the total lift generated by the drone's rotors. This indicates the tension at the connection point between the tethering cable and the drone; This represents the unit vector indicating the direction of cable tension in the ground-based inertial coordinate system. This indicates the angular velocity of the drone's rotation; express The cross product matrix; This represents the moment of inertia of the drone. This represents the rotational torque generated by the rotor. It represents the disturbance force and torque.

[0071] The dynamic model of the tethered cable in the first building module 301 is as follows: ; in, Indicates the first The mass of a segment of cable micro-element; Indicates the first The second derivative of the position vector of the infinitesimal element of the cable segment; Represents the gravitational acceleration vector; Indicates the first The air resistance experienced by a segment of the cable; Indicates the first The tensile force on a small element of a rope segment; This indicates the number of segments into which the mooring rope is divided.

[0072] In one embodiment, such as Figure 6 As shown, the first input module 320 includes: The first input unit 321 is used to input position data, velocity data and attitude angle data into the position loop agent model and output agent action data; The second input unit 322 is used to input the agent's motion data, position data, velocity data and desired position data to the position loop non-singular terminal sliding mode controller, and output desired pitch angle data, desired roll angle data and desired lift data.

[0073] In this embodiment, the first input unit 321 and the second input unit 322 use a position loop agent model based on deep reinforcement learning to perform online adaptive tuning of the parameters of the position loop non-singular terminal sliding mode controller, so as to adjust the agent action data and realize the rapid convergence of position error and dynamic performance optimization of the tethered UAV under multi-source disturbance.

[0074] Secondly, a non-singular terminal sliding mode control strategy is introduced into the position loop to improve the system's speed and robustness. Combined with deep reinforcement learning algorithms, the system achieves adjustment capability and anti-interference capability under multi-source disturbance conditions through interactive learning between the position loop agent model and the environment.

[0075] In one embodiment, the mathematical expression for the position loop non-singular terminal sliding mode controller is: ; in, This indicates the three-axis output of the position loop controller; This indicates the error between the drone's current position and its desired position. This represents a three-axis non-singular terminal sliding surface; Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the action output of the position-loop agent model; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function; It represents the acceleration due to gravity.

[0076] The mathematical expression for the attitude loop non-singular terminal sliding mode controller is: ; in, This indicates that the attitude loop controller is in the roll angle channel. Pitch angle channel and yaw angle channel The three-channel output; This represents the moment of inertia of the three axes of the drone; This indicates the error between the current attitude and the desired attitude of the drone; This represents the sliding mode surface of a three-channel non-singular terminal. Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function.

[0077] In this embodiment, a control system for tethered UAV with strong anti-disturbance hovering is provided. A non-singular terminal sliding mode control strategy is introduced in the position outer loop to improve the system's speed and robustness. Combined with the position loop intelligent agent model, the system can adaptively adjust the control parameters under multi-source disturbance conditions. A non-singular terminal sliding mode control strategy is introduced in the attitude inner loop to achieve precise hovering.

[0078] Example 3: Figure 7This is a schematic diagram of the structure of an electronic device provided in this embodiment. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the control method for strong anti-disturbance hovering of the tethered UAV in Embodiment 1. Figure 7 The electronic device 90 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0079] like Figure 7 As shown, the electronic device 90 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including memory 92 and processor 91).

[0080] Bus 93 includes a data bus, an address bus, and a control bus.

[0081] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0082] The memory 92 may also include a program / utility 925 having a set (at least one) of program modules 924, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0083] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92, such as the control method for tethered unmanned aerial vehicle with strong anti-disturbance hovering in Embodiment 1 of the present invention.

[0084] Electronic device 90 can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 95. Furthermore, the model-generated electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 96. Figure 7 As shown, network adapter 96 communicates with other modules of the model-generated electronic device 90 via bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0085] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0086] Example 4: This embodiment provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps in the control method for strong anti-disturbance hovering of a tethered UAV in Embodiment 1.

[0087] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0088] In a possible implementation, the present invention can also be implemented as a program product comprising program code, which, when the program product is run on a terminal device, causes the terminal device to execute the steps in the control method for tethered UAV strong anti-disturbance hovering of Embodiment 1.

[0089] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0090] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.

Claims

1. A control method for tethered unmanned aerial vehicles (UAVs) with strong anti-disturbance hovering, characterized in that, The control method includes: Acquire real-time status data and desired status data of the tethered drone; the real-time status data includes position data, velocity data, attitude angle data and angular velocity data, and the desired status data includes desired position data and desired yaw angle data; The position data, velocity data, attitude angle data, and desired position data are input into a pre-built position controller, which outputs desired pitch angle data, desired roll angle data, and desired lift data. The position controller includes a position loop non-singular terminal sliding mode controller and a position loop agent model. The position loop agent model adopts the maximum entropy reinforcement learning algorithm suitable for the continuous action space MDP problem, with the optimization objective of maximizing cumulative reward while increasing policy entropy. In reinforcement learning training, wind disturbance, cable tension change, and external random disturbance are used as time-varying disturbance inputs. Random parameters are introduced to generate disturbance scenarios of different intensities and frequencies to enhance the generalization ability of the training task. The desired pitch angle data, the desired roll angle data, the attitude angle data, the angular velocity data, and the desired yaw angle are input to a pre-built attitude controller, and the desired torque data is output; the attitude controller includes an attitude loop non-singular terminal sliding mode controller; The current control command is determined based on the desired lift data and the desired torque data, so as to perform strong anti-disturbance hovering control on the tethered UAV using the current control command.

2. The control method for tethered unmanned aerial vehicles with strong anti-disturbance hovering as described in claim 1, characterized in that, The deep reinforcement learning network structure of the position loop agent model includes an action value network and a policy network. The action value network is used to calculate the agent's state vector at time t. Execute agent actions at time The expected cumulative reward that can be obtained afterward; the policy network is used to determine the state vector at time t. Matching the corresponding agent actions .

3. The control method for tethered unmanned aerial vehicles with strong anti-disturbance hovering as described in claim 2, characterized in that, The position loop non-singular terminal sliding mode controller takes the desired three-axis hovering position, the current position of the tethered UAV, and the agent's actions as inputs, and the desired three-axis acceleration as output; the attitude loop non-singular terminal sliding mode controller takes the desired attitude angle as input and the desired rotational torque as output.

4. The control method for tethered unmanned aerial vehicles with strong anti-disturbance hovering as described in claim 2, characterized in that, The step of inputting the position data, velocity data, attitude angle data, and desired position data into a pre-built position controller, and outputting desired pitch angle data, desired roll angle data, and desired lift data includes: The position data, velocity data, and attitude angle data are input into the position loop agent model, and the agent action data is output. The agent's motion data, position data, velocity data, and desired position data are input to the position loop non-singular terminal sliding mode controller, which outputs the desired pitch angle data, desired roll angle data, and desired lift data.

5. The control method for tethered unmanned aerial vehicles with strong anti-disturbance hovering as described in claim 2, characterized in that, The mathematical expression for the position loop non-singular terminal sliding mode controller is: ； in, This indicates the three-axis output of the position loop controller; This indicates the error between the drone's current position and its desired position. This represents a three-axis non-singular terminal sliding surface; Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the action output of the position-loop agent model; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function; It represents the acceleration due to gravity.

6. The control method for tethered unmanned aerial vehicles with strong anti-disturbance hovering as described in claim 2, characterized in that, The mathematical expression for the attitude loop non-singular terminal sliding mode controller is: ； in, This indicates that the attitude loop controller is in the roll angle channel. Pitch angle channel and yaw angle channel The three-channel output; This represents the moment of inertia of the three axes of the drone; This indicates the error between the current attitude and the desired attitude of the drone; This represents the sliding mode surface of a three-channel non-singular terminal. Indicates the parameters of the sliding surface; Indicates the exponential parameter; Indicates the approximation gain; Indicates the tendency index; This represents the constant robust gain that compensates for bounded external disturbances and model uncertainties. Represents a symbolic function.

7. The control method for tethered unmanned aerial vehicles with strong anti-disturbance hovering as described in claim 1, characterized in that, The control method further includes: A dynamic model of the tethered UAV is constructed; the dynamic model includes the pose dynamic model of the tethered UAV and the dynamic model of the tethered cable; the pose dynamic model is used to determine the relationship between the three-axis position, velocity and attitude angle in the inertial coordinate system; the tethered cable dynamic model is used to discretize the cable into multiple micro-elements, and determine the motion relationship of each micro-element based on the mass of each micro-element, air resistance and tension of adjacent micro-elements; Based on the posture dynamics model and the tethered cable dynamics model, a position loop non-singular terminal sliding mode controller and an attitude loop non-singular terminal sliding mode controller are constructed.

8. A control system for tethered unmanned aerial vehicles (UAVs) with strong anti-interference hovering capability, characterized in that, The control system includes: The acquisition module is used to acquire real-time status data and expected status data of the tethered UAV; the real-time status data includes position data, velocity data, attitude angle data and angular velocity data, and the expected status data includes expected position data and expected yaw angle data; The first input module is used to input the position data, velocity data, attitude angle data, and desired position data into a pre-constructed position controller, and output desired pitch angle data, desired roll angle data, and desired lift data. The position controller includes a position loop non-singular terminal sliding mode controller and a position loop agent model. The position loop agent model adopts the maximum entropy reinforcement learning algorithm suitable for the continuous action space MDP problem, with the optimization objective of maximizing cumulative reward while improving policy entropy. In the reinforcement learning training, wind disturbance, cable tension change, and external random disturbance are used as time-varying disturbance inputs. Random parameters are introduced to generate disturbance scenarios of different intensities and frequencies to enhance the generalization ability of the training task. The second input module is used to input the desired pitch angle data, the desired roll angle data, the attitude angle data, the angular velocity data, and the desired yaw angle to a pre-built attitude controller and output the desired torque data; the attitude controller includes an attitude loop non-singular terminal sliding mode controller; The control module is used to determine the current control command based on the desired lift data and the desired torque data, so as to use the current control command to perform strong anti-disturbance hovering control on the tethered UAV.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the control method for tethered unmanned aerial vehicle with strong anti-disturbance hovering as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the control method for tethered unmanned aerial vehicles with strong anti-disturbance hovering as described in any one of claims 1-7.

Images