Method and system for safe and flexible lifting of unmanned aerial vehicle based on vision

Abstract

The invention discloses a vision-based unmanned aerial vehicle safe and flexible lifting method and system, and the method comprises the steps: S1, obtaining an initial image corresponding to a target in real time through a target detection frame, and carrying out the preprocessing of the initial image to obtain an optimized image sequence; s2, converting the pixel coordinate into a sight line direction vector, and converting the sight line direction vector into a navigation coordinate system in combination with a real-time attitude angle of the unmanned aerial vehicle; solving a swing state vector; s3, constructing a current flight state vector, and generating a corresponding expected flight path state vector; s4, adjusting the attitude of the unmanned aerial vehicle according to the attitude compensation control quantity until the expected flight path state vector is met; and S5, releasing the load when the lightweight flexible release condition is met. According to the invention, attitude control and release conditions during flexible lifting of the unmanned aerial vehicle are optimized, the load swing amplitude is reduced in the main lifting stage, and the flight stability is improved; natural release is achieved only when the release condition is met in the release stage, and safety, simplicity, convenience and reliability are achieved.

Description

Technical Field

[0001] This invention relates to the field of collaborative control technology for unmanned aerial vehicles (UAVs), specifically to a vision-based method and system for safe and flexible lifting of UAVs. Background Technology

[0002] With the continuous development of drone technology, its application in lifting operations in logistics, engineering hoisting, and emergency rescue is becoming increasingly widespread. During hoisting operations, the load is usually suspended below the drone by a flexible cable. During takeoff, acceleration, turning, and landing, the load is prone to swaying.

[0003] However, this type of swaying can cause the following problems: on the one hand, load swaying can significantly reduce the accuracy of deployment or positioning and prolong the operation time; on the other hand, the additional disturbances generated by the swaying can affect the flight attitude stability of the UAV, and in extreme cases, may even cause flight instability or safety accidents.

[0004] Currently, common technical solutions to the problem of load swaying during drone lifting mainly include the following categories: 1) Manual experience-based compensation method: The operator visually observes the load swing and manually performs reverse control compensation on the drone. This method is highly dependent on the operator's experience, the control effect is highly subjective, and it is difficult to maintain a stable and consistent sway suppression effect under complex working conditions.

[0005] 2) Passive flight control parameter optimization method: This method enhances the system's ability to suppress disturbances by adjusting the control parameters in the UAV flight controller. However, this method has a slow response time and is difficult to adaptively adjust for loads of different weights, shapes, or mounting methods, resulting in limited overall sway suppression effect.

[0006] 3) Independent control method for intelligent pods: Some solutions propose integrating independent sensors and control units inside the pod to achieve proactive sensing and control of the load status. However, this type of solution usually significantly increases the structural complexity, weight, and manufacturing cost of the pod, while also increasing the difficulty of system maintenance, which is not conducive to engineering application and promotion.

[0007] In summary, existing technologies for suppressing load swing during UAV lifting still suffer from problems such as insufficient perception accuracy, poor control coordination, high system complexity, or excessive cost. There is an urgent need for a new technical solution that can ensure control effectiveness while also considering system simplification and engineering feasibility. Summary of the Invention

[0008] To address the aforementioned problems, the present invention aims to propose a vision-based method and system for safe and flexible unmanned aerial vehicle (UAV) lifting operations. This method utilizes forward-looking and downward-looking cameras to detect and track the load in real time, and calculates the load's swing angle and angular velocity relative to the UAV using target pixel information. Based on this, the system constructs a comprehensive state variable that integrates the UAV's flight state and the load's swing state. By designing a cooperative control law, attitude compensation is generated to actively suppress load swing while tracking the desired flight trajectory. When the load is stable and meets safety conditions, the system triggers a gravity-release dispenser to achieve low-impact, flexible delivery. This method forms a complete closed loop from perception, calculation, cooperative control to safe delivery, significantly improving the safety, stability, and delivery accuracy of UAV lifting operations in complex environments.

[0009] This was achieved through the following technical solutions: First, a vision-based method for safe and flexible lifting by unmanned aerial vehicles (UAVs) is proposed, characterized by the following steps: S1. The camera mounted on the drone treats the payload as the target and acquires the initial image corresponding to the target in real time through the target detection box; the initial image is preprocessed to obtain an optimized image sequence; the preprocessing includes: using non-maximum suppression to filter to obtain the final target box and extracting the center pixel coordinates, and performing Kalman filtering; the center pixel coordinates are in the camera coordinate system; S2. Based on the camera's intrinsic parameter matrix, first convert the pixel coordinates in the image sequence into the line-of-sight vector of the load relative to the UAV's body coordinate system. Then, combined with the UAV's real-time attitude angle, convert the line-of-sight vector into the navigation coordinate system. Next, calculate the load's swing angle and swing angular velocity to obtain the swing state vector. S3. Construct the current flight state vector of the UAV, and generate the corresponding expected flight trajectory state vector based on the mission plan of the ground station. S4. Define the current integrated state based on the current flight state vector and the sway state vector, and then obtain the attitude compensation control quantity based on the desired flight trajectory state vector. Adjust the attitude of the UAV according to the attitude compensation control quantity until the desired flight trajectory state vector is met. S5. When the lifting meets the conditions for lightweight and flexible deployment, the drone will naturally release the load.

[0010] Optionally, before acquiring the initial image in step S1, the load is monitored in real time based on the YOLOv5 object detection algorithm, and the target bounding box is output. ,in, This indicates the center position of the load in the coordinate system of the initial image. This indicates the width and height in the target dimensions. s i Indicates the confidence level. ci Indicates the target category. k This indicates the current period. The YOLOv5 target detection algorithm enables high-precision, high-efficiency real-time target detection, ensuring that the load is quickly and reliably identified in various environments.

[0011] Optionally, during the non-maximum suppression screening in step S1, the detection confidence of the target candidate box is recorded as... s (k) The target detection confidence threshold is denoted as s th ; to satisfy each s(k) ≥ s th The detected bounding boxes are considered valid and used as the corresponding final bounding boxes; the horizontal and vertical pixel coordinates of the target in the final bounding box are denoted as follows: Then the center pixel coordinates are , T This indicates transpose. By setting a confidence threshold, noise and false positives are effectively filtered out, improving the reliability of target detection. Non-maximum suppression eliminates redundant overlapping boxes, ensuring that each load target corresponds to only one optimal bounding box, laying the foundation for accurate coordinate extraction in the future.

[0012] Optionally, after obtaining the center pixel coordinates, the pixel state vector is defined as follows: ,in, This represents the rate of change of the center pixel coordinates of the target within the image plane.

[0013] Optionally, in step S2, the intrinsic parameter matrix is ​​denoted as K Then the line-of-sight vector in the camera coordinate system is The superscript -1 denotes the inverse matrix; the rotation matrix from the camera coordinate system to the UAV body coordinate system is denoted as... Then the line-of-sight vector in the UAV body coordinate system is The real-time attitude angle of the UAV is denoted as... ,in, ϕ Indicates the roll angle of the drone. i Indicates the pitch angle of the drone. ψ This indicates the UAV's heading. Back-projection using camera intrinsic parameters is a crucial step in establishing the mapping from 2D image points to 3D spatial directions, ensuring the accuracy of geometric relationships. By combining the camera mounting matrix and the UAV's attitude, the line-of-sight vector is unified to the body and navigation coordinate systems, providing a unified mathematical foundation for solving and controlling the load state at the full spatial scale.

[0014] Optionally, in step S2, when calculating the swing angle and angular velocity of the load, the numerical direction unit vector of the machine system coordinate system is denoted as... align the target's line of sight with By comparison, the swing angle of the load is defined as... ; then Discrete difference calculations are performed to obtain a preliminary estimate of the oscillation angular velocity. Δ t The time interval between two adjacent sampling moments is defined as follows: Then, the swing state vector is constructed. .

[0015] Optionally, in step S3, the current flight state vector is ,in, p u (k) This represents the position vector of the UAV in the navigation coordinate system. This represents the velocity vector of the UAV in the navigation coordinate system. the(k) This represents the Euler angle vector representing the attitude of the UAV. ω(k) This represents the angular velocity vector of the UAV; the desired flight trajectory vector is... ,in, p d (k) Indicates the desired position. v d (k) Indicates the desired speed. or d (k) This represents the desired attitude. It defines a complete flight state vector, providing the control system with comprehensive feedback on the UAV's own attitude and motion.

[0016] Optionally, the attitude compensation control quantity in step S4 is ,in, This indicates the amount of attitude compensation in the roll direction. This indicates the pitch attitude compensation amount. This indicates the angle of sway of the load relative to the vertical direction of the drone. Indicates the angular velocity of the load swing. K s The sway control gain matrix is ​​used to superimpose the attitude compensation amount onto the desired attitude. ,Will The data is sent to the ground station control module to control the UAV to adjust its attitude. The load swing angle and angular velocity are used as direct inputs to generate inverse attitude compensation commands, realizing active damping suppression of the swing dynamics. The compensation is superimposed on the desired attitude, enabling the UAV to reduce load swing through its own active attitude adjustment, achieving deep coordination between flight control and load swing suppression.

[0017] Optionally, the lightweight flexible deployment conditions in step S5 include: meeting the following requirements. s(k) ≥s th The load sway condition meets the sway suppression condition; the drone's altitude, speed, and attitude meet the preset safe delivery requirements; among these, a maximum load angle threshold is set for the load sway. Maximum load swing angular velocity threshold The swing suppression condition is By setting multiple safety conditions for joint judgment, the deployment action is ensured to be triggered only at absolutely safe moments when the load is stable and the drone is in a stable state, greatly reducing the risk of accidents.

[0018] Secondly, a vision-based UAV safety flexible hoisting system is proposed to execute the above method. The system includes a UAV flight platform, a hoisting mechanism, a vision perception and ground station control module, a positioning module, and a power module. The UAV flight platform transports the load and acquires images of the load through the hoisting mechanism. The vision perception and ground station control module controls the attitude of the UAV flight platform based on the load images. The positioning module provides the position information of the UAV flight platform to the vision perception and ground station control module or the onboard computing unit. The power module provides flight power and performs attitude adjustment.

[0019] The beneficial effects of this invention compared to the prior art are: This invention utilizes a camera mounted on a drone to detect and track the hoisted load in real time, calculating the load's swing angle and angular velocity relative to the drone using target pixel information. Based on this, a comprehensive state variable integrating the drone's flight state and the load's swing state is constructed. Through coordinated control design, attitude compensation is generated to actively suppress load swing while tracking the desired flight trajectory. Furthermore, a gravity-release dispenser is triggered only when the load is stable and meets the delivery conditions, achieving a low-impact, flexible delivery. This method forms a complete closed loop from perception, calculation, and coordinated control to safe delivery, significantly improving the safety, stability, and delivery accuracy of drone hoisting operations in complex environments. Attached Figure Description

[0020] Figure 1 A flowchart of a vision-based method for safe and flexible lifting by unmanned aerial vehicles; Figure 2 This is a schematic diagram of a device for the safe and flexible lifting of drones. Detailed Implementation

[0021] The following will be based on embodiments of the present invention. Figure 1 and Figure 2 The technical solutions in the embodiments of the present invention will be described in detail below.

[0022] like Figure 1The diagram shows a flowchart of a vision-based method for safe and flexible lifting of a drone. The method uses onboard vision to perceive and calculate the load sway state in real time, integrates it with the drone's flight state, and generates active attitude compensation commands. While tracking the desired trajectory, it suppresses load sway, forming a sway suppression closed loop of perception-calculation-control. Furthermore, when the load is stable and meets multiple deployment conditions, it triggers a natural release of low-impact gravity, achieving autonomous control of the entire process from stable lifting to safe deployment. This method is suitable for safe operation in complex environments.

[0023] The method specifically includes the following steps: S1. During the hoisting process, the camera mounted on the UAV treats the load as the target and acquires the initial image corresponding to the target in real time through the target detection box; the initial image is preprocessed to obtain an optimized image sequence; the preprocessing includes: using non-maximum suppression to filter to obtain the final target box and extracting the center pixel coordinates, and performing Kalman filtering; the center pixel coordinates are in the camera coordinate system.

[0024] In this embodiment, when the load is considered as the target, the load is monitored in real time based on the YOLOv5 target detection algorithm, and the target bounding box is output. ,in, This indicates the center position of the load in the coordinate system of the initial image. This indicates the width and height in the target dimensions. s i Indicates the confidence level. c i Indicates the target category. k This represents the current period (discrete-time index), where i is the number of targets (i=1, 2, 3...). The YOLOv5 target detection algorithm enables high-precision and high-efficiency real-time target detection, ensuring that the target is quickly and reliably identified in various environments.

[0025] Then, during non-maximum suppression screening, the detection confidence of the target candidate box is denoted as... s(k) The target detection confidence threshold is denoted as s th ; to satisfy each s(k) ≥ s th The detected bounding boxes are considered valid and used as the corresponding final bounding boxes. The horizontal and vertical pixel coordinates of the target in the final bounding box are denoted as follows: Then the center pixel coordinates are , TThis represents the transpose (i.e., a column vector consisting of two pixel coordinates). By setting a confidence threshold, noise and false positives are effectively filtered out, improving the reliability of target detection; non-maximum suppression eliminates redundant overlapping boxes, ensuring that each target corresponds to only one optimal bounding box, laying the foundation for accurate coordinate extraction later.

[0026] To obtain continuous and smooth pixel trajectories, a Kalman filter is used to track the center pixel coordinates, and the pixel state vector is defined as follows: ,in, This represents the rate of change of the target's center pixel coordinates within the image plane, in order to eliminate the impact of detection jitter on subsequent wobbling calculations.

[0027] It should be noted that when using the YOLOv5 object detection algorithm for detection, the detection center ( u, v The pixel velocity can be jittery (due to noise, NMS fluctuations, occlusion, etc.). Directly using traditional differential calculations would amplify the noise, resulting in unstable pixel velocity. Therefore, this solution employs Kalman filtering to... By constructing a joint form of state variables, the problem of detection jitter in traditional differential computation is overcome, and a smoother and more reliable rate of change estimation can be obtained.

[0028] S2. Based on the camera's intrinsic parameter matrix, first convert the pixel coordinates in the image sequence into the line-of-sight vector of the load relative to the UAV's body coordinate system. Then, combined with the UAV's real-time attitude angle, convert the line-of-sight vector into the navigation coordinate system. Next, calculate the load's swing angle and swing angular velocity to obtain the swing state vector.

[0029] Let the intrinsic parameter matrix be denoted as K Then the line-of-sight vector in the camera coordinate system is The superscript -1 denotes the inverse matrix. Based on the mounting relationship between the camera and the UAV body, let the rotation matrix from the camera coordinate system to the UAV body coordinate system be denoted as... Then the line-of-sight vector in the UAV body coordinate system is Furthermore, the real-time attitude angle of the UAV is denoted as... multiply by the dot. d b This can be converted to a navigation coordinate system, where, ϕ Indicates the roll angle of the drone. i Indicates the pitch angle of the drone. ψ This represents the drone's heading. Essentially, it's about converting the vector d in the drone's body coordinate system... b Multiply by a rotation matrix of "body → navigation" to obtain the same physical direction vector under the navigation system, which is used for unified control calculation.

[0030] Back projection using camera intrinsic parameters is a key step in establishing the mapping from two-dimensional image points to three-dimensional spatial directions, ensuring the accuracy of geometric relationships. Combining the camera mounting matrix and UAV attitude, the line-of-sight vector is unified to the body and navigation coordinate system, providing a unified mathematical basis for solving and controlling the load state at the full spatial scale.

[0031] When calculating the swing angle and angular velocity of the load, the unit vector of the numerical direction in the machine coordinate system is denoted as... align the target's line of sight with By comparison, the swing angle of the load is defined as... .

[0032] Based on the above calculation of the swing angle, to further obtain the dynamic characteristics of the load swing, it is necessary to analyze the change of the swing angle over time to calculate the swing angular velocity. Since the swing angle obtained from the visual calculation is a discrete-time series and is inevitably affected by factors such as image noise, target detection jitter, and camera measurement errors, directly performing simple differencing on the swing angle can easily introduce high-frequency noise, leading to unstable angular velocity estimation and thus affecting the subsequent coordinated control effect. Therefore, further analysis of the swing angle is needed. Discrete difference calculations are performed to obtain a preliminary estimate of the oscillation angular velocity. Δ t The time interval is between two adjacent sampling times. The above differential calculation can reflect the changing trend of the load swing angle within adjacent control cycles, and is used to describe the instantaneous dynamic characteristics of the load swing.

[0033] Considering the amplification effect of visual measurement noise and discrete difference operations on high-frequency noise, this invention further filters the difference results to improve the stability and reliability of the swing angular velocity estimation. During the filtering process, the swing angle and swing angular velocity are used together as system state variables to construct the load swing state vector. .

[0034] S3. Construct the current flight state vector of the UAV, and generate the corresponding expected flight trajectory state vector based on the mission plan of the ground station.

[0035] The current flight state vector is ,in, p u (k) This represents the position vector of the UAV in the navigation coordinate system. This represents the velocity vector of the UAV in the navigation coordinate system. the(k) This represents the Euler angle vector representing the attitude of the UAV. ω(k)The vector represents the angular velocity of the UAV; the desired flight trajectory vector (generated by the ground station based on the operational area, and interpolated and smoothed at the airborne end to form the desired flight trajectory state vector) is: ,in, p d (k) Indicates the desired position. v d (k) Indicates the desired speed. or d (k) This represents the desired attitude. It defines a complete flight state vector, providing the control system with comprehensive feedback on the UAV's own attitude and motion.

[0036] S4. Define the current integrated state based on the current flight state vector and the sway state vector, and then obtain the attitude compensation control quantity based on the desired flight trajectory state vector. Adjust the attitude of the UAV according to the attitude compensation control quantity until the desired flight trajectory state vector is met.

[0037] In this embodiment, the current synthesis state can be defined as follows: During the main hoisting phase, the target state (trajectory target + pitching target) is defined as follows: ,in, This represents the desired load swing angle (usually set to 0). This represents the desired load oscillation angular velocity (usually set to 0), which is to minimize load oscillation while maintaining the desired trajectory as much as possible.

[0038] In this embodiment, the attitude compensation control quantity is: ,in, This indicates the amount of attitude compensation in the roll direction. This indicates the pitch attitude compensation amount. This indicates the angle of sway of the load relative to the vertical direction of the drone. Indicates the angular velocity of the load swing. K s The sway control gain matrix is ​​used to superimpose the attitude compensation amount onto the desired attitude. ,Will The data is sent to the ground station control module to control the UAV to adjust its attitude. The load swing angle and angular velocity are used as direct inputs to generate inverse attitude compensation commands, realizing active damping suppression of the swing dynamics. The compensation is superimposed on the desired attitude, enabling the UAV to reduce load swing through its own active attitude adjustment, achieving deep coordination between flight control and load swing suppression.

[0039] S5. When the lifting meets the conditions for lightweight and flexible deployment, the drone will naturally release the load.

[0040] Lightweight and flexible deployment conditions include: meeting s(k) ≥ s th The load sway condition meets the sway suppression conditions; the drone's altitude, speed, and attitude meet the preset safe delivery requirements. A maximum load angle threshold is set for the load sway. Maximum load swing angular velocity threshold The swing suppression condition is When the trajectory error is large, priority is given to ensuring the flight trajectory. Once the trajectory error meets the threshold condition, the weight of sway control is increased, achieving a dynamic trade-off between trajectory tracking and sway control. By setting multiple safety conditions for joint judgment, it is ensured that the deployment action is triggered only at absolutely safe moments when the load is stable and the drone is in a stable state, greatly reducing the risk of accidents.

[0041] like Figure 2 The diagram shown is a schematic of a device for the safe and flexible lifting of drones. Figure 2 As shown, this solution also proposes a vision-based UAV safety flexible hoisting system to perform the above method. The system includes a UAV flight platform, a hoisting mechanism, and a vision perception and ground station control module. The UAV flight platform transports the load and acquires images of the load through the hoisting mechanism. The vision perception and ground station control module controls the attitude of the UAV flight platform based on the images of the load.

[0042] The UAV flight platform includes a flight controller, an onboard computing unit, an inertial measurement unit (IMU), forward-looking and downward-looking cameras, a positioning module, and a power module. The flight controller performs attitude and position control; the forward-looking and downward-looking cameras acquire images of the payload; the onboard computing unit handles visual perception, state calculation, and control command generation; and the IMU provides attitude angle, angular velocity, and acceleration information. The lifting mechanism includes an anti-sway pod, a flexible cable, and a gravity-release dispenser. The anti-sway pod helps suppress payload sway during lifting, while the gravity-release dispenser ensures low-impact release under safe conditions. The onboard computing unit communicates with the flight controller via a communication interface (e.g., a MAVROS interface) to acquire flight status and send control commands. The visual perception and ground station control module receives data from the UAV flight platform and is used for mission planning, configuring the desired flight path and delivery area, and monitoring operational status. The positioning module provides the UAV flight platform's position information to the ground station or onboard computing unit. The power module provides flight power and adjusts attitude in response to control commands.

[0043] The operating principle and beneficial effects of this system can be found in the description of the aforementioned method, and will not be repeated here.

[0044] In summary, this invention uses a camera mounted on a UAV to detect and track the hoisted load in real time, and calculates the load's swing angle and angular velocity relative to the UAV using target pixel information. Based on this, a comprehensive state variable integrating the UAV's flight state and the load's swing state is constructed. Through coordinated control design, attitude compensation is generated to actively suppress load swing while tracking the desired flight trajectory. Furthermore, a gravity-release dispenser is triggered only when the load is stable and meets the delivery conditions, achieving low-impact, flexible delivery. This method forms a complete closed loop from perception, calculation, and coordinated control to safe delivery, significantly improving the safety, stability, and delivery accuracy of UAV hoisting operations in complex environments, demonstrating significant advancements.

[0045] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A vision-based method for safe and flexible lifting operations using unmanned aerial vehicles (UAVs), characterized in that, The steps are as follows: S1. The camera mounted on the drone treats the payload as a target and acquires the initial image corresponding to the target in real time through the target detection box; The initial image is preprocessed to obtain an optimized image sequence; the preprocessing includes: using non-maximum suppression to filter and obtain the final target box and extract the center pixel coordinates, and performing Kalman filtering; the center pixel coordinates are in the camera coordinate system; S2. Based on the camera's intrinsic parameter matrix, first convert the pixel coordinates in the image sequence into the line-of-sight vector of the load relative to the UAV's body coordinate system. Then, combined with the UAV's real-time attitude angle, convert the line-of-sight vector into the navigation coordinate system. Next, calculate the load's swing angle and swing angular velocity to obtain the swing state vector. S3. Construct the current flight state vector of the UAV, and generate the corresponding expected flight trajectory state vector based on the mission plan of the ground station. S4. Define the current integrated state based on the current flight state vector and the sway state vector, and then obtain the attitude compensation control quantity based on the desired flight trajectory state vector. Adjust the attitude of the UAV according to the attitude compensation control quantity until the desired flight trajectory state vector is met. S5. When the lifting meets the conditions for lightweight and flexible deployment, the drone will naturally release the load.

2. The vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 1, characterized in that, Before acquiring the initial image in step S1, the load is monitored in real time based on the YOLOv5 object detection algorithm, and the target bounding box is output. ,in, This indicates the center position of the load in the coordinate system of the initial image. This indicates the width and height in the target dimensions. s i Indicates the confidence level. c i Indicates the target category, k Indicates the current period.

3. The vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 2, characterized in that, In step S1, during non-maximum suppression screening, the detection confidence of the target candidate box is recorded as... s(k) The target detection confidence threshold is denoted as s th ; to satisfy each s(k) ≥ s th The detected bounding boxes are considered valid and used as the corresponding final bounding boxes; the horizontal and vertical pixel coordinates of the target in the final bounding box are denoted as follows: Then the center pixel coordinates are , T This indicates transpose.

4. The vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 1, characterized in that, After obtaining the center pixel coordinates, the pixel state vector is defined as follows: ,in, This represents the rate of change of the center pixel coordinates of the target within the image plane.

5. A vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 4, characterized in that, In step S2, the intrinsic parameter matrix is ​​denoted as... K Then the line-of-sight vector in the camera coordinate system is The superscript -1 indicates the inverse matrix; Let the rotation matrix from the camera coordinate system to the UAV body coordinate system be denoted as... Then the line-of-sight vector in the UAV body coordinate system is ; The real-time attitude angle of the drone is denoted as ,in, ϕ Indicates the roll angle of the drone. θ Indicates the pitch angle of the drone. ψ Indicates the drone's heading.

6. A vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 5, characterized in that, In step S2, when calculating the swing angle and angular velocity of the load, the unit vector of the numerical direction in the machine system coordinate system is denoted as... align the target's line of sight with By comparison, the swing angle of the load is defined as... ; Again Discrete difference calculations are performed to obtain a preliminary estimate of the oscillation angular velocity. Δ t The time interval between two adjacent sampling times; Then, the swing state vector is constructed. .

7. A vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 6, characterized in that, In step S3, the current flight state vector is ,in, p u (k) This represents the position vector of the UAV in the navigation coordinate system. This represents the velocity vector of the UAV in the navigation coordinate system. η(k) This represents the Euler angle vector representing the attitude of the UAV. ω(k) This represents the angular velocity vector of the UAV; the desired flight trajectory vector is... ,in, p d (k) Indicates the desired position. v d (k) Indicates the desired speed. η d (k) It expresses a desired attitude.

8. A vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 1, characterized in that, In step S4, the attitude compensation control quantity is ,in, This indicates the amount of attitude compensation in the roll direction. This indicates the pitch attitude compensation amount. This indicates the angle of sway of the load relative to the vertical direction of the drone. Indicates the angular velocity of the load swing. K s This is the sway control gain matrix; The attitude compensation is superimposed onto the desired attitude. ,Will It is sent to the ground station control module to control the UAV to adjust its attitude.

9. A vision-based method for safe and flexible lifting of unmanned aerial vehicles according to claim 8, characterized in that, The lightweight flexible deployment conditions in step S5 include: meeting the following requirements. s(k) ≥ s th The load sway condition meets the sway suppression condition; the drone's altitude, speed, and attitude meet the preset safe delivery requirements; among these, a maximum load angle threshold is set for the load sway. Maximum load swing angular velocity threshold The swing suppression condition is .

10. A vision-based unmanned aerial vehicle (UAV) safe flexible lifting system for performing the method as described in any one of claims 1-9, characterized in that, It includes an unmanned aerial vehicle (UAV) flight platform, a hoisting mechanism, a visual perception and ground station control module, a positioning module, and a power module. The UAV flight platform transports the load and acquires images of the load through the hoisting mechanism. The visual perception and ground station control module controls the attitude of the UAV flight platform based on the images of the load. The positioning module provides the position information of the UAV flight platform to the visual perception and ground station control module or the onboard computing unit. The power module provides flight power and performs attitude adjustment.

Images