Control method and control device of unmanned aerial vehicle and unmanned aerial vehicle

Abstract

This application discloses a control method, control device, and drone for a drone, belonging to the field of drone technology. The drone's fuselage is equipped with an image acquisition device and a first attitude sensor. The image acquisition device is used to acquire image information of a sling. One end of the sling is connected to the fuselage, and the other end of the sling is equipped with a second attitude sensor. The method includes: determining a first swing angle of the sling based on the image information of the sling acquired by the image acquisition device and the first attitude data acquired by the first attitude sensor; determining a swing angle variable of the sling based on the second attitude data acquired by the second attitude sensor; obtaining swing angle information of the sling based on the first swing angle data and the swing angle variable; and controlling the drone to perform an anti-swing action based on the swing angle information of the sling. This method can accurately measure the swing angle of the sling connected to the drone, effectively improving the anti-swing effect.

Description

Technical Field

[0001] This application belongs to the field of unmanned aerial vehicle (UAV) technology, and particularly relates to a control method, control device, and UAV for a UAV. Background Technology

[0002] Cleaning and maintenance of rooftop solar panels can be accomplished using drones to transport cleaning robots to the roof. However, due to wind disturbances and drone flight vibrations, the suspended end is prone to significant swaying, causing the cleaning robot to collide with exterior walls, balconies, or solar panel supports. This poses safety risks and reduces operational efficiency. Therefore, real-time detection and proactive mitigation of swaying are necessary. Existing sway mitigation solutions mostly rely on external markers for sway measurement, resulting in low measurement accuracy and unreliable sway angle information, leading to poor sway mitigation effects. Summary of the Invention

[0003] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a control method, control device, and drone for unmanned aerial vehicles (UAVs), which can accurately measure the swing angle of the suspension rope attached to the UAV and effectively improve the anti-swing effect.

[0004] In a first aspect, this application provides a control method for an unmanned aerial vehicle (UAV), wherein the UAV's fuselage is equipped with an image acquisition device and a first attitude sensor, the image acquisition device is used to acquire image information of a sling, one end of the sling is connected to the fuselage, and the other end of the sling is equipped with a second attitude sensor, the method comprising: Based on the image information of the suspension rope acquired by the image acquisition device and the first attitude data acquired by the first attitude sensor, the first swing angle data of the suspension rope is determined; Based on the second attitude data collected by the second attitude sensor, the swing angle variable of the suspension rope is determined; Based on the first swing angle data and the swing angle variable, the swing angle information of the suspension rope is obtained; Based on the swing angle information of the suspension rope, the drone is controlled to perform a swing-off action.

[0005] According to the UAV control method of this application, the first swing angle data of the suspension rope is calculated by using the first attitude data of the fuselage and the image information of the suspension rope. The swing angle variable of the suspension rope is calculated based on the second attitude data of the suspension rope. The first swing angle data calculated by fusing the image data and attitude data is used as the reference value for swing angle estimation. The swing angle variable calculated by combining the attitude data of the suspension rope end is used to estimate the swing angle of the suspension rope in real time, analyze the swing state of the suspension rope, and execute anti-swing action. This method can accurately measure the swing angle of the suspension rope connected to the UAV and effectively improve the anti-swing effect.

[0006] According to one embodiment of this application, determining the first swing angle data of the suspension rope based on the suspension rope image information acquired by the image acquisition device and the first attitude data acquired by the first attitude sensor includes: Based on the first posture data, a first transformation parameter is determined, which is used to characterize the transformation relationship between the reference coordinate system and the world coordinate system of the image acquisition device. Based on the image information of the suspension rope, skeleton extraction and line fitting are performed to obtain the first direction vector of the suspension rope in the reference coordinate system; Based on the first transformation parameter and the first direction vector, the second direction vector of the suspension rope in the world coordinate system is obtained; Based on the second direction vector, the length information of the suspension rope, and the position information of the connection point between the suspension rope and the fuselage, the first swing angle data is obtained.

[0007] According to one embodiment of this application, the first swing angle data includes a first initial swing angle; The step of obtaining the swing angle information of the suspension rope based on the first swing angle data and the swing angle variable includes: The first initial swing angle and the swing angle variable are added together to obtain the first swing angle of the suspension rope.

[0008] According to one embodiment of this application, the first swing angle data further includes a first instantaneous swing angle; after obtaining the first swing angle of the suspension rope, the method further includes: In response to the image acquisition device meeting the preset observation conditions at a first moment, the first instantaneous swing angle corresponding to the first moment is acquired; Based on the first instantaneous swing angle, construct the observation residual; Based on the observed residual, the first swing angle is corrected to obtain the second swing angle of the suspension rope.

[0009] According to one embodiment of this application, after obtaining the second swing angle of the suspension rope, the method further includes: Based on the length information of the suspension rope and the pendulum dynamics model, a constraint residual is constructed; Based on the constraint residual, the second swing angle is corrected to obtain the third swing angle of the suspension rope.

[0010] According to one embodiment of this application, determining the swing angle variable of the suspension rope based on the second attitude data collected by the second attitude sensor includes: Based on the second attitude data, the first angular velocity data of the suspension rope is obtained; Based on the first angular velocity data, the first angular velocity variable of the suspension rope is obtained; Integrating the first angular velocity variable yields the swing angle variable.

[0011] According to one embodiment of this application, obtaining the first angular velocity variable of the suspension rope based on the first angular velocity data includes: Based on the second attitude data, the initial angular velocity of the suspension rope is obtained, and the difference between the acquisition time of the initial angular velocity and the acquisition time of the first swing angle data is less than a preset time interval. Based on the first angular velocity data and the initial angular velocity, the first angular velocity variable is obtained.

[0012] According to one embodiment of this application, controlling the drone to perform a swing-off action based on the swing angle information of the suspension rope includes: Based on the swing angle information, the swing level of the suspension rope is determined, wherein the swing angle information includes the swing angle and the rate of change of the swing angle of the suspension rope; Control the drone to perform a sway-reducing action corresponding to the sway level.

[0013] According to one embodiment of this application, determining the swing level of the suspension rope based on the swing angle information includes: If the swing angle of the suspension rope is greater than or equal to a first swing angle threshold and less than a second swing angle threshold, and the rate of change of the swing angle is less than a preset rate of change threshold, the swing level of the suspension rope is determined to be the first swing level. Alternatively, if the swing angle of the suspension rope is greater than or equal to the second swing angle threshold, or if the swing angle change rate is greater than or equal to the preset change rate threshold, the swing level of the suspension rope is determined to be the second swing level.

[0014] According to one embodiment of this application, the other end of the hoisting rope is further provided with a driving mechanism, which is used to drive the other end of the hoisting rope to move; The control of the drone to perform a sway-reducing action corresponding to the sway level includes: When the swing level of the suspension rope is the first swing level, the drive mechanism is controlled to perform a swing-off action; Alternatively, if the swing level of the suspension rope is the second swing level, the machine body and the drive mechanism are controlled to perform an anti-swing action.

[0015] Secondly, this application provides a control device for a drone. The drone's fuselage is equipped with an image acquisition device and a first attitude sensor. The image acquisition device is used to acquire image information of a sling. One end of the sling is connected to the fuselage, and the other end of the sling is equipped with a second attitude sensor. The device includes: The first processing module is used to determine the first swing angle data of the suspension rope based on the suspension rope image information acquired by the image acquisition device and the first attitude data acquired by the first attitude sensor. The second processing module is used to determine the swing angle variable of the suspension rope based on the second attitude data collected by the second attitude sensor. The third processing module is used to obtain the swing angle information of the suspension rope based on the first swing angle data and the swing angle variable; The fourth processing module is used to control the drone to perform anti-swing actions based on the swing angle information of the suspension rope.

[0016] Thirdly, this application provides a drone, comprising: The fuselage is equipped with an image acquisition device and a first attitude sensor. The image acquisition device is used to acquire image information of the suspension rope. One end of the suspension rope is connected to the fuselage, and the other end of the suspension rope is equipped with a second attitude sensor. The control device for the unmanned aerial vehicle as described in the second aspect above is connected to the image acquisition device, the first attitude sensor, and the second attitude sensor.

[0017] Fourthly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the drone control method described in the first aspect above.

[0018] Fifthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the UAV control method as described in the first aspect above.

[0019] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the UAV control method as described in the first aspect.

[0020] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the drone control method described in the first aspect above.

[0021] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating the control method for a drone provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of the drone carrying the load provided in the embodiment of this application; Figure 3 This is a second schematic flowchart of the drone control method provided in the embodiments of this application; Figure 4 This is the third flowchart illustrating the control method for a drone provided in this application embodiment; Figure 5 This is the fourth flowchart illustrating the control method for an unmanned aerial vehicle provided in this application embodiment; Figure 6 This is the fifth flowchart illustrating the control method for an unmanned aerial vehicle provided in this application embodiment; Figure 7 This is a schematic diagram of the structure of the control device for the unmanned aerial vehicle provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0023] Figure label: Airframe 201, first attitude sensor 210, second attitude sensor 220, image acquisition device 230. Suspension rope 300, drive mechanism 400, load 500. First processing module 710, second processing module 720, third processing module 730, fourth processing module 740. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] The following description, in conjunction with the accompanying drawings, details the control method, control device, drone, electronic device, and readable storage medium for a drone provided in this application, through specific embodiments and application scenarios.

[0027] like Figure 2 As shown, the fuselage 201 of the UAV is equipped with an image acquisition device 230 and a first attitude sensor 210. The image acquisition device 230 is used to acquire image information of the suspension rope 300.

[0028] The first attitude sensor 210 is used to collect real-time attitude data of the fuselage 201. The image acquisition device 230 can be installed on the bottom of the UAV fuselage 201. The optical axis of the image acquisition device 230 can be vertically downward to collect images of the suspension rope 300.

[0029] The suspension rope 300 is used to suspend the load 500 (such as a photovoltaic cleaning robot). One end of the suspension rope 300 is connected to the body 201, and the other end of the suspension rope 300 (which can be called the end) is equipped with a second attitude sensor 220. The load 500 can be connected to the end of the suspension rope 300. The second attitude sensor 220 is used to collect real-time attitude data of the end of the suspension rope 300.

[0030] In actual implementation, the first attitude sensor 210 and the second attitude sensor 220 can be inertial measurement units (IMUs) that can measure attitude data such as acceleration and angular velocity.

[0031] Understandably, the drone's fuselage 201 provides functions such as flight and attitude control, and can drive the drone's fuselage 201 to move laterally. By adjusting the position of the fuselage 201, the swing of the suspension rope 300 can be suppressed.

[0032] It should be noted that the other end of the suspension rope 300 may also be provided with a drive mechanism 400, which is used to drive the load 500 at the other end of the suspension rope 300 to move.

[0033] In actual implementation, the drive mechanism 400 may include components such as a miniature quadcopter, a jet propulsion unit, or a torque module. By applying thrust or torque to the other end of the suspension rope 300, the other end of the suspension rope 300 is driven to move, thereby achieving active anti-sway.

[0034] The drone control method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the drone control method. The drone control method provided in this application embodiment is described below using an electronic device as the execution subject.

[0035] like Figure 1 As shown, the control method of the UAV includes steps 110, 120, 130 and 140.

[0036] Step 110: Based on the image information of the suspension rope acquired by the image acquisition device 230 and the first attitude data acquired by the first attitude sensor 210, determine the first swing angle data of the suspension rope 300.

[0037] In this step, the first attitude data collected by the first attitude sensor 210 reflects the real-time attitude of the UAV fuselage 201. The image acquisition device 230 is set on the fuselage 201. Based on the real-time attitude of the fuselage 201, the image of the suspension rope collected by the image acquisition device 230 can be projected onto the world coordinate system, and the angle between the suspension rope 300 and the direction of the plumb bob (i.e., the direction of gravity) can be calculated to obtain the first swing angle data of the suspension rope 300.

[0038] It is understandable that the image acquisition device 230 can continuously acquire images of the suspension rope, and the first attitude sensor 210 can also continuously acquire first attitude data. By using the suspension rope image and the first attitude data at the same moment, the first swing angle data at that moment can be calculated.

[0039] Step 120: Determine the swing angle variable of the suspension rope 300 based on the second attitude data collected by the second attitude sensor 220.

[0040] In this embodiment, the second attitude sensor 220 is disposed at the other end of the suspension rope 300, that is, the end of the suspension rope 300 used to suspend the load 500, and the second attitude sensor 220 collects the real-time attitude of the other end of the suspension rope 300.

[0041] In this step, the change in the swing angle of the suspension rope 300 is calculated based on the second attitude data collected by the second attitude sensor 220, that is, the swing angle variable of the suspension rope 300.

[0042] Step 130: Based on the first swing angle data and the swing angle variable, obtain the swing angle information of the suspension rope 300.

[0043] In this embodiment, after obtaining the first swing angle data at a certain moment, the swing angle variable from that moment to the current moment is calculated. Based on the first swing angle data and the swing angle variable, the swing angle of the suspension rope 300 at the current moment can be calculated, and the swing angle information of the suspension rope 300 can be obtained.

[0044] Step 140: Based on the swing angle information of the suspension rope 300, control the drone to perform the anti-swing action.

[0045] In this embodiment, the swing state of the suspension rope 300 is determined based on the swing angle information of the suspension rope 300. When the swing state of the suspension rope 300 meets the triggering condition for the anti-swing action, the drone is controlled to perform the anti-swing action to suppress the swing of the suspension rope 300.

[0046] In actual operation, the drone's fuselage 201 can be controlled to perform anti-swaying actions, and the swaying can be suppressed by adjusting the position of the fuselage 201. Alternatively, the drive mechanism 400 at the other end of the suspension rope 300 can be controlled to perform anti-swaying actions, and active anti-swaying can be achieved by applying thrust or torque at the location of the load 500.

[0047] In related technologies, swing angle estimation mostly involves attaching external markers such as reflective balls, QR codes, and marker points to the suspension rope 300 or load 500, and identifying the position and attitude of the markers through a camera or laser tracking system. This method relies on external hardware, which increases the cost of deployment and maintenance.

[0048] Some studies only use downward-looking cameras for detection. In complex slinging missions, the sling 300 may be obstructed by obstacles, causing visual tracking to be interrupted and the swing angle estimation to be unable to be continuously output, thus affecting flight safety.

[0049] In addition, high-precision swing angle measurement based on vision relies on image feature matching, 3D reconstruction or deep learning inference, which requires a lot of computing resources. When running on embedded flight controllers or computing boards, the processing delay often reaches hundreds of milliseconds, which cannot meet the real-time requirements of flight control at the tens of millisecond level. Excessive swing angle estimation delay will cause the UAV control actions to lag, making it impossible for compensation commands to eliminate the swing synchronously, and may also exacerbate the swing due to control phase delay.

[0050] In this embodiment, the drone's fuselage 201 is equipped with an image acquisition device 230 and a first attitude sensor 210. One end of the suspension rope 300 is connected to the fuselage 201, and the other end of the suspension rope 300 is equipped with a second attitude sensor 220. By fusing the data from the first attitude sensor 210, the second attitude sensor 220, and the image acquisition device 230, the swing angle of the suspension rope 300 can be calculated. There is no need to place external markers, which can reduce deployment and maintenance costs.

[0051] Based on the real-time attitude of the UAV body 201 reflected by the first attitude data collected by the first attitude sensor 210, the image of the suspending rope collected by the image acquisition device 230 is projected onto the world coordinate system to obtain the first swing angle data of the suspending rope 300. Based on the real-time attitude of the other end of the suspending rope 300 collected by the second attitude sensor 220, the swing angle variable of the suspending rope 300 is calculated. Combining the first swing angle data and the swing angle variable, the swing angle of the suspending rope 300 is estimated in real time. The first swing angle data calculated by fusing image data and attitude data is used as the reference value for swing angle estimation. Subsequently, the swing angle of the suspending rope 300 is estimated by combining the swing angle variable. There is no need for continuous image processing, the dependence on image quality and target visibility is low, and the computational resources are less consumed. Even if visual tracking is interrupted, it will not affect the real-time estimation of the swing angle. Based on the swing angle information of the suspending rope 300, the anti-swing action can be triggered in time, effectively improving the anti-swing effect.

[0052] According to the UAV control method provided in the embodiments of this application, the first swing angle data of the suspension rope 300 is calculated using the first attitude data of the fuselage 201 and the image information of the suspension rope. The swing angle variable of the suspension rope 300 is calculated based on the second attitude data of the suspension rope 300. The first swing angle data calculated by fusing the image data and attitude data is used as the reference value for swing angle estimation. The swing angle variable obtained by combining the end attitude data of the suspension rope 300 is calculated in real time to estimate the swing angle of the suspension rope 300, analyze the swing state of the suspension rope 300, and execute the anti-swing action. This method can accurately measure the swing angle of the suspension rope 300 connected to the UAV and effectively improve the anti-swing effect.

[0053] The process of fusing image data and attitude data to solve for the first swing angle data is explained in detail below.

[0054] In some embodiments, determining the first swing angle data of the suspension rope 300 based on the image information of the suspension rope acquired by the image acquisition device 230 and the first attitude data acquired by the first attitude sensor 210 may include: Based on the first attitude data, the first transformation parameters are determined. The first transformation parameters are used to characterize the transformation relationship between the reference coordinate system and the world coordinate system of the image acquisition device 230. Based on the image information of the suspension rope, skeleton extraction and line fitting are performed to obtain the first direction vector of the suspension rope 300 in the reference coordinate system; Based on the first transformation parameters and the first direction vector, the second direction vector of the suspension rope 300 in the world coordinate system is obtained; Based on the second direction vector, the length information of the suspension rope 300, and the position information of the connection point between the suspension rope 300 and the fuselage 201, the first swing angle data is obtained.

[0055] It is understandable that the first attitude sensor 210 outputs first attitude data in real time, reflecting the real-time attitude of the fuselage 201. Combined with the calibration extrinsic parameters between the image acquisition device 230 and the first attitude sensor 210, the extrinsic parameters from the reference coordinate system to the world coordinate system of the image acquisition device 230, i.e., the first transformation parameters, can be calculated for subsequent coordinate system transformation.

[0056] In this embodiment, the image information of the suspension rope acquired by the image acquisition device 230 can be preprocessed first, and then the preprocessed suspension rope image can be input into a lightweight skeleton extraction network for skeleton extraction. The key point information and direction vector information of the center line of the suspension rope 300 output by the skeleton extraction network are obtained. Based on the key point information and direction vector information, sub-pixel straight line fitting is performed to fit the straight line equation of the center line of the suspension rope 300, and the first direction vector of the suspension rope 300 in the reference coordinate system of the image acquisition device 230 is obtained.

[0057] Based on the first transformation parameters, the first direction vector in the reference coordinate system is transformed to the world coordinate system to obtain the second direction vector. Based on the length information of the suspension rope 300 and the position information of the connection point (i.e., the suspension point) between the suspension rope 300 and the vertical direction in the world coordinate system, the first swing angle data is calculated.

[0058] The following is a specific example.

[0059] like Figure 4 As shown, in step 1, the camera attitude is obtained by acquiring the attitude matrix (i.e., the first attitude data) output in real time by the first attitude sensor 210 of the camera body 201. The attitude matrix output in real time by the first attitude sensor 210 is denoted as... Combined with the camera extrinsic matrix of the image acquisition device 230 This yields the first transformation parameters used for subsequent coordinate system transformations.

[0060] Step 2: Acquire images of the suspension rope. Images of the suspension rope are continuously acquired using an image acquisition device 230 installed at the bottom of the main body 201. The image acquisition device 230 acquires a sequence of suspension rope images at a preset frame rate. Write the image sequence of the suspension rope to the circular buffer. Each frame carries a timestamp, which can be synchronized with the attitude matrix output by the first attitude sensor 210.

[0061] Step 3, Image preprocessing: Processing the acquired images of the suspension rope. Preprocessing operations are performed, including but not limited to center cropping (to place the lanyard 300 in the center of the field of view and reduce distortion), brightness normalization, contrast normalization, adaptive histogram equalization, and bilateral filtering (which can be guided filtering or other algorithms). These operations enhance the contrast between the lanyard 300 and the background, suppress texture noise while preserving edges, and preprocess to form the lanyard image. .

[0062] Step 4, lightweight skeleton extraction, extracting the preprocessed image. Input a lightweight skeleton extraction network, and the skeleton extraction network outputs a key point heatmap of the 300° centerline of the suspension rope. and direction vector field .

[0063] The lightweight skeleton extraction network can use MobileNetV3 or ShuffleNet as the backbone, combined with the extraction network of the lightweight decoder head; alternatively, image processing algorithms such as Canny edge detection and Hough linear transformation can be used to extract the skeleton of the center line of the suspension rope 300.

[0064] Step 5: Linear fitting, heatmap at key points Non-maximum suppression and thresholding are performed to extract the skeleton discrete point set. Then, algorithms such as random sampling consensus algorithm, least squares fitting algorithm, or Hough transform are used to process the discrete point set. By performing a straight line fitting, the equation of the straight line of the 300mm centerline of the suspension rope is obtained, and the direction vector field is used simultaneously. The consistency of the local average direction was verified, eliminating the influence of background pseudo-lines.

[0065] The equation of the straight line can be expressed as: .

[0066] Represents the coordinates of the image points. It can be the x-coordinate of a point in the image. It can be the ordinate of a point in the image.

[0067] It is understandable that the specific coordinates of a point are determined by the chosen origin, which can be set to any position in the image, such as the top left corner of the image.

[0068] in, In the equation of a straight line coefficient, In the equation of a straight line coefficient, This is a constant term in the equation of the line (related to the intercept of the equation of the line).

[0069] , This represents a column vector consisting of the parameters of the line. This represents the transpose symbol.

[0070] in, Let be the normal vector of the line. The normal vector is the direction vector perpendicular to the line; that is, the direction vector of the line can be... The slope of the straight line at this time , ≠0. Step 6, calculate the image direction based on the direction vector of the line equation. Obtain pixel plane orientation Calculate the unit direction vector of the suspension rope 300 in the reference coordinate system of the image acquisition device 230. That is, the first direction vector .

[0071] Step 7, coordinate system transformation, using the attitude matrix obtained in Step 1. and camera extrinsic matrix , the first direction vector Transform to world coordinates to obtain the second direction vector. .

[0072] Step 8: Calculate the pendulum angle using 3D back projection, based on the known length of the suspension rope (300). World coordinates of the suspension point Establish the ray equation Calculate the angle between the suspension rope 300° and the vertical direction in the world coordinate system, which is the first swing angle data.

[0073] In this embodiment, the perspective of the image acquisition device 230 is corrected in real time by the first posture data, so that the two-dimensional observation of the rope image information can be accurately mapped to the world coordinate system, reducing the perspective distortion of the image acquisition device 230. Through the lightweight skeleton extraction, line fitting and back projection processing flow, the three-dimensional swing angle of the rope 300 aligned with the direction of the plumb bob is output.

[0074] It should be noted that the first swing angle data calculated by fusing the first attitude data and the suspension rope image information can be used as an initial state quantity with absolute reference, zero drift and frequency alignment.

[0075] In some embodiments, the first swing angle data includes a first initial swing angle.

[0076] The first initial swing angle can be the initial swing angle for the swing angle estimation task.

[0077] For example, after the drone is powered on and takes off, it acquires the current first attitude data and the image information of the tether. Through a lightweight skeleton extraction, line fitting, and back projection processing flow, the first initial swing angle is calculated. .

[0078] In this embodiment, based on the first swing angle data and the swing angle variable, the swing angle information of the suspension rope 300 is obtained, including: Add the first initial swing angle and the swing angle variable to obtain the first swing angle of the suspension rope 300.

[0079] In this embodiment, the first initial swing angle is used as the initial state variable, and the swing angle variable calculated from the end attitude data of the suspension rope 300 is used as the state change variable. The swing angle of the suspension rope 300 is estimated in real time through algorithms such as filters to obtain the first swing angle of the suspension rope 300.

[0080] For example, the swing angle variable of the suspension rope 300 is calculated based on the second attitude data from the second attitude sensor 220. , swing angle variable With the first initial swing angle Adding them together, we get the real-time swing angle of the suspension rope at 300 degrees. That is, the first swing angle.

[0081] In some embodiments, determining the swing angle variable of the suspension rope 300 based on the second attitude data collected by the second attitude sensor 220 may include: Based on the second attitude data, the first angular velocity data of the suspension rope 300 is obtained; Based on the first angular velocity data, the first angular velocity variable of the suspension rope 300 is obtained; Integrating the first angular velocity variable yields the swing angle variable.

[0082] In this embodiment, the attitude data continuously output by the second attitude sensor 220 includes angular velocity data. The first angular velocity data is extracted from the second attitude data. Based on the first angular velocity data, the first angular velocity variable of the suspension rope 300 is calculated. The first angular velocity variable is integrated to obtain the swing angle variable over a period of time.

[0083] In some embodiments, the first angular velocity variable of the suspension rope 300 is obtained based on the first angular velocity data, including: Based on the second attitude data, the initial angular velocity of the suspension rope 300 is obtained; Based on the first angular velocity data and the initial angular velocity, the first angular velocity variable is obtained.

[0084] Among them, the difference between the acquisition time of the initial angular velocity and the acquisition time of the first swing angle data is less than the preset time interval.

[0085] It should be noted that when obtaining the initial swing angle for the swing angle estimation task, the average static angular velocity of the second attitude sensor 220 within a preset time interval (e.g., 1 second) is read simultaneously to obtain the initial angular velocity. The initial angular velocity is used as the zero bias initial value, which is convenient to subtract the zero bias initial value of the 300° angular velocity of the suspension rope when calculating the first angular velocity variable in the subsequent calculation, thereby reducing the error and improving the accuracy of the swing angle variable.

[0086] It is understandable that the first swing angle data is obtained based on the first attitude data and the suspension rope image information. The acquisition time of the first swing angle data refers to the acquisition time of the first attitude data and the suspension rope image information. When acquiring the first swing angle data, the average static angular velocity of the second attitude sensor 220 within a preset time interval (such as 1 second) is read synchronously as the initial angular velocity.

[0087] For example, after the drone is powered on and takes off, it acquires the current first attitude data and the image information of the tether. Through a lightweight skeleton extraction, line fitting, and back projection processing flow, the first initial swing angle is calculated. The average static angular velocity of the second attitude sensor 220 within 1 second is read synchronously to obtain the initial angular velocity. .

[0088] Obtain the first angular velocity data of the suspension rope 300. ,Will deduct The first angular velocity variable is obtained. According to time right Integrate to obtain the pendulum angle increment. ,Will and Adding them together, we get the real-time swing angle of the suspension rope at 300 degrees. .

[0089] In related technologies, the swing angle is obtained by integrating the angular velocity of a gyroscope or calculating the acceleration vector. However, due to the inherent bias and noise of the IMU, long-term integration of the angular velocity will cause cumulative errors, and the estimated angle will gradually deviate from the true value. Under conditions such as long-term flight, strong wind disturbances, or sharp turns, the error brought by the IMU will be rapidly amplified, which cannot meet the accuracy requirements of UAV slinging missions and cannot achieve long-term stable measurement.

[0090] In this embodiment, the first swing angle data is calculated by fusing the first attitude data and the suspension rope image information. This data serves as an absolute reference, zero drift, and frequency-aligned initial state quantity. Combined with the swing angle variable after deducting the zero bias initial value, the swing angle of the suspension rope 300 can be estimated accurately in real time. This effectively suppresses the error accumulation effect caused by the initial error and the zero bias coupling of the sensor during the recursive process of the integral algorithm.

[0091] In addition, the embodiments of this application can also correct the estimated swing angle in real time by observing the residual and constraining the residual, correcting the cumulative error introduced by sensor zero bias, integral drift and environmental disturbance, maintaining the global consistency and sub-degree accuracy of the swing angle estimation during the flight of the UAV, and providing a reliable, robust and high-frequency state input for the execution of subsequent swing cancellation actions.

[0092] In some embodiments, the first swing angle data may further include a first instantaneous swing angle; after obtaining the first swing angle of the suspension rope 300, the control method of the UAV may further include: In response to the image acquisition device 230 meeting the preset observation conditions at the first moment, the first instantaneous swing angle corresponding to the first moment is acquired; Based on the initial swing angle, construct the observation residuals; Based on the observed residuals, the first swing angle is corrected to obtain the second swing angle of the suspension rope at 30°.

[0093] The first instantaneous swing angle refers to the first swing angle data of the image acquisition device 230 at the moment when the preset observation conditions are met (i.e., the first moment). It can be calculated based on the first posture data and the rope image information at the first moment through a lightweight skeleton extraction, straight line fitting and back projection processing flow.

[0094] In this embodiment, when the image acquisition device 230 triggers the preset observation conditions, it acquires the corresponding first instantaneous swing angle. The first instantaneous swing angle is used as the absolute observation value and as the zero-drift observation input. It is then fed into the filter to construct the observation equation and observation residual. The first swing angle is corrected by the observation residual to obtain the second swing angle after drift compensation. This significantly suppresses the drift problem of attitude data integration accumulating over time and ensures the stability of the swing angle estimation of the suspension rope 300 under long-term operation.

[0095] In actual execution, the preset observation conditions can be set according to actual needs. For example, the integration time of the second attitude data can reach a preset threshold, or the acquisition of the first instantaneous swing angle can be triggered when the suspension rope 300 appears completely in the view of the image acquisition device 230.

[0096] In some embodiments, after obtaining the second swing angle of the suspension rope 300, the control method for the UAV may further include: Based on the length information of the suspension rope 300 and the pendulum dynamics model, constrained residuals are constructed. Based on the constraint residual, the second swing angle is corrected to obtain the third swing angle of the suspension rope 300.

[0097] In this embodiment, the length information of the suspension rope 300 and the pendulum dynamics model are used as pseudo-measurements to construct constraint residuals. In the filter update step, the constraint residuals are used to correct the second pendulum angle and output a high-precision third pendulum angle. By introducing geometric and dynamic constraints of the suspension rope 300, the pendulum angle estimation accuracy can be effectively improved, which is significantly better than the accuracy of single sensor measurement data in related technologies.

[0098] The following is a specific example.

[0099] After the drone is powered on and takes off, the first initial swing angle is calculated through a process of lightweight skeleton extraction, straight line fitting, and back projection. .

[0100] like Figure 5 As shown, in step 1, the angular velocity is obtained. The second attitude sensor 220, installed at the other end of the suspension rope 300, continuously outputs the first angular velocity data. , Subtracting zero bias initial value The first angular velocity variable in real time is obtained. .

[0101] Step 2, angular velocity integration, based on the first angular velocity variable Numerical integration over time is performed to calculate the swing angle variable. ;Will and Add them together to get the real-time swing angle. That is, the first swing angle.

[0102] Step 3: Generate visual observation. When the image acquisition device 230 triggers the preset observation conditions, it executes the lightweight skeleton extraction, line fitting, and back projection processing flow to calculate the first instantaneous swing angle. , as an absolute observation value.

[0103] Step 4, filter update, will As the zero-drift observation input, it is fed into a filter (such as an extended Kalman filter, an unscented Kalman filter, a particle filter, etc.) to construct the observation equation. The filter then adjusts the swing angle based on the observation residuals of the observation equation. Perform a state update to generate a drift-compensated swing angle estimate. That is, the second swing angle.

[0104] Step 5, Geometric-Dynamic Constraints: Using the length of the suspension rope 300 and the pendulum's dynamic equations as pseudo-measures, construct constraint residuals. The filter uses these constraint residuals in the update step. Perform secondary corrections to output high-precision swing angles. That is, the third swing angle, relative to the swing angle Execution status update.

[0105] In this step, a high-precision swing angle is output. After that, you can The initial state variables are updated, and the swing angle variable calculated from the end attitude data of the suspension rope 300 is used as the state change variable. The swing angle of the suspension rope 300 is estimated in real time through algorithms such as filters.

[0106] In this embodiment, by observing the residuals, the drift problem of attitude data accumulated over time can be significantly suppressed, ensuring that the swing angle estimation remains stable under long-term operation. Geometric-dynamic constraints (such as the invariance of rope length and the swing equation) are introduced and used as pseudo-measurements for secondary correction in the filtering update, making the swing angle estimation accuracy better than that of single sensor measurement.

[0107] It should be noted that when visual observation is missing or abnormal, an available swing angle estimate can be provided based on the second attitude sensor 220 and geometric-dynamic constraints. When the measurement noise of the second attitude sensor 220 is large, effective drift compensation can be provided through the visual observation constraints of the image acquisition device 230.

[0108] In this embodiment, the estimated swing angle is corrected in real time by observing the residual and constraining the residual, maintaining the global consistency and sub-degree accuracy of the swing angle estimation during the flight of the UAV. This provides a reliable, robust and high-frequency state input for the subsequent swing cancellation action. The UAV can execute the swing cancellation action according to the hierarchical swing cancellation strategy, and progressively call the fuselage 201 or the drive mechanism 400 according to the swing level. This avoids long-term online operation of high-power drive, ensuring the swing cancellation effect while maximizing system energy efficiency.

[0109] In some embodiments, controlling the drone to perform anti-sway actions based on the swing angle information of the suspension rope 300 may include: Based on the swing angle information, the swing level of the suspension rope 300 is determined; Control the drone to perform anti-sway actions corresponding to the sway level.

[0110] The swing angle information includes the swing angle and the rate of change of the swing angle of the suspension rope 300.

[0111] It is understandable that the swing angle used to determine the swing level of the suspension rope 300 can be the first swing angle calculated based on the integration of attitude data, the second swing angle corrected by the observation residual, or the third swing angle corrected by the observation residual and the constraint residual.

[0112] Among them, the rate of change of the pendulum angle refers to the angular velocity of the pendulum angle corresponding to the pendulum angle.

[0113] In this embodiment, the swing level of the suspension rope 300 is determined based on the swing angle and the rate of change of the swing angle, and the drone is controlled to perform swing elimination actions according to the graded swing elimination strategy.

[0114] In some embodiments, determining the swing level of the suspension rope 300 based on the swing angle information includes: If the swing angle of the suspension rope 300 is greater than or equal to the first swing angle threshold and less than the second swing angle threshold, and the rate of change of the swing angle is less than the preset rate of change threshold, the swing level of the suspension rope 300 is determined to be the first swing level. Alternatively, if the swing angle of the suspension rope 300 is greater than or equal to the second swing angle threshold, or the rate of change of the swing angle is greater than or equal to the preset rate of change threshold, the swing level of the suspension rope 300 is determined to be the second swing level.

[0115] The first swing level corresponds to the case of slight swing of the suspension rope 300, and the second swing level corresponds to the case of severe swing of the suspension rope 300.

[0116] In this embodiment, three threshold values ​​are set: a first swing angle threshold, a second swing angle threshold, and a preset rate of change threshold. The swing angle of the suspension rope 300 is compared with the first swing angle threshold and the second swing angle threshold, and the rate of change of the swing angle of the suspension rope 300 is compared with the preset rate of change threshold. Based on the comparison results, the current swing level of the suspension rope 300 is determined.

[0117] It should be noted that the second swing angle threshold is greater than the first swing angle threshold. When the swing angle of the suspension rope 300 is less than the first swing angle threshold, it indicates that the swing of the suspension rope 300 has little impact on the suspension task, and there is no need to perform anti-swing action.

[0118] It is understandable that the rate of change of the swing angle represents the trend of the swing angle. When the rate of change of the swing angle is greater than or equal to the preset rate of change threshold, it indicates that the swing of the suspension rope 300 is in an uncontrollable state, and the swing angle may increase sharply in a short period of time. In the case of severe swing of the suspension rope 300, corresponding swing suppression measures need to be taken.

[0119] In this embodiment, if the swing angle of the suspension rope 300 is greater than or equal to the first swing angle threshold and less than the second swing angle threshold, and the rate of change of the swing angle is less than the preset rate of change threshold, the suspension rope 300 is determined to be in a slight swing state, and the swing level of the suspension rope 300 is determined to be the first swing level.

[0120] If the swing angle of the suspension rope 300 is greater than or equal to the second swing angle threshold, or the rate of change of the swing angle is greater than or equal to the preset rate of change threshold, the suspension rope 300 is determined to be in a severe swing state, and the swing level of the suspension rope 300 is determined to be the second swing level.

[0121] In some embodiments, controlling the drone to perform a de-swaying action corresponding to the sway level includes: When the swing level of the suspension rope 300 is the first swing level, the control drive mechanism 400 performs a swing-off action.

[0122] In this embodiment, the swing level of the suspension rope 300 is the first swing level. The slight swing of the suspension rope 300 can be controlled by the drive mechanism 400 at the end of the suspension rope 300 to make fine adjustments to eliminate the swing. The drive mechanism 400 has a fast response speed and high adjustment accuracy, and can perform fast and precise swing elimination actions.

[0123] In actual operation, the desired displacement at the end of the suspension rope 300 to eliminate the swing can be calculated based on parameters such as the current swing angle and the rate of change of the swing angle. The drive mechanism 400 at the end of the suspension rope 300 is then driven to move to the desired displacement to quickly eliminate the swing until the swing is eliminated.

[0124] In some embodiments, controlling the drone to perform a de-swaying action corresponding to the sway level includes: When the swing level of the suspension rope 300 is the second swing level, the control body 201 and the drive mechanism 400 perform the anti-swing action.

[0125] In this embodiment, the swing level of the suspension rope 300 is the second swing level, the suspension rope 300 swings severely, and at the same time, the lateral displacement compensation of the body 201 and the end displacement compensation of the suspension rope 300 of the drive mechanism 400 are activated.

[0126] In actual operation, the lateral displacement of the fuselage 201 provides coarse adjustment displacement, and the drive mechanism 400 outputs secondary compensation torque according to the proportional-integral-derivative (PID) control algorithm to provide fine adjustment displacement. The displacement output of the fuselage 201 and the drive mechanism 400 can be dynamically adjusted by a weighted distributor according to the principle of minimum energy consumption.

[0127] In this embodiment, when the swing level of the suspension rope 300 is the second swing level, the lateral displacement compensation of the fuselage 201 and the active anti-swing of the drive mechanism 400 are used to achieve coordinated control between the UAV body and the end of the suspension rope 300 until the swing level of the suspension rope 300 is reduced to the first swing level, and then the anti-swing action corresponding to the first swing level is performed by the drive mechanism 400.

[0128] It should be noted that during the process of controlling the drone to perform the anti-sway action corresponding to the swing level, the anti-sway effect can be evaluated based on the swing angle information, the swing level of the suspension rope 300 can be updated in real time, and the corresponding anti-sway action can be executed. For example, if the suspension rope 300 is downgraded from the second swing level to the first swing level, the anti-sway action corresponding to the first swing level can be switched to be executed.

[0129] In actual operation, during the control of the drone to perform the anti-sway action corresponding to the sway level, the swing angle and the rate of change of the swing angle of the suspension rope 300 are continuously monitored to evaluate the anti-sway effect. If the swing angle of the suspension rope 300 is less than the first swing angle threshold and the duration of the swing angle change rate being less than the preset change rate threshold is greater than the preset duration threshold, it can be determined that the anti-sway action is completed and the system returns to the normal suspension state.

[0130] The following is a specific example.

[0131] like Figure 6 As shown, step 1, swing angle over-limit detection, involves real-time reading of the swing angle output by the filter. angular velocity of the pendulum and duration of oscillation According to the preset first swing angle threshold Second swing angle threshold and preset rate of change threshold The current swing level of the suspension rope 300 is divided into a first swing level of slight swing and a second swing level of severe swing.

[0132] In actual operation, the duration of the swing angle of the suspension rope 300 within a certain range can be timed to obtain the swing duration.

[0133] For example, if the duration of the swing angle of the suspension rope 300 between 2° and 5° is recorded as 10s, it indicates that the suspension rope 300 swings slightly for 10s. If the duration of the swing angle of the suspension rope 300 between 5° and 10° is recorded as 5s, it indicates that the suspension rope 300 swings severely for 5s.

[0134] Step 2: Strategies are tiered to determine and execute the anti-swing action.

[0135] When the suspension rope 300 is currently at the first swing level, the control drive mechanism 400 actively cancels the swing until the swing is canceled.

[0136] When the suspension rope 300 is currently in the second swing level, the lateral displacement compensation of the fuselage 201 and the active anti-sway mechanism 400 are activated in parallel. After that, it can automatically downgrade to the first swing level.

[0137] Step 3: Evaluate the effectiveness of the swing reduction strategy and continuously monitor it during the implementation period. and ;when < and < Duration When the time exceeds the preset time threshold, the system is deemed to have completed the swing cancellation and returns to the normal hanging state.

[0138] In this embodiment, the swing level of the suspension rope 300 is determined in stages, and the drone is controlled to perform anti-swing actions corresponding to the swing level. The body 201 or the drive mechanism 400 is called progressively according to the swing level to avoid long-term online operation of high-power drive, so as to ensure the anti-swing effect and maximize system energy efficiency.

[0139] In the embodiments of this application, such as Figure 3 As shown, by initializing the swing angle, correcting the swing angle fusion, and eliminating the swing angle in stages, the accurate estimation and effective elimination of the 300° swing angle of the drone's suspending rope are achieved.

[0140] During the initialization of the swing angle estimation, based on the real-time output attitude data from the first attitude sensor 210 of the fuselage 201, the image of the suspension rope acquired by the image acquisition device 230 is projected onto the world coordinate system. A lightweight skeleton extraction network is used to obtain the sub-pixel level two-dimensional skeleton of the center line of the suspension rope 300, and a straight line is fitted. The initial angle between the suspension rope 300 and the vertical direction is obtained by back projection, thus realizing the initial value estimation of the swing angle of the suspension rope 300.

[0141] During the swing angle fusion correction process, the second attitude sensor 220 at the end of the suspension rope 300 measures the angular velocity in real time. After removing the initial deviation, the velocity is integrated to obtain the swing angle variable, which is then added to the initial value of the swing angle to form a real-time swing angle estimate. At the same time, the swing angle detected by the image acquisition device 230 is used as the observation input from time to time. The current estimate is corrected by a filter to suppress drift accumulation error. Furthermore, geometric-dynamic constraints are introduced to correct and update the swing angle state, thereby achieving high-precision swing angle estimation.

[0142] During the graded sway elimination process, the corresponding sway elimination method is selected according to the sway angle size, sway angle change rate and duration. The sway is suppressed by the lateral displacement compensation of the UAV body 201, and the sway is actively eliminated by the drive mechanism 400 of the suspension rope 300 module. The sway is eliminated quickly and accurately while maximizing the system energy efficiency.

[0143] The drone control method provided in this application can be executed by a drone control device. This application uses the drone control device executing the drone control method as an example to illustrate the drone control device provided in this application.

[0144] This application embodiment also provides a control device for an unmanned aerial vehicle (UAV). The UAV fuselage 201 is equipped with an image acquisition device 230 and a first attitude sensor 210. The image acquisition device 230 is used to acquire image information of a sling 300. One end of the sling 300 is connected to the fuselage 201, and the other end of the sling 300 is equipped with a second attitude sensor 220.

[0145] like Figure 7As shown, the control device of the drone includes: The first processing module 710 is used to determine the first swing angle data of the suspension rope 300 based on the suspension rope image information acquired by the image acquisition device 230 and the first attitude data acquired by the first attitude sensor 210. The second processing module 720 is used to determine the swing angle variable of the suspension rope 300 based on the second attitude data collected by the second attitude sensor 220. The third processing module 730 is used to obtain the swing angle information of the suspension rope 300 based on the first swing angle data and the swing angle variable; The fourth processing module 740 is used to control the drone to perform anti-swing actions based on the swing angle information of the suspension rope 300.

[0146] According to the control device for the UAV provided in the embodiments of this application, the first swing angle data of the suspension rope 300 is calculated based on the first attitude data of the fuselage 201 and the image information of the suspension rope. The swing angle variable of the suspension rope 300 is calculated based on the second attitude data of the suspension rope 300. The first swing angle data calculated by fusing the image data and attitude data is used as the reference value for swing angle estimation. The swing angle variable calculated by combining the attitude data of the end of the suspension rope 300 is combined with the swing angle variable calculated in real time to estimate the swing angle of the suspension rope 300, analyze the swing state of the suspension rope 300, and perform anti-swing action. This device can accurately measure the swing angle of the suspension rope 300 connected to the UAV and effectively improve the anti-swing effect.

[0147] In some embodiments, the first processing module 710 is used to determine the first swing angle data of the suspension rope 300 based on the suspension rope image information acquired by the image acquisition device 230 and the first attitude data acquired by the first attitude sensor 210, including: Based on the first attitude data, the first transformation parameters are determined. The first transformation parameters are used to characterize the transformation relationship between the reference coordinate system and the world coordinate system of the image acquisition device 230. Based on the image information of the suspension rope, skeleton extraction and line fitting are performed to obtain the first direction vector of the suspension rope 300 in the reference coordinate system; Based on the first transformation parameters and the first direction vector, the second direction vector of the suspension rope 300 in the world coordinate system is obtained; Based on the second direction vector, the length information of the suspension rope 300, and the position information of the connection point between the suspension rope 300 and the fuselage 201, the first swing angle data is obtained.

[0148] In some embodiments, the first swing angle data includes a first initial swing angle; The third processing module 730 is used to obtain the swing angle information of the suspension rope 300 based on the first swing angle data and the swing angle variable, including: Add the first initial swing angle and the swing angle variable to obtain the first swing angle of the suspension rope 300.

[0149] In some embodiments, the first swing angle data further includes a first instantaneous swing angle; after obtaining the first swing angle of the suspension rope 300, the third processing module 730 is further configured to: In response to the image acquisition device 230 meeting the preset observation conditions at the first moment, the first instantaneous swing angle corresponding to the first moment is acquired; Based on the initial swing angle, construct the observation residuals; Based on the observed residuals, the first swing angle is corrected to obtain the second swing angle of the suspension rope at 30°.

[0150] In some embodiments, after obtaining the second swing angle of the suspension rope 300, the third processing module 730 is further configured to: Based on the length information of the suspension rope 300 and the pendulum dynamics model, constrained residuals are constructed. Based on the constraint residual, the second swing angle is corrected to obtain the third swing angle of the suspension rope 300.

[0151] In some embodiments, the second processing module 720 is used to determine the swing angle variable of the suspension rope 300 based on the second attitude data collected by the second attitude sensor 220, including: Based on the second attitude data, the first angular velocity data of the suspension rope 300 is obtained; Based on the first angular velocity data, the first angular velocity variable of the suspension rope 300 is obtained; Integrating the first angular velocity variable yields the swing angle variable.

[0152] In some embodiments, the second processing module 720 is used to obtain the first angular velocity variable of the suspension rope 300 based on the first angular velocity data, including: Based on the second attitude data, the initial angular velocity of the suspension rope 300 is obtained. The difference between the acquisition time of the initial angular velocity and the acquisition time of the first swing angle data is less than a preset time interval. Based on the first angular velocity data and the initial angular velocity, the first angular velocity variable is obtained.

[0153] In some embodiments, the fourth processing module 740 is used to control the drone to perform anti-sway actions based on the swing angle information of the suspension rope 300, including: Based on the swing angle information, the swing level of the suspension rope 300 is determined. The swing angle information includes the swing angle and the rate of change of the swing angle of the suspension rope 300. Control the drone to perform anti-sway actions corresponding to the sway level.

[0154] In some embodiments, the fourth processing module 740 is configured to determine the swing level of the suspension rope 300 based on the swing angle information, including: If the swing angle of the suspension rope 300 is greater than or equal to the first swing angle threshold and less than the second swing angle threshold, and the rate of change of the swing angle is less than the preset rate of change threshold, the swing level of the suspension rope 300 is determined to be the first swing level. Alternatively, if the swing angle of the suspension rope 300 is greater than or equal to the second swing angle threshold, or the rate of change of the swing angle is greater than or equal to the preset rate of change threshold, the swing level of the suspension rope 300 is determined to be the second swing level.

[0155] In some embodiments, the other end of the suspension rope 300 is further provided with a drive mechanism 400, which is used to drive the other end of the suspension rope 300 to move. The fourth processing module 740 is used to control the UAV to perform anti-sway actions corresponding to the sway level, including: When the swing level of the suspension rope 300 is the first swing level, the control drive mechanism 400 performs a swing-off action; Alternatively, when the swing level of the suspension rope 300 is the second swing level, the control body 201 and the drive mechanism 400 perform a swing-off action.

[0156] The control device for the drone in this application embodiment can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip.

[0157] The drone control device provided in this application embodiment can realize the various processes implemented in the above-described drone control method embodiment. To avoid repetition, it will not be described again here.

[0158] This application also provides an unmanned aerial vehicle (UAV).

[0159] The drone includes a fuselage 201 and a control device as described above. The fuselage 201 is equipped with an image acquisition device 230 and a first attitude sensor 210. The image acquisition device 230 is used to acquire image information of the suspension rope 300. One end of the suspension rope 300 is connected to the fuselage 201, and the other end of the suspension rope 300 is equipped with a second attitude sensor 220. The control device is connected to the image acquisition device 230, the first attitude sensor 210, and the second attitude sensor 220.

[0160] like Figure 3 As shown, the control device achieves accurate estimation and effective elimination of the 300° swing angle of the drone's sling by means of swing angle estimation initialization, swing angle fusion correction, and graded swing elimination.

[0161] During the initialization of the swing angle estimation, based on the real-time output attitude data from the first attitude sensor 210 of the fuselage 201, the image of the suspension rope acquired by the image acquisition device 230 is projected onto the world coordinate system. A lightweight skeleton extraction network is used to obtain the sub-pixel level two-dimensional skeleton of the center line of the suspension rope 300, and a straight line is fitted. The initial angle between the suspension rope 300 and the vertical direction is obtained by back projection, thus realizing the initial value estimation of the swing angle of the suspension rope 300.

[0162] During the swing angle fusion correction process, the second attitude sensor 220 at the end of the suspension rope 300 measures the angular velocity in real time. After removing the initial deviation, the velocity is integrated to obtain the swing angle variable, which is then added to the initial value of the swing angle to form a real-time swing angle estimate. At the same time, the swing angle detected by the image acquisition device 230 is used as the observation input from time to time. The current estimate is corrected by a filter to suppress drift accumulation error. Furthermore, geometric-dynamic constraints are introduced to correct and update the swing angle state, thereby achieving high-precision swing angle estimation.

[0163] During the graded sway elimination process, the corresponding sway elimination method is selected according to the sway angle size, sway angle change rate and duration. The sway is suppressed by the lateral displacement compensation of the UAV body 201, and the sway is actively eliminated by the drive mechanism 400 of the suspension rope 300 module. The sway is eliminated quickly and accurately while maximizing the system energy efficiency.

[0164] According to the UAV provided in the embodiments of this application, the first swing angle data of the suspension rope 300 is calculated based on the first attitude data of the fuselage 201 and the image information of the suspension rope. The swing angle variable of the suspension rope 300 is calculated based on the second attitude data of the suspension rope 300. The first swing angle data calculated by fusing the image data and attitude data is used as the reference value for swing angle estimation. The swing angle variable calculated by combining the end attitude data of the suspension rope 300 is combined with the swing angle variable calculated in real time to estimate the swing angle of the suspension rope 300, analyze the swing state of the suspension rope 300, and perform anti-swing action. This can accurately measure the swing angle of the suspension rope 300 connected to the UAV and effectively improve the anti-swing effect.

[0165] In some embodiments, such as Figure 8 As shown, this application embodiment also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements the various processes of the above-described drone control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0166] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0167] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described drone control method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0168] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0169] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described drone control method.

[0170] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0171] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described UAV control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0172] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0173] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0175] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0176] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0177] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A control method for an unmanned aerial vehicle (UAV), characterized in that, The drone's fuselage is equipped with an image acquisition device and a first attitude sensor. The image acquisition device is used to acquire image information from a sling. One end of the sling is connected to the fuselage, and the other end of the sling is equipped with a second attitude sensor. The method includes: Based on the image information of the suspension rope acquired by the image acquisition device and the first attitude data acquired by the first attitude sensor, the first swing angle data of the suspension rope is determined; Based on the second attitude data collected by the second attitude sensor, the swing angle variable of the suspension rope is determined; Based on the first swing angle data and the swing angle variable, the swing angle information of the suspension rope is obtained; Based on the swing angle information of the suspension rope, the drone is controlled to perform a swing-off action.

2. The control method for an unmanned aerial vehicle according to claim 1, characterized in that, The determination of the first swing angle data of the suspension rope based on the suspension rope image information acquired by the image acquisition device and the first attitude data acquired by the first attitude sensor includes: Based on the first posture data, a first transformation parameter is determined, which is used to characterize the transformation relationship between the reference coordinate system and the world coordinate system of the image acquisition device. Based on the image information of the suspension rope, skeleton extraction and line fitting are performed to obtain the first direction vector of the suspension rope in the reference coordinate system; Based on the first transformation parameter and the first direction vector, the second direction vector of the suspension rope in the world coordinate system is obtained; Based on the second direction vector, the length information of the suspension rope, and the position information of the connection point between the suspension rope and the fuselage, the first swing angle data is obtained.

3. The control method for an unmanned aerial vehicle according to claim 1, characterized in that, The first swing angle data includes the first initial swing angle; The step of obtaining the swing angle information of the suspension rope based on the first swing angle data and the swing angle variable includes: The first initial swing angle and the swing angle variable are added together to obtain the first swing angle of the suspension rope.

4. The control method for an unmanned aerial vehicle according to claim 3, characterized in that, The first swing angle data also includes a first instantaneous swing angle; after obtaining the first swing angle of the suspension rope, the method further includes: In response to the image acquisition device meeting the preset observation conditions at a first moment, the first instantaneous swing angle corresponding to the first moment is acquired; Based on the first instantaneous swing angle, construct the observation residual; Based on the observed residual, the first swing angle is corrected to obtain the second swing angle of the suspension rope.

5. The control method for an unmanned aerial vehicle according to claim 4, characterized in that, After obtaining the second swing angle of the suspension rope, the method further includes: Based on the length information of the suspension rope and the pendulum dynamics model, a constraint residual is constructed; Based on the constraint residual, the second swing angle is corrected to obtain the third swing angle of the suspension rope.

6. The control method for a drone according to any one of claims 1-5, characterized in that, The determination of the swing angle variable of the suspension rope based on the second attitude data collected by the second attitude sensor includes: Based on the second attitude data, the first angular velocity data of the suspension rope is obtained; Based on the first angular velocity data, the first angular velocity variable of the suspension rope is obtained; Integrating the first angular velocity variable yields the swing angle variable.

7. The control method for an unmanned aerial vehicle according to claim 6, characterized in that, The process of obtaining the first angular velocity variable of the suspension rope based on the first angular velocity data includes: Based on the second attitude data, the initial angular velocity of the suspension rope is obtained, and the difference between the acquisition time of the initial angular velocity and the acquisition time of the first swing angle data is less than a preset time interval. Based on the first angular velocity data and the initial angular velocity, the first angular velocity variable is obtained.

8. The control method for an unmanned aerial vehicle according to any one of claims 1-5, characterized in that, The control of the drone to perform anti-sway actions based on the swing angle information of the suspension rope includes: Based on the swing angle information, the swing level of the suspension rope is determined, wherein the swing angle information includes the swing angle and the rate of change of the swing angle of the suspension rope; Control the drone to perform a sway-reducing action corresponding to the sway level.

9. The control method for an unmanned aerial vehicle according to claim 8, characterized in that, Determining the swing level of the suspension rope based on the swing angle information includes: If the swing angle of the suspension rope is greater than or equal to a first swing angle threshold and less than a second swing angle threshold, and the rate of change of the swing angle is less than a preset rate of change threshold, the swing level of the suspension rope is determined to be the first swing level. Alternatively, if the swing angle of the suspension rope is greater than or equal to the second swing angle threshold, or if the swing angle change rate is greater than or equal to the preset change rate threshold, the swing level of the suspension rope is determined to be the second swing level.

10. The control method for an unmanned aerial vehicle according to claim 9, characterized in that, The other end of the hoisting rope is also provided with a driving mechanism, which is used to drive the other end of the hoisting rope to move. The control of the drone to perform a sway-reducing action corresponding to the sway level includes: When the swing level of the suspension rope is the first swing level, the drive mechanism is controlled to perform a swing-off action; Alternatively, if the swing level of the suspension rope is the second swing level, the machine body and the drive mechanism are controlled to perform an anti-swing action.

11. A control device for an unmanned aerial vehicle (UAV), characterized in that, The drone's fuselage is equipped with an image acquisition device and a first attitude sensor. The image acquisition device is used to acquire image information from a sling. One end of the sling is connected to the fuselage, and the other end of the sling is equipped with a second attitude sensor. The device includes: The first processing module is used to determine the first swing angle data of the suspension rope based on the suspension rope image information acquired by the image acquisition device and the first attitude data acquired by the first attitude sensor. The second processing module is used to determine the swing angle variable of the suspension rope based on the second attitude data collected by the second attitude sensor. The third processing module is used to obtain the swing angle information of the suspension rope based on the first swing angle data and the swing angle variable; The fourth processing module is used to control the drone to perform anti-swing actions based on the swing angle information of the suspension rope.

12. An unmanned aerial vehicle (UAV), characterized in that, include: The fuselage is equipped with an image acquisition device and a first attitude sensor. The image acquisition device is used to acquire image information of the suspension rope. One end of the suspension rope is connected to the fuselage, and the other end of the suspension rope is equipped with a second attitude sensor. The control device for the unmanned aerial vehicle as described in claim 11, wherein the control device is connected to the image acquisition device, the first attitude sensor and the second attitude sensor.

Images