Unmanned aerial vehicle hoisting transportation swing suppression method based on visual feedback

Abstract

The invention discloses an unmanned aerial vehicle hoisting transportation swing suppression method based on visual feedback, and relates to the technical field of unmanned aerial vehicles, and the method comprises the following steps: obtaining depth information through a binocular vision camera installed on an unmanned aerial vehicle, and calculating the real-time distance between an object and the unmanned aerial vehicle; carrying out data acquisition through a binocular vision camera mounted on the unmanned aerial vehicle, and calculating the swing amplitude and frequency of the hoisted object; according to the obtained calculation data, unmanned aerial vehicle attitude control is performed in combination with an unmanned aerial vehicle attitude adaptive control strategy, so that anti-swing is realized, and the stability of the unmanned aerial vehicle is ensured; according to the method, the binocular vision camera carried by the unmanned aerial vehicle is utilized, and through image processing and a deep learning algorithm, rapid, accurate and robust recognition of targets in pictures and videos and accurate tracking and positioning of specific targets are realized, and the movement amount of an object is accurately calculated; therefore, the efficiency and effect of tracking and suppressing swinging of the unmanned aerial vehicle according to object swinging are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, in particular to a method for swing suppression of unmanned aerial vehicle hoisting transportation based on visual feedback. BACKGROUND

[0002] Unmanned aerial vehicles have been widely used in many fields such as military, security, rescue, transportation and agriculture due to their high flexibility, low cost, high safety and other advantages. Among them, unmanned aerial vehicle transportation, as one of its core functions, can effectively reduce cost, pollution, save time and relieve ground transportation pressure by hoisting transportation.

[0003] However, when the unmanned aerial vehicle is hoisting and transporting, the swing of the hoisted goods will have multiple negative effects through the coupling of the rope-body dynamics, the swing inertia moment will interfere with the control accuracy of the unmanned aerial vehicle, and in severe cases, it will cause the unmanned aerial vehicle to lose stability and crash, seriously affecting the flight safety of the unmanned aerial vehicle. Therefore, a method for swing suppression of unmanned aerial vehicle hoisting transportation based on visual feedback is needed to solve the above problems. SUMMARY

[0004] The purpose of the present application is to provide a method for swing suppression of unmanned aerial vehicle hoisting transportation based on visual feedback to solve the problems existing in the prior art mentioned in the background.

[0005] To achieve the above purpose, the present application provides the following technical scheme:

[0006] A method for swing suppression of unmanned aerial vehicle hoisting transportation based on visual feedback, comprising the following steps:

[0007] S1: Obtain the depth information through the binocular vision camera installed on the unmanned aerial vehicle, and calculate the real-time distance between the object and the unmanned aerial vehicle;

[0008] S2: Data acquisition is performed through the binocular vision camera installed on the unmanned aerial vehicle, and the swing amplitude and frequency of the hoisted goods are calculated;

[0009] S3: According to the obtained calculation data, combined with the unmanned aerial vehicle attitude adaptive control strategy, the unmanned aerial vehicle attitude control is performed to realize swing suppression and ensure the stability of the unmanned aerial vehicle.

[0010] Preferably, the specific steps of S1 are as follows:

[0011] Binocular vision distance measurement: calculate the distance Δz between the unmanned aerial vehicle and the hoisted goods based on the principle of parallax, the formula is:

[0012]

[0013] Wherein B is the baseline distance of the binocular vision camera, f is the actual focal length of the binocular vision camera lens, Δd is the disparity of the left and right image matching points, and s is the pixel-physical size conversion coefficient.

[0014] Preferably, the specific steps of S2 are:

[0015] S21: The binocular vision camera of the unmanned aerial vehicle extracts image information, segments the hoisting object contour using the improved YOLOv8 model, and calculates the continuous frame pixel displacement (Δx image ,Δy image ) of the hoisting object contour on the x-axis and y-axis using the optical flow method;

[0016] S22: Map the image displacement to the three-dimensional space through coordinate system conversion, establish a swing trajectory model, convert the pixel displacement to actual physical displacement, and then combine the depth information to calculate the angle of the object on the horizontal plane in the three-dimensional space, i.e. the swing angle of the object on the x-axis and y-axis;

[0017] S23: Calculate the swing angular rate of the hoisting object on the x-axis and y-axis on the horizontal plane in the three-dimensional space using the second-order difference method;

[0018] S24: Calculate the frequency and extract the main frequency component by analyzing the displacement time series data using FFT.

[0019] Preferably, the specific steps of S3 are:

[0020] S31: Divide the swing amplitude and swing frequency into three fuzzy sets, design a dynamic fuzzy rule base, and calculate the accurate gain value using the barycentric method;

[0021] S32: Add the operation of eliminating the swing of the unmanned aerial vehicle by taking the acceleration of the unmanned aerial vehicle as a consideration factor, and the unmanned aerial vehicle executes the acceleration threshold grading control strategy;

[0022] S33: Use outer loop control to suppress the swing of the hoisted object, design the control to generate the expected speed increment command on the x-axis and y-axis, and suppress the low-frequency swing;

[0023] S34: Finally, generate the expected speed increment on the x-axis and y-axis according to the swing amplitudes A x , A y , the frequencies f x , f y , and the dynamic gain of the fuzzy PID output fed back by the binocular vision.

[0024] Preferably, the specific calculation method of the swing angle of the object on the x-axis and y-axis in S22 is:

[0025]

[0026]

[0027] wherein k is a pixel-physical size conversion coefficient determined by binocular vision camera calibration parameters, Δz is the depth information of the binocular vision camera, i.e. the real-time distance between the object and the unmanned aerial vehicle, Δx image is the pixel displacement of the object in the horizontal direction X-axis in the image coordinate system, Δy image is the pixel displacement of the object in the horizontal direction Y-axis in the image coordinate system, θx is the swing angle of the hoisted object in the x-axis, i.e. the amplitude in the x-axis, and θy is the swing angle of the hoisted object in the y-axis, i.e. the amplitude in the y-axis.

[0028] Preferably, the swing angular velocity ω x and ω y of the hoisted object in the horizontal plane in the three-dimensional space in S23 are calculated as follows:

[0029]

[0030] Preferably, the specific steps in S24 are as follows:

[0031] The swing angle signals θ x and θ y are sampled at equal intervals to obtain discrete time series data, the sampling time interval is Δt = 1 / Fs, the sampling frequency is Fs, the sampling frequency is at least 2 times the maximum frequency at which the hoisted object can swing, the total number of sampling points is N, and the total time length T = N·Δt.

[0032] FFT transform of the swing angle time series data, converts the time domain signal to the frequency domain, for data with a time length of N, the discrete signal time series x[n] is decomposed into the superposition of multiple complex sinusoidal waves, and the frequency component after FFT transform is:

[0033]

[0034] wherein N is the number of sampling points, x[n] is the discrete time domain signal, i.e. the time series data, is the swing angle signal collected at the discrete time point n, n is the time index, the value range is n = 0, 1, 2, 3, …, N-1, corresponding to the collection time t = n·Δt, X[k] is the discrete frequency domain signal, i.e. the complex frequency domain representation after FFT transform of x[n], which describes the amplitude and phase of the signal at frequency f k , k corresponds to the frequency index in the frequency domain, the value range is k = 0, 1, 2, 3, …, N-1, and the corresponding physical frequency is f k ;

[0035] Amplitude calculation, each frequency component X(f k) is a complex number, and the modulus |X(f k )| represents the amplitude of the frequency component:

[0036]

[0037] where f k is the kth frequency component, X(f k ) is the complex amplitude at f k component, Re(X(f k )) is the real part of X(f k ), and Im(X(f k )) is the imaginary part of X(f k );

[0038] By analyzing |X(f k )|, the frequency with the maximum amplitude, i.e. the component with the lowest frequency, can be found, which is the main frequency f main of the swing, which is the main frequency component of the displacement time series data, i.e. the swing frequency of the hoisted object relative to the UAV. According to the swing angle signals θ x and θ y , the swing main frequencies f x and f y of the hoisted object on the x-axis and y-axis are calculated respectively by equal-interval sampling.

[0039] Preferably, the specific steps in S31 are as follows:

[0040] According to the swing amplitude classification to adjust the gain, the swing amplitude is divided into three fuzzy sets, small amplitude S: A < 5°, medium amplitude M: 5°≤A≤15°, and large amplitude L: A≥15°, wherein the swing amplitude is the absolute value of the swing angle of the hoisted object on the x-axis and y-axis respectively;

[0041] The swing frequency is also divided into three fuzzy sets, low frequency LF: 0Hz < f≤1Hz, medium frequency MF: 1Hz < f≤3Hz, and high frequency HF: f>3Hz, wherein the swing frequency is the swing frequency of the hoisted object on the x-axis and y-axis respectively;

[0042] Defuzzification method: the precise gain value is calculated by using the center of area method COA:

[0043]

[0044] where μ i is the rule triggering weight, K p , K d are the gain values defined in the rule table, and i takes the value of i=1, 2, 3, …, 9.

[0045] Preferably, the specific steps in S32 are as follows:

[0046] The acceleration of the UAV is divided into three fuzzy sets, high acceleration HA: a≥2m / s 2 , medium acceleration MA: 0.2m / s 2 ≤a≤2m / s 2 , zero acceleration ZA: a≤0.2m / s 2 When the acceleration of the UAV is greater than the set safety threshold, the anti-swing is completely disabled, when a≥a max , the K p =0, K d =0 is forcibly set, and the flight stability is preferentially ensured; when a mid <a<a max , the gain is proportionally reduced:

[0047]

[0048] Preferably, the specific steps in S33 are:

[0049] The control law is designed, and the input is composed of an equivalent control term and a switching term:

[0050] Δv x =K p ·θx+K d ·ω x +η·sat(s x / Φ)

[0051] Wherein the equivalent control term is K p ·θx+K d ·ω x , and the K p , K d based on the fuzzy PID output generate the basic compensation amount, and the switching term is η·sat(s x / Φ), s x , s y take the swing angle, swing angular rate and frequency as input:

[0052] s x =K A ·θx·sin(2πf x t)+K d ·ω x

[0053] s y =K A ·θy·sin(2πf y t)+K d ·ω y

[0054] where K A is the amplitude-velocity gain coefficient, K d is the differential damping coefficient;

[0055] switching gain η, which is adaptively adjusted according to the swing energy, sat(): saturation function, boundary layer thickness Φ = 0.1, to suppress chattering:

[0056]

[0057] where η max is the maximum switching gain, η min is the minimum switching gain, E is the swing energy, E max is the maximum swing energy:

[0058]

[0059] where m is the mass of the hoisted object;

[0060] Finally, the expected velocity increments in the x-axis and y-axis are generated according to the swing amplitudes A x , A y and frequencies f x and f y feedback from binocular vision and the dynamic gain of fuzzy PID output:

[0061] Δv x = K p · θx+ K d · ω x + η · sat(s x / Φ)

[0062] Δv y = K p · θy+ K d · ω y + η · sat(s y / Φ)

[0063] where the proportional term K p · θx, K p · θy: generates compensation according to the current swing amplitude, K p is the proportional gain coefficient, which is used to generate the compensation of the velocity according to the swing amplitude of the hoisted object, the differential term K d · ω x , K d · ω y suppresses the amplitude change rate to prevent overshoot, K d is the differential gain coefficient, which is used to suppress the amplitude change rate, i.e., the angular rate ω x , ω ycaused by the dynamic error, the switching term η sat(s x / Φ), η sat(s y / Φ) resist model uncertainty and external disturbance; disturbance suppression is realized by adaptive gain η and saturation function sat(), the physical essence is high-frequency nonlinear compensation force, resist model uncertainty and external disturbance, and the final desired speed of the unmanned aerial vehicle is the given speed of the current unmanned aerial vehicle plus the given desired speed increment.

[0064] Compared with the prior art, the beneficial effects of the present application are:

[0065] The present application utilizes the binocular vision camera carried by the unmanned aerial vehicle, realizes the rapid, accurate and robust identification of the target in the picture and video, and the accurate tracking and positioning of the specific target through image processing and deep learning algorithm, and accurately calculates the movement amount of the object, so as to improve the efficiency and effect of the unmanned aerial vehicle in tracking and suppressing the swing according to the swing of the object. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 It is a schematic diagram of the coordinate system of the present application.

[0067] Figure 2 It is a schematic diagram of the binocular vision camera of the present application.

[0068] Figure 3 It is a swing suppression strategy diagram according to the acceleration of the unmanned aerial vehicle of the present application. DETAILED DESCRIPTION

[0069] In order to make the technical means, creative features, purposes and effects realized by the present application easy to understand, the present application is further described below in combination with specific embodiments.

[0070] In order to directly perceive the swing state of the hoisted object and timely capture the sudden swing, the nonlinear characteristics of the hoisting system such as rope flexibility deformation and air dynamic disturbance are accurately described, and the swing frequency of the hoisted object is accurately obtained. A new idea is to use a binocular vision camera. When monitoring the hoisted object, the binocular vision camera can not only obtain rich image information, but also can feedback the object swing state without additional sensor configuration. This not only avoids the noise interference problem in multi-sensor data fusion, but also uses the depth information provided by the binocular camera to more accurately calculate the object swing angle, improve the robustness in complex scenes, and design a swing suppression method to more effectively suppress the swing of the hoisted object of the unmanned aerial vehicle.

[0071] Please refer to Figures 1-3 The present application provides the following technical solutions:

[0072] A swing suppression method for unmanned aerial vehicle hoisting and transportation based on visual feedback, comprising the following steps:

[0073] The coordinate system is defined as shown in the figure, wherein the x-axis is the direction of the hoisted object in the horizontal direction consistent with the pitch direction of the UAV, i.e., the head direction of the UAV as the positive direction of the x-axis, the y-axis is the direction of the hoisted object in the horizontal direction consistent with the roll direction of the UAV, i.e., the right side direction of the UAV as the positive direction of the y-axis, and the z-axis is the vertical direction, with the direction pointing to the ground as the positive direction of the z-axis. Figure 1

[0074] The binocular vision camera mounted on the UAV is used to collect the RGB image and depth information of the hoisted object in real time, track the object contour through a feature matching algorithm, identify the hoisting point and object contour through a YOLO algorithm, calculate the centroid coordinates, output the lateral offset (Δx, Δy) and swing frequency, the inertial measurement unit (IMU) mounted on the UAV acquires the attitude angle (roll / pitch / yaw) and acceleration data of the UAV, and the GPS system mounted on the UAV acquires the speed of the UAV.

[0075] S1: Obtain the depth information through the binocular vision camera mounted on the UAV, and calculate the real-time distance between the object and the UAV. Specifically, the following steps are included:

[0076] The binocular vision camera mounted on the UAV measures the distance to the hoisted object below, finds the corresponding pixel points of the same object in the left and right images, and calculates the parallax:

[0077] Δd = uL-uR

[0078] The distance Δz between the UAV and the hoisted object is calculated based on the parallax principle, and the formula is:

[0079]

[0080] where B is the baseline distance of the binocular vision camera, in meters (m), f is the actual focal length of the binocular vision camera lens, Δd is the parallax of the matching points in the left and right images, in pixels, and s is the pixel size conversion coefficient, i.e., the physical width of a single pixel, in meters per pixel (m / pixel).

[0081]

[0082] where W sensor is the sensor width of the binocular vision camera, in meters (m), and W pixel is the image resolution, in pixels.

[0083] S2: Data acquisition is performed through the binocular vision camera mounted on the UAV, and the swing amplitude and frequency of the object on the x-axis and y-axis are calculated, specifically including the following steps:

[0084] ​S21: The binocular vision camera of the unmanned aerial vehicle extracts image information, the binocular vision camera is installed below the unmanned aerial vehicle, directly opposite the below hoisting object, is used for measuring the hoisting object distance and swing, calibrates the binocular vision camera, internal parameter calibration: through the chessboard calibration method, the focal length f, the principal point coordinates (u0, v0), the distortion coefficient and other parameters are obtained, and the binocular coordinate system is aligned. External parameter calibration: the baseline distance b and the relative pose (rotation matrix R, translation vector T) between the two cameras of the binocular vision camera are calculated, and a binocular three-dimensional coordinate system is established. Through the PTP protocol, the binocular image time alignment is ensured, the high-precision clock synchronization is used to eliminate the time sequence deviation between the sensors, the binocular vision camera is connected to the same local area network, the switch supporting the PTP protocol is interconnected, the flight control of the unmanned aerial vehicle is specified as the PTP master clock, and the binocular vision camera is specified as the slave clock. The master clock periodically sends a synchronization message (Sync) containing the master clock timestamp; the slave clock records the receiving time, and requests the master clock to feed back the transmission delay through the Delay_Req message. The hardware timestamp function of FPGA or special network card is enabled to eliminate the software processing delay. According to the round-trip time difference (Offset) and the transmission delay (Delay), the local time of the slave clock is dynamically adjusted, and the formula is:

[0085] t 从 =t 主 +Offset-Delay

[0086] The edge deep learning model of the hoisting object is based on the improved YOLOv8 model, the hoisting object contour is segmented, the high-contrast region (such as the corner point, the texture region) is selected on the contour, the SIFT / SURF algorithm is used to extract the feature points, and the optical flow method is used to calculate the pixel displacement (Δx image ,Δy image ) between the continuous frames.

[0087] S22: The image displacement is mapped to the three-dimensional space through the coordinate system conversion, the swing trajectory model is established, the pixel displacement is converted into the actual physical displacement, and then the depth information is combined to calculate the included angle of the hoisting object in the x-axis and y-axis planes in the three-dimensional space, that is, the swing angle of the object in the x-axis and y-axis:

[0088]

[0089] Wherein, k is the pixel-physical size conversion coefficient (unit: meter / pixel), which is determined by the binocular vision camera calibration parameters (focal length, resolution, etc.), that is, the actual physical displacement corresponding to the unit pixel displacement, which changes with the depth Δz. Δz is the depth information of the binocular vision camera, that is, the real-time distance between the object and the unmanned aerial vehicle (unit: meter), Δx image is the pixel displacement (unit: pixel) of the object in the vertical direction (X-axis) in the image coordinate system, and Δy imageθx is the pixel displacement (in pixels) of the object in the vertical direction (Y-axis) of the image coordinate system. θx is the swing angle (in radians) of the hoisted object on the x-axis, i.e., the amplitude on the x-axis. θy is the swing angle (in radians) of the hoisted object on the y-axis, i.e., the amplitude on the y-axis.

[0090] S23: Calculate the angular velocities of an object on the x-axis and y-axis of the horizontal plane in three-dimensional space using the second-order difference method:

[0091]

[0092] S24: Calculate the frequency by extracting the dominant frequency component from the displacement time series data using FFT analysis.

[0093] For the swing angle signal θ x and θ y Perform equal-interval sampling to obtain discrete-time series data. Let the sampling time interval be Δt = 1 / Fs, the sampling frequency be Fs (unit: Hz), the sampling frequency be at least twice the maximum frequency at which the hoisted object may swing, and the total number of sampling points be N. Then the total signal duration is T = N·Δt.

[0094] The FFT transform of the swing angle time series data converts the time-domain signal to the frequency domain. For data with a time length of N, the discrete signal time series x[n] is decomposed into a superposition of multiple complex sine waves. The frequency components after the FFT transform are:

[0095]

[0096] Where N is the number of sampling points, x[n] is the discrete time domain signal, i.e., time series data, which is the swing angle signal collected at discrete time point n. n is the time index, with a value range of n = 0, 1, 2, 3, ..., N-1, corresponding to the acquisition time t = n·Δt. X[k] is the discrete frequency domain signal, i.e., the complex frequency domain representation after x[n] undergoes FFT transformation, describing the amplitude and phase of the signal at frequency fk. k corresponds to the frequency index in the frequency domain, with a value range of k = 0, 1, 2, 3, ..., N-1, corresponding to the physical frequency fk.

[0097] Amplitude calculation, each frequency component X(f) obtained by FFT transformation k ) is a complex number whose modulus is |X(f) k | indicates the amplitude of the frequency component:

[0098]

[0099] Where f k For the k-th frequency component, X(f) k ) for in f kThe complex amplitude under the component, Re(X(f k )) is the real part of X(f k ), and Im(X(f k )) is the imaginary part of X(f k ).

[0100] By analyzing |X(f k )|, the frequency with the maximum amplitude, i.e., the component with the lowest frequency, can be found, which is the main frequency f main of the swing, which is the main frequency component of the displacement time series data, i.e., the swing frequency of the hoisted object relative to the unmanned aerial vehicle. According to the swing angle signals θ x and θ y , the swing main frequencies f x and f y of the hoisted object on the x-axis and y-axis are calculated respectively by equal-interval sampling.

[0101] S3: According to the unmanned aerial vehicle attitude adaptive control strategy, the unmanned aerial vehicle attitude control is controlled to realize swing elimination and ensure the stability of the unmanned aerial vehicle, which specifically includes the following steps:

[0102] S31: According to the swing amplitude grading adjustment gain, the swing amplitude is divided into three fuzzy sets, small amplitude (S): A < 5°, medium amplitude (M): 5°≤A≤15°, large amplitude (L): A≥15°. The swing amplitude is the absolute value of the swing angle of the hoisted object on the x-axis and y-axis respectively. The membership function is designed (using trapezoidal function). The membership degree calculation formula is:

[0103]

[0104]

[0105] The swing frequency is also divided into three fuzzy sets, low frequency (LF): 0Hz<f≤1Hz, medium frequency (MF): 1Hz<f≤3Hz, high frequency (HF): f>3Hz. The swing frequency is the swing frequency of the hoisted object on the x-axis and y-axis respectively. The membership function is designed (using triangular function). The membership degree calculation formula is:

[0106]

[0107] Dynamic fuzzy rule base design:

[0108]

[0109]

[0110] Output variable:

[0111]

[0112] For each rule, take the minimum value of the current antecedent (A and f) membership as the trigger weight μ of the ith rule i :

[0113] μ1 = min(μ S (A), μ LF (f))

[0114] μ2 = min(μ S (A), μ MF (f))

[0115] μ3 = min(μ S (A), μ HF (f))

[0116] μ4 = min(μ M (A), μ LF (f))

[0117] μ5 = min(μ M (A), μ MF (f))

[0118] μ6 = min(μ M (A), μ HF (f))

[0119] μ7 = min(μ L (A), μ LF (f))

[0120] μ8 = min(μ L (A), μ MF (f))

[0121] μ9 = min(μ L (A), μ HF (f))

[0122] Defuzzification method: use the center of area method (COA) to calculate the accurate gain value:

[0123]

[0124] Where μ i is the rule trigger weight, K p , K d is the gain value defined in the rule table, and i takes the value i = 1, 2, 3, …, 9, corresponding to the 9 rules of the dynamic fuzzy rule base.

[0125] S32: Divide the acceleration of the unmanned aerial vehicle into three fuzzy sets, high acceleration (HA): a ≥ 2 m / s 2Medium acceleration (MA): 0.2 m / s 2 ≤ a ≤ 2 m / s 2 Zero acceleration (ZA): a ≤ 0.2 m / s 2 The acceleration threshold grading control strategy takes the acceleration of the UAV as a consideration factor and adds the UAV sway elimination operation. The UAV executes the acceleration threshold grading control strategy. When the acceleration of the UAV is greater than the set safety threshold, the sway elimination is completely disabled. When a ≥ a max K p = 0, K d = 0, the flight stability is preferentially ensured. When a < amid, that is, the acceleration is 0, the UAV is in a static or uniform speed state, the complete sway elimination is performed, and the complete fuzzy PID rule base is enabled. When a mid <a < a max The gain is proportionally reduced:

[0126]

[0127] The introduction of the acceleration feedback in the fuzzy PID design is a reasonable and necessary technical means. Through the triple mechanisms of dynamic decoupling, actuator protection, and safety enhancement, the comprehensive performance of the UAV hoisting system is significantly improved. The sway elimination strategy according to the UAV acceleration is applicable to any flight route.

[0128] S33: The outer loop control (position control) is used to suppress the hoisted object sway. The expected speed increment command (expected speed increment of pitch and roll) on the x-axis and y-axis is designed to suppress the low-frequency sway:

[0129] The control law design is composed of an equivalent control term and a switching term:

[0130] Δv x = K p · θx+ K d · ω x + η · sat(s x / Φ)

[0131] The equivalent control term is K p · θx+ K d · ω x , which is based on the K p , K d of the fuzzy PID output to generate the basic compensation. The switching term is η · sat(s x / Φ). s x , s y take the sway angle (amplitude), sway angular rate, and frequency as inputs:

[0132] s x = K A• θx• sin(2πf x t) + K d • ω x

[0133] s y = K A • θy• sin(2πf y t) + K d • ω y

[0134] where K A is the amplitude-velocity gain coefficient, K d is the differential damping coefficient.

[0135] Switching gain η, adaptive adjustment according to swing energy, sat(): saturation function, boundary layer thickness Φ = 0.1, to suppress chattering:

[0136]

[0137] where η max is the maximum switching gain, η min is the minimum switching gain, E is the swing energy, E max is the maximum swing energy:

[0138]

[0139] where m is the mass of the hoisted object (unit: kg), θxand θyare the swing angles of the hoisted object in the x-axis and y-axis planes, respectively, ω x and ω y are the swing angular velocities of the hoisted object in the x-axis and y-axis planes, respectively.

[0140] Finally, according to the swing amplitudes A x , A y and frequencies f x , f y of the binocular vision feedback and the dynamic gain of the fuzzy PID output, the expected velocity increments in the x-axis and y-axis are generated:

[0141] Δv x = K p • θx+ K d • ω x + η• sat(s x / Φ)

[0142] Δv y = K p • θy+ K d • ω y + η• sat(s y / Φ)

[0143] where θx and θy are the swing amplitudes of the hoisted object on the x-axis and y-axis, i.e. the swing angles of the hoisted object on the x-axis and y-axis in the horizontal plane, the proportional term K p • θx, K p • θy: generates compensation according to the current swing amplitude, K p is the proportional gain coefficient, which is used to generate the compensation of the speed according to the swing amplitude of the hoisted object. The differential term K d • ω x , K d • ω y suppresses the amplitude change rate and prevents overshoot, K d is the differential gain coefficient, which is used to suppress the amplitude change rate, i.e. the angular rate ω x , ω y caused dynamic error. ω x , ω y is the amplitude change rate, i.e. the swing angular rate. The switching term η · sat(s x / Φ), η · sat(s y / Φ): resists model uncertainty and external disturbance. The disturbance suppression is achieved by the adaptive gain η and the saturation function sat(), and the physical essence is a high-frequency nonlinear compensation force. Resists model uncertainty (such as rope length change, load mass fluctuation) and external disturbance (such as gust, turbulence).

[0144] The sliding mode control has strong anti-interference ability to model uncertainty (such as rope length change, mass disturbance) and external wind disturbance, without the need for an accurate dynamic model. The final given speed of the unmanned aerial vehicle is the given speed of the current unmanned aerial vehicle plus the given speed increment.

[0145] The present application aims at detecting the swing amplitude of the hoisted object by the visual feedback of the unmanned aerial vehicle, and the method for suppressing the swing of the object and stabilizing the attitude of the unmanned aerial vehicle. When the binocular vision camera is used, the hoisted object below is recognized, the contour movement of the object is tracked in real time, the pixel displacement between each continuous frame is calculated, and then the actual swing amplitude of the object is calculated. According to the swing data obtained, the corresponding optimized swing damping stabilization strategy is carried out, so as to ensure the safe and stable flight of the unmanned aerial vehicle.

[0146] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for swing suppression of unmanned aerial vehicle hoisting transportation based on visual feedback, characterized in that, The method comprises the following steps: S1: obtaining depth information through a binocular vision camera installed on the unmanned aerial vehicle, and calculating real-time distance between the object and the unmanned aerial vehicle; S2: collecting data through the binocular vision camera installed on the unmanned aerial vehicle, and calculating swing amplitude and frequency of the hoisted object; S3: performing unmanned aerial vehicle attitude control according to the obtained calculation data and combining with an unmanned aerial vehicle attitude adaptive control strategy, so as to realize swing elimination and ensure the stability of the unmanned aerial vehicle. 2.The method of claim 1, wherein, The specific steps of S1 are as follows: Binocular vision distance measurement: the distance Δz between the unmanned aerial vehicle and the hoisted object is calculated based on the parallax principle, and the formula is as follows: Wherein B is the baseline distance of the binocular vision camera, f is the actual focal length of the binocular vision camera lens, Δd is the parallax of the matching points of left and right images, and s is the pixel-physical size conversion coefficient. 3.The method of claim 1, wherein, The specific steps of S2 are as follows: S21: the binocular vision camera of the unmanned aerial vehicle extracts image information, uses an improved YOLOv8 model to segment the hoisting object contour, and combines a light flow method to calculate the continuous frame-to-frame pixel displacement amount (Δx image ,Δy image ) of the hoisting object contour on the x-axis and y-axis; S22: mapping the image displacement to the three-dimensional space through coordinate system conversion, establishing a swing trajectory model, converting the pixel displacement into actual physical displacement, and then combining with the depth information to calculate the included angle of the object in the horizontal plane in the three-dimensional space, that is, the swing angle of the object on the x-axis and y-axis; S23: calculating the swing angular velocity of the hoisted object on the x-axis and y-axis in the horizontal plane in the three-dimensional space by using the second-order difference method; S24: calculating the frequency by extracting the main frequency component through FFT analysis of the displacement time sequence data. 4.The method of claim 1, wherein, The specific steps of S3 are as follows: S31: dividing the swing amplitude and the swing frequency into three fuzzy sets respectively, designing a dynamic fuzzy rule base, and calculating the accurate gain value by using the center of gravity method; S32: taking the acceleration of the unmanned aerial vehicle as a consideration factor, adding the operation of eliminating the swing of the unmanned aerial vehicle, and executing the acceleration threshold grading control strategy of the unmanned aerial vehicle; S33: using outer loop control to suppress the swing of the hoisted object, designing a control to generate expected speed increment instructions on the x-axis and y-axis, and suppressing low-frequency swing. S34: Finally, the swing amplitude A according to the binocular vision feedback x , A y and frequency f x and f y and the dynamic gain of the fuzzy PID output, the expected speed increment on the x-axis and y-axis is generated. 5.The method of claim 3, wherein, The specific calculation method of the swing angle of the object on the x-axis and y-axis in S22 is as follows: wherein k is a pixel-physical size conversion coefficient determined by binocular vision camera calibration parameters, Δz is depth information of the binocular vision camera, i.e. real-time distance between the object and the unmanned aerial vehicle, Δx image is pixel displacement of the object horizontal direction X axis in the image coordinate system, Δy image is pixel displacement of the object horizontal direction Y axis in the image coordinate system, θx is the swing angle of the hoisted object on the x axis, i.e. amplitude on the x axis, and θy is the swing angle of the hoisted object on the y axis, i.e. amplitude on the y axis. 6.The method of claim 3, wherein, The swing angular velocity ω of the hoisted object in the x-axis and y-axis on the horizontal plane in the three-dimensional space in the S23 x and ω y The specific calculation method is:

7. The unmanned aerial vehicle hoisting and transporting swing suppression method based on visual feedback according to claim 3, characterized in that, The specific steps in S24 are as follows: The swing angle signal θ x and θ y are sampled at equal intervals to obtain discrete time sequence data. The sampling time interval Δt = 1 / Fs, the sampling frequency is Fs, the sampling frequency is at least 2 times greater than the maximum frequency of the swinging of the hoisted article, the total number of sampling points is N, and the total time length T = N·Δt. FFT transformation of the swing angle time sequence data, converting the time domain signal to the frequency domain, decomposing the discrete signal time sequence x[n] into the superposition of multiple complex sinusoidal waves, and the frequency component after the FFT transformation is as follows: where N is the number of sampling points, x[n] is a discrete time domain signal, i.e., time series data, which is the swing angle signal collected at discrete time point n, n is a time index, and its value range is n = 0, 1, 2, 3, …, N-1, corresponding to the collection time t = n·Δt, X[k] is a discrete frequency domain signal, i.e., a complex frequency domain representation after FFT transformation of x[n], which describes the amplitude and phase of the signal at frequency f k ; k corresponds to the frequency index in the frequency domain, and its value range is k = 0, 1, 2, 3, …, N-1, and the corresponding physical frequency is f k ; The amplitude calculation, each frequency component X(f k ) obtained by the FFT transform is a complex number, whose modulus |X(f k )| represents the amplitude of the frequency component: where f k is the kth frequency component, X(f k ) is the complex amplitude at f k component, Re(X(f k )) is the real part of X(f k ), and Im(X(f k )) is the imaginary part of X(f k ). By analyzing |X(f k )|, the frequency with the maximum amplitude, i.e. the component with the lowest frequency, can be found, which is the main frequency f main of the swing, which is the main frequency component of the displacement time sequence data, i.e. the swing frequency of the hoisted object relative to the unmanned aerial vehicle, according to the swing angle signals θ x and θ y , the swing main frequencies f x and f y of the hoisted object on the x-axis and y-axis are respectively calculated at equal intervals. 8.The method of claim 4, wherein, The specific steps in S31 are as follows: According to the swing amplitude grading adjustment of the gain, the swing amplitude is divided into three fuzzy sets, that is, small amplitude S: A<5°, medium amplitude M: 5°≤A≤15°, and large amplitude L: A≥15°, wherein the swing amplitude is the absolute value of the swing angle of the hoisted object on the x-axis and y-axis respectively; The swing frequency is also divided into three fuzzy sets, that is, low frequency LF: 0Hz<f≤1Hz, medium frequency MF: 1Hz<f≤3Hz, and high frequency HF: f>3Hz, wherein the swing frequency is the swing frequency of the hoisted object on the x-axis and y-axis respectively; Defuzzification method: the accurate gain value is calculated by using the center of gravity method COA: where μ i is the rule trigger weight, K p ,K d is the gain value defined in the rule table, and i has a value in the range i = 1, 2, 3,..., 9. 9.The method of claim 4, wherein, The specific steps in S32 are as follows: The acceleration of the UAV is divided into three fuzzy sets, high acceleration HA: a≥2m / s 2 , medium acceleration MA: 0.2m / s 2 ≤a≤2m / s 2 , zero acceleration ZA: a≤0.2m / s 2 When the acceleration of the UAV is greater than the set safety threshold, the sway is completely disabled, when a≥a max , the K p =0, K d =0 is forcibly set, and the flight stability is prioritized; when a mid <a<a max , the gain is proportionally reduced: 10.The method of claim 4, wherein, The specific steps in S33 are as follows: Control law design: the input is composed of an equivalent control term and a switching term: Δv x = K p · θx+ K d · ω x + η · sat(s x / Φ) where the equivalent control term is K p • θx+ K d • ω x , K p based on the fuzzy PID output d , the base compensation amount is generated, and the switching term is η • sat(s x / Φ), s x , s y with the swing angle, swing angle rate and frequency as inputs: s x = K A · θx· sin(2πf x t) + K d · ω x s y = K A · θy· sin(2πf y t) + K d · ω y where K A is the amplitude-velocity gain coefficient, K d is the differential damping coefficient; Switching gain η: according to the swing energy adaptive adjustment, sat(): saturation function, boundary layer thickness Φ=0.1, and chattering suppression: where η max is the maximum switching gain, η min is the minimum switching gain, E is the swing energy, E max is the maximum swing energy: where m is the mass of the hoisted article; Finally, the swing amplitude A according to the binocular vision feedback x , y and the frequency f x and f y and the dynamic gain of the fuzzy PID output, the expected speed increment on the x-axis and y-axis is generated: Δv x = K p · θx+ K d · ω x + η · sat(s x / Φ) Δv y = K p · θy + K d · ω y + η · sat(s y / Φ) wherein the proportional term K p • θx, K p • θy: generates a compensation amount according to the current swing amplitude, K p is a proportional gain coefficient for generating a compensation amount for the speed according to the swing amplitude of the hoisted article, the differential term K d • ω x , K d • ω y inhibits the amplitude change rate and prevents overshoot, K d is a differential gain coefficient for inhibiting the amplitude change rate, i.e. the angular rate ω x , ω y caused dynamic error, the switching term η · sat(s x / Φ), η · sat(s y / Φ): resist model uncertainty and external disturbance; disturbance suppression is achieved through adaptive gain η and saturation function sat(), the physical essence is high-frequency nonlinear compensation force, resist model uncertainty and external disturbance, the final given desired speed of the unmanned aerial vehicle is the given speed of the current unmanned aerial vehicle plus the given desired speed increment.

Images