An unmanned aerial vehicle autonomous perching visual servoing control method based on an interference observer

Abstract

This invention discloses a visual servo control method for autonomous berthing of unmanned aerial vehicles (UAVs) based on an interference observer, belonging to the field of UAV control technology. The method first utilizes an inertial measurement unit (IMU) to construct a virtual vertical image plane, eliminating visual-attitude coupling caused by the tilt of the underactuated UAV through projection transformation. Second, it extracts the upper and lower edge features of the target beam, constructing decoupled visual features containing position, distance, and yaw information. Yaw and distance errors are calculated using the slope difference and spacing of the line features, respectively. Simultaneously, a nonlinear interference observer is constructed to estimate interference caused by wind disturbance and center of gravity changes in real time. A cascaded flight control law, including an outer loop of visual kinematics and an inner loop of dynamics, is designed based on the backstepping method, and the interference estimate is introduced into the control loop for feedforward compensation. This invention effectively eliminates the uncertainty of model parameters and the steady-state error caused by external environmental disturbances, achieving accurate target positioning and robust berthing of UAVs in complex environments.

Description

Technical Field

[0001] This invention belongs to the field of autonomous control and machine vision technology for unmanned aerial vehicles (UAVs), specifically relating to a visual servo control method for achieving precise landing of underactuated multirotor UAVs in complex disturbance environments. Background Technology

[0002] Multi-rotor drones, due to their simple mechanical structure and strong hovering capabilities, have been widely used in fields such as power line inspection, environmental monitoring, and bridge inspection. In long-endurance tasks such as power line inspection, to conserve onboard battery energy and maintain long-term stationary monitoring or boom-assisted operations, drones typically need to autonomously perch on angle steel, beams, or transmission lines using a mechanical gripper. Visual servoing technology, which guides drone flight by processing onboard camera images in real time, is a key technology for achieving precise end-point perching.

[0003] While there is some research on visual servoing and perching control for UAVs, stability in complex disturbance environments remains insufficient. For example, the paper "Visual Servoing of Quadrotors for Perching by HangingFrom Cylindrical Objects" (IEEE Robotics and Automation Letters, 2016) proposes using a geometric model to estimate pose for perching on a cylinder. However, this method heavily relies on specific cylindrical geometric features, and the control law design does not adequately consider the attitude coupling effect and external wind disturbance when the underactuated UAV approaches the target, making it prone to oscillations at the moment of contact. The paper "Image Dynamics-Based Visual Servo Control for Unmanned Aerial Manipulator With a Virtual Camera" (IEEE / ASME Transactions on Mechatronics, 2022) decouples image features from UAV attitude motion by constructing a virtual image plane and compensates for changes in the center of gravity. However, this method mainly focuses on kinematic decoupling and lacks active estimation and feedforward suppression mechanisms based on dynamic models for sudden strong crosswinds or severe dynamic disturbances caused by mechanical movements, thus limiting its disturbance rejection performance. In addition, the paper "Robust Nonlinear Model Predictive Control Based Visual Servoing of Quadrotor UAVs" (IEEE / ASME Transactions on Mechatronics, 2021) proposed a robust nonlinear model predictive control scheme to handle external disturbances. However, this method has a huge computational load and is difficult to run at high frequency in real time on the airborne terminal with limited computing power. Moreover, its passive disturbance rejection strategy is difficult to balance the dynamic response speed and steady-state accuracy of the system when facing the uncertainty of variable structural parameters in the dwelling mission. Summary of the Invention

[0004] The purpose of this invention is to provide an autonomous territorial visual servo control method for unmanned aerial vehicles based on an interference observer, which can effectively solve the problems of visual-attitude coupling caused by underactuated characteristics, steady-state error caused by uncertainty of variable structure model, and poor robustness caused by external wind disturbance in existing visual servo control methods.

[0005] To address the aforementioned problems, this invention employs the following technical solution: A visual servo control method for autonomous UAV berthing based on an interference observer is designed. First, a virtual vertical plane is constructed using IMU information, and the original image features are de-rotated and projected to eliminate visual coupling caused by underactuation. Second, the bi-line features of the crossbeam are extracted, and a decoupled visual servo control law is designed. Simultaneously, a nonlinear interference observer is constructed to estimate the lumped interference caused by wind disturbance and center of gravity changes in real time, and a control loop is introduced for feedforward compensation. Specifically, the method includes the following steps:

[0006] Step S1:

[0007] Real-time acquisition of image stream data from the UAV's onboard camera and state data from the inertial measurement unit, defining an inertial coordinate system. Body coordinate system and camera coordinate system .

[0008] Step S2:

[0009] Based on Lie groups A virtual vertical camera model is constructed using rotational transformation theory. Homography transformation is then used to project feature points from the original image onto the virtual image plane, establishing a visual-kinematic decoupling mapping. The projection transformation model for the virtual feature points is as follows:

[0010]

[0011] In the formula: These are the homogeneous feature coordinates on the virtual image plane. The original image feature coordinates, For the camera intrinsic parameter matrix, Let the rotation matrix from the body to the camera satisfy the manifold constraint:

[0012]

[0013] in These are the roll angle and pitch angle of the drone, respectively. Using the basic rotation matrix, this homography mapping ensures that the optical axis of the virtual image plane remains horizontal, thereby decoupling the feature changes caused by the translational motion of the UAV from the feature distortions caused by the rotational motion in the image domain.

[0014] Step S3:

[0015] Extract the geometric features of the target within the virtual image plane space, and construct a decoupled visual feature vector based on image moments and line feature topology. And derive its corresponding image Jacobian matrix (Interaction Matrix), the visual feature vector Defined as:

[0016]

[0017] In the formula: The centroid coordinates of the target feature. The topological tilt angle of the feature line segment on the virtual plane. The projected width of the feature line segment;

[0018] To construct the image Jacobian matrix This invention, based on the pinhole camera imaging model and the optical flow continuity equation, analyzes the first-order differential relationship between the motion of feature points in the virtual image plane and the spatial motion of the camera. The specific derivation process is as follows:

[0019] 1. For centroid coordinate features :

[0020] According to the classical optical flow equation, a point on the image plane speed of movement With camera linear velocity and angular velocity The relationship is:

[0021]

[0022] Because the present invention introduces a virtual vertical image plane in step S2, the body roll is eliminated. and pitch The direct impact on the image, in the above equation The correlation terms are decoupled and eliminated. Therefore, the rate of change of the centroid feature simplifies to:

[0023]

[0024] 2. Regarding angular features :

[0025] The angle of inclination of the feature line segment on the image plane Subject only to rotation about the optical axis (i.e., yaw rate) Due to the influence of ), within the virtual vertical plane, translational motion does not change the slope of the line segment; therefore, its differential relationship is:

[0026]

[0027] 3. For width features :

[0028] Projected width of feature line segment Depth of target Inversely proportional, that is Taking the derivative with respect to time, we get:

[0029]

[0030] In summary, by combining the above differential relationships, we can establish a formula to describe the rate of change of the eigenvector. With virtual camera spatial velocity The Jacobian matrix of the mapping relationship between them The image Jacobian matrix Describes the rate of change of the eigenvector With virtual camera spatial velocity Mapping relationship between them:

[0031]

[0032] The sparsity of this matrix (i.e., the presence of a large number of zero elements) indicates that, through the virtual camera projection and feature selection of this invention, lateral / vertical translation, depth motion, and yaw motion are successfully decomposed into independent control channels in visual space, thereby ensuring the stability and convergence speed of subsequent control law design. Where: This is the estimated depth of the target. As a normalized focal length, this matrix indicates that the components of the eigenvector and the control degrees of freedom have a diagonally dominant decoupling characteristic.

[0033] Step S4:

[0034] Based on the six-degree-of-freedom rigid body dynamics equations of an unmanned aerial vehicle (UAV), a nonlinear disturbance observer (NDOB) is constructed, and auxiliary state variables are introduced. The lumped disturbance term, which includes model parameter perturbations and external environmental disturbances, is asymptotically estimated. The UAV dynamics model is expressed as follows:

[0035]

[0036] The state evolution equation of the nonlinear disturbance observer is designed as follows:

[0037] Force perturbation observer:

[0038]

[0039] Torque perturbation observer:

[0040]

[0041] In the formula: For the auxiliary state variables of the force observer, It is a positive definite diagonal gain matrix. For force disturbance estimates, For translational dynamics, the nominal control input term is... These are auxiliary state variables for the torque observer. It is a positive definite diagonal gain matrix. This is an estimate of the torque disturbance; As the nominal control input for rotational dynamics, the above structure allows the observer to estimate the composite disturbance in real time without directly measuring the acceleration signal, utilizing the exponential convergence characteristic.

[0042] Step S5:

[0043] This step employs a backstepping control strategy. Combining the nonlinear disturbance observer state obtained in step S4, a Lyapunov candidate function is constructed to derive the control command that guarantees the global stability of the closed-loop system.

[0044] 1. Translational loop control law design (position / velocity loop):

[0045] Define the translational velocity tracking error manifold for:

[0046]

[0047] In the formula, The desired speed command output by the vision servo system. Given the current speed of the UAV, differentiate the speed error and substitute it into the equation, including the lumped disturbance of translation. From the dynamic equations, we obtain the error dynamic equations:

[0048]

[0049] To ensure the convergence of the velocity error, a Lyapunov candidate function for the translational subsystem is constructed. :

[0050]

[0051] right Differentiate over time and substitute in the case of force perturbations. From the translational dynamic equations, we get:

[0052]

[0053] In order to Negative definite (i.e.) Design virtual thrust control vector as follows:

[0054]

[0055] In the formula: It is a positive definite diagonal gain matrix; This is the force disturbance feedforward compensation term output by NDOB. Substituting this control law into the derivative equation, we get:

[0056]

[0057] This theoretically guarantees that the velocity error exponent converges to the observer residual neighborhood. Subsequently, the total thrust amplitude is calculated through orthogonal projection. and the desired body attitude rotation matrix :

[0058] The total thrust amplitude is the projection of the virtual control vector onto the vertical axis of the fuselage:

[0059]

[0060] The desired Z-axis direction vector of the organism is:

[0061]

[0062] Combined with desired yaw angle Construct the desired rotation matrix using geometric methods , as the tracking target of the attitude loop;

[0063] 2. Rotational Loop Control Law Design (Attitude / Angular Velocity Loop):

[0064] Define attitude error on a manifold and angular velocity error :

[0065]

[0066]

[0067] In the formula, The virtual angular velocity command required for maintaining a calm attitude.

[0068] Constructing Lyapunov candidate functions for rotating subsystems :

[0069]

[0070] right Differentiate and substitute the result into the equation containing the torque disturbance. The equations of motion for a rigid body are given by:

[0071]

[0072] In order to Negative definite (i.e.) Design the final torque control command. :

[0073]

[0074] in: This is the nonlinear cancellation term for the gyro effect. For error feedback stabilization, For the torque disturbance feedforward compensation term of the NDOB output;

[0075] Lyapunov function of a fully closed-loop system combining translational and rotational loops. satisfy This ensures that the UAV achieves consistent ultimate bounded stability (UUB) in its six-degree-of-freedom flight state under varying structural parameter perturbations and external environmental disturbances.

[0076] The present invention has the following beneficial effects:

[0077] 1. This invention decouples visual kinematics from body attitude dynamics by constructing a virtual vertical image plane, and eliminates the interference of underactuated UAV attitude tilt on image features by derotating projection. It can effectively solve the problem of control divergence caused by field of view oscillation when approaching the target in traditional visual servoing methods. It has stable visual feedback, good real-time performance, and high control accuracy.

[0078] 2. This invention combines the backstepping control strategy with the nonlinear disturbance observer (NDOB). By estimating the center of gravity shift moment caused by external wind disturbance and the extension and retraction of the robotic arm in real time and performing feedforward compensation, it effectively improves the problem that traditional nonlinear controllers are prone to steady-state errors or even system instability under uncertain model parameters and strong disturbance environments, making the UAV berthing process more stable and safe.

[0079] 3. This invention adopts a dual-line feature extraction strategy based on the crossbeam edge to construct a decoupled feature vector containing position, yaw angle and distance information. It can maintain a stable control response even when there is noise in the sensor and perturbation of model parameters, thus enhancing the robustness and environmental adaptability of the UAV in autonomously navigating in complex outdoor power inspection environments. Attached Figure Description

[0080] Figure 1 This is a schematic diagram of the overall process of a visual servo control method for autonomous dwelling of unmanned aerial vehicles based on an interference observer according to the present invention;

[0081] Figure 2 This is a schematic diagram of the visual kinematic decoupling principle based on the virtual vertical image plane in this invention;

[0082] Figure 3This is a comparison diagram of the three-dimensional flight trajectories of the method of the present invention and the traditional backstepping method without interference observers under crosswind and center of gravity shift in the embodiments of the present invention;

[0083] Figure 4 This is a comparison diagram of the time-domain response of visual tracking error under the influence of crosswind interference in the embodiments of the present invention;

[0084] Figure 5 This is a comparison diagram of the temporal response of visual tracking error under the interference of internal parameters (center of gravity shift) in the embodiments of the present invention;

[0085] Figure 6 This is a comparison chart of the estimated and actual values ​​of external force disturbances and internal torque disturbances by the nonlinear disturbance observer in this embodiment of the invention.

[0086] Figure 7 These are feature extraction results from the UAV's forward-looking camera at the initial and final moments of the dwelling mission, as shown in this embodiment of the invention. Detailed Implementation

[0087] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0088] Figure 1 This is a flowchart of a visual servo control method for autonomous dwelling of unmanned aerial vehicles (UAVs) based on an interference observer, according to the present invention. This method is designed for dwelling tasks of underactuated UAVs in environments with varying structures and external disturbances, and specifically includes the following steps:

[0089] Step 1: Information Collection and Initialization

[0090] On the UAV's onboard unit, a forward-looking monocular camera acquires real-time environmental image information including the target crossbeam, while an inertial measurement unit (IMU) obtains the UAV's current attitude angle information (roll angle). With pitch angle ) and angular velocity information Set the physical parameters of the target beam and the intrinsic parameter matrix of the camera. And the expected feature values ​​of visual servoing.

[0091] Step 2: Construct a virtual vertical image plane for visual decoupling

[0092] To address the coupling characteristic of underactuated UAV translational motion requiring attitude tilt, as shown in Figure 2, a virtual image plane that is always perpendicular to the ground is constructed, and a de-rotation matrix is ​​built using the attitude angles obtained from the IMU. Feature points on the original image plane Projecting onto the virtual image plane, the projection transformation formula for virtual feature points is as follows:

[0093]

[0094] In the formula, This mapping eliminates the nonlinear interference of roll and pitch motion on image features, making the feature motion in the virtual plane only related to the position and yaw angle of the UAV, thus achieving geometric decoupling of visual kinematics.

[0095] Step 3: Extract bilinear features and construct decoupled feature vectors:

[0096] In the virtual image plane, an edge detection algorithm is used to extract the upper and lower edge lines of the beam. To achieve six degrees of freedom control, the following decoupled visual feature vectors are constructed:

[0097] 1. Calculate the geometric center of the top and bottom edges. , used to characterize the lateral and vertical positional deviations of the UAV;

[0098] 2. Calculate the difference in slope between the upper and lower edge lines. Since the virtual plane has eliminated the effect of the roll angle, the slope difference is only caused by the perspective distortion caused by the yaw angle, and is used to characterize the yaw error;

[0099] 3. Calculate the pixel spacing between the top and bottom edges. This is used to characterize the distance the drone travels relative to the crossbeam.

[0100] Step 4: Design of Nonlinear Disturbance Observer (NDOB):

[0101] To address the center of gravity shift (uncertainty in the variable structure model) caused by the telescopic arm movement of the UAV and external gust interference, a nonlinear interference observer is constructed based on the UAV dynamics model:

[0102] Establish the dynamic equations that include lumped perturbations:

[0103]

[0104] Design an observer state evolution equation to estimate force perturbations in real time. With torque disturbance :

[0105]

[0106]

[0107] In the formula, As an auxiliary variable, is the observer gain matrix.

[0108] Step 5: Design of robust control law based on backstepping:

[0109] A cascaded flight controller is designed using the backstepping method, and the disturbance estimate obtained in step S4 is introduced into the control loop for feedforward compensation. First, based on the Lyapunov function... Designing the virtual control force for a translational loop. This includes force disturbance compensation items. :

[0110]

[0111] Subsequently, control torque commands were designed for the rotation circuit. :

[0112]

[0113] The above compensation ensures the global asymptotic stability of the system under combined disturbances.

[0114] The method of this invention was verified by simulation using Matlab / Simulink software: Figures 3 to 6 The control effect of the method of the present invention is shown under the combined disturbances of strong crosswind (1.5N) and center of gravity shift (0.2Nm) caused by the extension of the robotic arm. The coordinate axis unit is m.

[0115] Figure 3 This is a comparison of the 3D flight trajectories of the UAV. In the image, the thick red line represents the target beam, the dashed blue line represents the traditional backstepping trajectory without interference observers, and the solid red line represents the trajectory of the method described in this invention. It can be seen that... After crosswind interference, traditional methods show significant drift; while... When the center of gravity shift interference is introduced, the trajectory of the traditional method deviates completely from the predetermined route and exhibits a divergent trend, making it impossible to complete the landing. In contrast, the method of this invention quickly corrects the trajectory after a brief fluctuation, maintaining a precise approach to the target at all times.

[0116] Figure 4 The temporal response of visual tracking errors was recorded in detail, intuitively revealing the impact mechanism of different types of interference on the system. With wind disturbance added, the traditional method (dashed line), lacking feedforward compensation, produced a steady-state error exceeding 150 pixels, causing the drone's hovering position to shift. After the center of gravity shifts, the persistent eccentric torque disrupts the balance of the attitude loop, causing the error curve of the traditional method to exhibit a linear divergence trend (rapidly exceeding 2000 pixels), indicating that the control system has completely become unstable. In contrast, the method of this invention (solid line) utilizes NDOB for effective estimation and compensation. Whether facing force disturbances that cause steady-state errors or torque disturbances that cause system instability, it can rapidly converge the error to the zero neighborhood, verifying the extremely strong robustness of this invention under complex and strong disturbances.

[0117] Figure 5 , 6 The estimation performance of the Nonlinear Disturbance Observer (NDOB) in this invention is demonstrated. The figure contains two sub-figures: Figure 5 For force disturbance estimation, Figure 6 For torque disturbance estimation. Figure 5 In the diagram, the dashed line represents the actual applied 1.5N lateral wind force, and the solid line represents the estimated value of NDOB. It can be seen that initially, due to the drone's acceleration, the observer estimated wind drag. Subsequently, after the lateral wind disturbance was introduced, the estimated curve quickly achieved tracking. Figure 6 In the middle, the dashed line represents the result of the robotic arm extending. The actual value of the eccentric moment is shown on the solid line, while the estimated value is shown on the solid line. The results show that NDOB can not only quickly estimate the force disturbance caused by external wind disturbance, but also accurately reconstruct the moment disturbance caused by changes in internal parameters (center of gravity shift), verifying the effectiveness of the observer design.

[0118] Figure 7 illustrates the changes in the forward-looking camera's field of view. The left image shows the initial moment, when the UAV exhibits significant positional and attitude deviations, with the crossbeam feature showing a marked tilt and off-center state. The right image shows the final hovering moment, where the upper and lower edges of the crossbeam appear parallel and centered in the image plane, and the feature points are essentially aligned with the central crosshair, indicating that the UAV has successfully corrected its positional error. Notably, the crossbeam feature line retains a stable tilt angle relative to the horizontal line at the final moment, due to the system being in an anti-disturbance equilibrium state. This is to counteract continuous lateral wind disturbances (corresponding to...). Figure 5 The force disturbance estimate in the figure), the control law of this invention drives the UAV to maintain a specific roll angle to generate lateral reaction force, thereby achieving force balance. This phenomenon precisely confirms that, under the compensation effect of the nonlinear disturbance observer, the UAV achieves precise steady-state dwelling with active wind resistance capability in complex disturbance environments.

[0119] The specific implementation schemes described above further illustrate the inventive purpose, technical solution, and beneficial effects of the present invention. The above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that any modifications or equivalent substitutions made to the technical solution of the present invention are included within the scope of protection of the present invention.

Claims

1. A visual servo control method for autonomous dwelling of unmanned aerial vehicles based on an interference observer, characterized in that, Includes the following steps: Step S1: Acquire environmental image information collected by the UAV's onboard camera, as well as the UAV's current attitude angle and angular velocity information collected by the inertial measurement unit; Step S2: Construct a virtual vertical image plane based on the attitude angle information, and use a projection transformation matrix to project the original feature points in the image information onto the virtual vertical image plane to obtain the de-rotated virtual feature points, so as to eliminate the coupling effect of the UAV body tilt on the image features; Step S3: Extract the upper and lower edge line features of the target beam on the virtual vertical image plane, and construct a decoupled visual feature vector containing position error, distance error and yaw error based on the upper and lower edge line features; Step S4: Establish a six-degree-of-freedom dynamic model of the UAV, and build a nonlinear disturbance observer based on the model. Use the motion state of the UAV and the nominal control input to estimate the external force disturbance value and the internal torque disturbance value in real time. Step S5: Design the flight control law based on the backstepping method, calculate the nominal control command according to the decoupled visual feature vector, and superimpose the external force disturbance value and the internal torque disturbance value as feedforward into the nominal control command to generate the final thrust and torque command to drive the UAV to fly.

2. The method for autonomous dwelling visual servo control of unmanned aerial vehicles based on an interference observer according to claim 1, characterized in that, In step S2, the specific formula for calculating the virtual feature points using the projection transformation matrix is ​​as follows: In the formula, The pixel coordinate vector of the feature point on the virtual vertical image plane. The coordinate vector of feature points in the original image. For the camera intrinsic parameter matrix, For the rotation matrix; The derotation matrix The roll angle of the drone and pitch angle Build: in, and These represent the rotation matrices around the Z-axis and X-axis of the aircraft body, respectively.

3. The method for autonomous dwelling visual servo control of unmanned aerial vehicles based on an interference observer according to claim 1, characterized in that, In step S3, the specific method for constructing the decoupled visual feature vector includes: Calculate the geometric center coordinates of the upper and lower edge lines on the virtual vertical image plane, and define the difference between them and the image center as the horizontal position error and the vertical position error; The difference in slope between the upper and lower edge lines on the virtual vertical image plane is calculated and defined as the yaw error. Calculate the pixel spacing between the upper and lower edge lines on the virtual vertical image plane, and define the difference between the pixel spacing and the preset expected spacing as the distance error.

4. The method for autonomous dwelling visual servo control of unmanned aerial vehicles based on an interference observer according to claim 1, characterized in that, In step S4, the nonlinear disturbance observer includes a force disturbance observer and a torque disturbance observer. The specific form of the force disturbance observer is as follows: In the formula, For the force observer, auxiliary state variables, For force observation gain matrix, For the quality of drones, For drone speed, For total thrust, It is a vertical unit vector. The estimated external force disturbance value; The specific form of the torque disturbance observer is as follows: In the formula, For the torque observer, auxiliary state variables, This is the torque observation gain matrix. Here is the rotational inertia matrix. Angular velocity, To control the torque, This represents the estimated internal torque disturbance value.

5. The method for autonomous territorial visual servoing control of a UAV based on an interference observer according to claim 4, characterized in that, In step S5, the specific steps for designing the flight control law based on the backstepping method include: (1) Define translational velocity tracking error ,in The expected speed is calculated based on visual feature errors; (2) Construct the Lyapunov function of the translational subsystem and design a virtual thrust vector that includes force perturbation compensation. : In the formula, It is the acceleration due to gravity. It is a vertical unit vector. It is a positive definite gain matrix; (3) Based on the virtual thrust vector Decomposition yields total thrust command and the desired pose rotation matrix; Define attitude angular velocity error ,in The desired angular velocity is calculated based on the desired attitude. (4) Construct the Lyapunov function of the rotating subsystem and design a control torque command that includes torque disturbance compensation. : In the formula, It is a positive definite gain matrix. This is the gyro torque compensation term.

6. An unmanned aerial vehicle (UAV) system, characterized in that, include: Airframe, power system, airborne cameras, inertial measurement unit, and airborne processor; The onboard processor is used to execute the UAV autonomous homing visual servo control method based on an interference observer as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images