Unmanned aerial vehicle vision control method based on speed observation and model prediction

Abstract

A state-observed and updated model predictive control strategy is proposed for image-based visual servoing (IBVS) of micro air vehicles (MAVs). The proposed strategy can accurately adjust the pose of MAVs without global positioning system (GPS). Specifically, image features are defined on a virtual image plane to decouple the translational motion of MAVs. Then, a linear velocity observer is developed to provide high-quality linear velocity information for MAVs in real time. The image dynamics on the virtual image plane are linearized by first-order Taylor expansion, and a controller is constructed based on model predictive control to efficiently solve the optimal control input. Furthermore, the state input of the controller is updated at each control period to eliminate the cumulative error of the rolling optimization on the linear approximation dynamics, ensuring the accuracy of IBVS. Experimental results demonstrate the performance of the proposed observer and control strategy.

Description

Technical Field

[0001] This patent pertains to visual servoing tasks in the control of quadrotor micro air vehicles (MAVs), combining velocity observation and model prediction to provide visual servoing for MAV-guided tasks. Background Technology

[0002] Quadrotor micro-aircraft (MAVs), with their compact structure and high maneuverability, can perform landing, hovering, and tracking tasks in complex indoor and outdoor environments. The integration of quadcopter UAVs with visual servo control is a current research hotspot in the field of autonomous UAV flight, driven by the need to address autonomous operation in environments with missing or unreliable GPS signals. Visual sensors can provide rich information about the external environment without accumulating errors, making vision-based technologies a solution for autonomous flight.

[0003] Image-based visual servoing (IBVS) is a control mode that controls robot motion through image feedback. IBVS directly utilizes image feature errors on the image plane to generate control inputs, thus forming closed-loop control and exhibiting robustness to camera models and environmental maps. However, in real-world image-based flight missions, MAVs face the coupled effects of multiple factors, including missing depth information, unmeasurable linear velocity information, and saturation constraints in the control system.

[0004] In traditional methods, the linear velocity of a UAV is obtained by integrating the IMU (Integrated Device Unit). However, this method is susceptible to sensor noise and vibration interference, and errors accumulate during long-duration flights or severe maneuvers, leading to decreased control accuracy or even mission failure. Achieving high-precision control under conditions of incomplete state observation is a crucial problem that must be solved.

[0005] To achieve stable convergence of the quadcopter UAV's position in the IBVS mission, researchers have extensively explored various control methods, including: designing control laws based on image errors using backstepping techniques, analyzing nonlinear control methods using Lyapunov stability theory or Babarette's lemma, and constructing controllers using QP or LQR. For attitude adjustment, the desired attitude is typically derived from virtual control inputs, or geometric control methods are employed. Unlike the aforementioned methods, MPC (Multi-Process Control) can balance response speed and steady-state accuracy with a lightweight design, solving for the optimal control sequence at every moment online. Due to its ability to explicitly handle system constraints, optimize future control sequences, and possess rolling time-domain correction capabilities, MPC exhibits significant advantages in the field of robot control.

[0006] Introducing Dynamic Performance Controller (MPC) into visual servoing tasks allows for the planning of optimal control inputs while satisfying actuator and field-of-view constraints, thereby improving the system's dynamic response performance and robustness. However, effective implementation of MPC relies on precise knowledge of the system state. In the absence of GPS, linear velocity cannot be directly measured, posing a new challenge to the integration of MPC and visual servoing. Therefore, combining a linear velocity observer with the MPC strategy constitutes an effective solution. Summary of the Invention

[0007] This patent focuses on high-precision IBVS control of a quadrotor UAV in a GPS-free environment, proposing an MPC strategy that integrates state observation and online updates. The method first defines image features on a virtual image plane to provide decoupled image dynamics for the MAV's translational motion. Then, based on an approximate observation method, a fast-converging, high-precision linear velocity observer is designed to provide real-time velocity estimation for the control system. Building upon this, the image dynamics under the virtual image plane are approximately linearized, constructing an MPC framework that transforms the IBVS task into a rolling time-domain optimization problem. In each control cycle, the linear velocity estimation provided by the observer is used to update and optimize the system state online. Simulations and experiments verify the effectiveness of the proposed control strategy.

[0008] The contributions of this invention can be summarized in two aspects as follows: 1. A high-precision linear velocity observer was designed for the quadcopter MAV to overcome the difficulty of unmeasurable linear velocity information during the IBVS process; 2. An MPC framework was constructed to perform online updates and optimization solutions using the state estimation provided by the observer in each control cycle, which can generate the optimal control sequence under the premise of satisfying actuator constraints, and significantly improve the control accuracy of the system in the absence of GPS.

[0009] 1. A vision control method for a quadcopter MAV, mainly comprising the following steps:

[0010] First, a new method for velocity observation.

[0011] Define auxiliary correction amount and Observe the image error and its derivative q(t) respectively. definition Indicates MAV speed v The observed values ​​of v(t), It can be obtained through ζ(t) and δ(t):

[0012]

[0013] Constructing an approximate transition term ρ i Optimize the initial stage of observation, which converges rapidly from infinity:

[0014]

[0015] Based on this, the following observer is designed to observe the linear velocity of the quadcopter:

[0016]

[0017] Where blkdiag(·) denotes diagonalizing the values ​​or matrices within the parentheses. For i = 1, 2, 3, the observation gain η i ε i γ is a positive constant, meaning the observation gain matrix is ​​positive definite, and γ needs to be sufficiently small to achieve accurate observation results. dt is the control period, and ∑dt=t is the total observation time. Approximate transition term. i ρ1、 i ρ2 is a positive constant and i ρ1 needs to be large enough to optimize the initial stage of the approximate observation when it converges rapidly from infinity, so that the observation results are closer to the actual values.

[0018] Second, an MPC controller for vision servoing in quadcopter MAVs.

[0019] Therefore, in the system state expression In the middle, the state variable is selected as Control input is f T (t) and These represent the force vector and yaw rate, respectively, and the control objective is for the system to rapidly move towards the zero equilibrium point x. d (t)=0、u d (t) = 0 converges.

[0020] To achieve a linear approximation of the system, a first-order Taylor expansion of F(x, u) is performed at the zero equilibrium point. By decomposing the nonlinear terms, a linear approximation of the system is achieved.

[0021] The linear discrete state equations of the system can then be obtained:

[0022] x(k+1)=Ax(k)+Bu(k) (7)

[0023] The system state matrix A and input matrix B can be directly calculated from the system parameters. This is achieved using positive definite weight matrices Q, R, and Q'. f For the three performance metrics of system state error, feasible range of input consumption, and closed-loop stability of system terminals, the following cost function is defined for rolling optimization to obtain the optimal control sequence U at the current time step. * (k):

[0024] Attached image description:

[0025] Figure 1 This is a flowchart of the IBVS control method.

[0026] Figure 2 This is a diagram of the MPC control architecture.

[0027] Figure 3 This is a diagram showing the experimental setup for a MAV vision servoing task.

[0028] Figure 4 This is a diagram showing the experimental results of the MAV trajectory during flight.

[0029] Figure 5 These are the target points captured in the actual image.

[0030] Figure 6 This is a diagram showing the experimental results of the MAV's position during flight.

[0031] Figure 7 The figure shows the experimental results of MAV Euler angles and control inputs during flight.

[0032] Figure 8 The figure shows the experimental results of MAV velocity and its observations during flight.

[0033] Figure 9 The figure shows the experimental results of MAV image feature errors during flight.

[0034] Figure 10 The figures show the experimental results of the target point trajectories under the virtual image plane and the real image plane. Detailed implementation method:

[0035] 1. Experimental Setup

[0036] The hardware platform is built on a DJI Matrice 100 quadcopter, equipped with a dual-vision perception system: the front vision unit uses an Intel RealSense D435i depth camera for forward environmental perception and state estimation; the lower vision unit also uses an Intel RealSense D435i camera, utilizing the OpenCV computer vision library to achieve color-feature-based visual target detection and recognition. The onboard computing unit uses an Intel NUC8i7BEH mini-PC, pre-installed with Ubuntu and the Robot Operating System (ROS), and includes the necessary software packages. The linear velocity observer, MPC controller, and related data processing are all performed on the onboard computer. The optimal control sequence is solved using the eigen-quadprog tool from the Eigen matrix library. Furthermore, the quadcopter platform is equipped with a DJI Lightbridge 2 image transmission device to transmit real-time images and flight data from the onboard computer to the ground station, facilitating experimental monitoring and data recording. The hardware configuration, system architecture, and communication methods of the entire experimental setup are as follows: Figure 3 As shown.

[0037] In the experiment, the target plane consisted of four circular images placed horizontally on the ground, with the center coordinates at... The values ​​are [1.2, -0.2, 0]. T m, [1, -0.2, 0] T m, [1, -0.4, 0] T m, [1.2, -0.4, 0] T m. The origin is set by initializing VINS-Mono to the value before takeoff. The origins coincide. The quadcopter's mass m = 4.5 kg, and the camera's focal length λ = 1.8 mm. The ideal vertical distance between the target and the camera is set to z. * =1m, quadcopter in The ideal position and Euler angles are set to [0, 0, 1] respectively. T m and [0, 0, 0] T rad, parameter a * =2.5×10 -7 .

[0038] Throughout the experiment, the sampling frequency was set to 50Hz, i.e., the sampling period dt = 0.02s, and the observation gain was set to η1 = η2 = η3 = 8, ε1 = ε2 = ε3 = 8, γ = 0.1, with approximate transition terms. 1 ρ1、 2 ρ1、 3 ρ1 is 1×104 1×10 4 1.2×10 4 , 1 ρ2、 2 ρ2、 3 ρ2 is 1, 1, 1.2 respectively, and the control weight matrices are Q, R, Q... f They are blkdiag(1500, 1500, 1500, 5, 8, 8, 8), blkdiag(0.5, 0.5, 0.5, 0.5), and blkdiag(1300, 1300, 1300, 3, 8, 8, 8).

[0039] Second, experimental results

[0040] Experimental results are as follows Figures 4 to 10 As shown. Figure 4 and Figure 6 It demonstrates the evolution of the quadcopter drone's position throughout the entire flight. Figure 5 Provides images captured by the downward-facing camera during flight. Figure 7 The Euler angles and thrust control inputs are displayed. Figure 8 The linear velocity and its observed values ​​during autonomous flight are shown. It can be seen that the velocity response remains stable during convergence without significant oscillations. The designed observer enables the observed values ​​to converge rapidly to their true values ​​within a short time and maintain stable tracking near the target value. Under the combined action of the designed velocity observer and the MPC controller, the quadrotor finally converges stably to the desired hovering pose, verifying the effectiveness of the proposed method in state observation and control coordination. Image feature errors are shown below. Figure 9 As shown, Figure 10 The motion trajectories of the target's center feature points on the virtual and real image planes during the IBVS phase are shown respectively. It is evident that the MAV rapidly converges to the target position during the IBVS phase, fully demonstrating the good convergence performance and system stability of the proposed control strategy in image space. Based on the above experimental results, the velocity observer designed in this patent can accurately observe the linear velocity of the quadrotor, and the MPC controller can work in conjunction with the velocity observer to precisely adjust the quadrotor's pose, enabling it to hover stably at the desired position relative to the visual target.

Claims

1. A vision control method for a quadcopter MAV, mainly comprising the following steps: First, a new method for velocity observation. Define auxiliary correction amount and Observe the image error and its derivative q(t) respectively. definition Indicates MAV speed v The observed values ​​of v(t), It can be obtained through ζ(t) and δ(t): Constructing an approximate transition term ρ i Optimize the initial stage of observation, which converges rapidly from infinity: Based on this, the following observer is designed to observe the linear velocity of the quadcopter: Where blkdiag(·) denotes diagonalizing the values ​​or matrices within the parentheses. For i = 1, 2, 3, the observation gain η i ε i γ is a positive constant, meaning the observation gain matrix is ​​positive definite, and γ needs to be sufficiently small to achieve accurate observation results. dt is the control period, and ∑dt=t is the total observation time. Approximate transition term. i ρ1、 i ρ2 is a positive constant and i ρ1 needs to be large enough to optimize the initial stage of the approximate observation when it converges rapidly from infinity, so that the observation results are closer to the actual values. Second, an MPC controller for vision servoing in quadcopter MAVs. Therefore, in the system state expression In the middle, the state variable is selected as Control input is f T (t) and These represent the force vector and yaw rate, respectively, and the control objective is for the system to rapidly move towards the zero equilibrium point x. d (t)=0、u d (t) = 0 converges. To achieve a linear approximation of the system, a first-order Taylor expansion of F(x, u) is performed at the zero equilibrium point. By decomposing the nonlinear terms, a linear approximation of the system is achieved. The linear discrete state equations of the system can then be obtained: x(k+1)=A x(k)+B u(k) (7) The system state matrix A and input matrix B can be directly calculated from the system parameters. This is achieved using positive definite weight matrices Q, R, and Q'. f For the three performance metrics of system state error, feasible range of input consumption, and closed-loop stability of system terminals, the following cost function is defined for rolling optimization to obtain the optimal control sequence U at the current time step. * (k):

Images