Unmanned aerial vehicle image visual servo method based on robust observation and output feedback

Abstract

The invention provides an unmanned aerial vehicle image visual servo method based on robust observation and output feedback, and the method comprises the steps: carrying out the dynamic modeling of an unmanned aerial vehicle, employing an image moment as a visual feature, mapping a translation optical flow to an image error, and deducing a system dynamic formula containing the translation and image features of the unmanned aerial vehicle; constructing an image moment error observer and a robust observer about a virtual image feature translation optical flow; aiming at the uncertainty of the image feature depth, a self-adaptive control rule is provided, so that a controller has robustness for unknown depth information; and establishing an IBVS controller with output feedback. According to the method, the robustness of the unmanned aerial vehicle in an unknown depth environment is effectively improved, a smoother image path can be obtained on a virtual image plane and a real image plane, and the image moment error and the unmanned aerial vehicle speed can be observed more accurately.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and mobile robots, and in particular relates to a UAV image visual servoing method based on robust observation and output feedback. Background Art

[0002] With the rapid development of drones in practical applications, researchers are constantly designing new models and advanced controllers to improve the reliability of drones in a variety of potential applications. Drones can be used as aerial vision devices, such as providing various aerial footage, and can also perform tasks in dangerous and hard-to-reach areas, such as infrastructure inspections. Furthermore, drones play an important role in reconnaissance and rescue missions, enabling rapid response to emergencies and providing critical information and support. Drone research generally involves developing efficient actuators, using precise sensors, and designing reliable controllers. Drone sensor systems typically include inertial measurement units (IMUs) and global positioning systems (GPS). While IMU systems can provide reliable angular velocity and attitude information for drones, they cannot accurately estimate translational position and linear velocity. Furthermore, GPS provides relatively high-precision positioning information, but it does not operate effectively indoors and in urban canyon environments. Furthermore, obtaining linear velocity in GPS-denied areas is also challenging.

[0003] Recent research has focused on providing drones with environmental and motion information, particularly in environments without GPS signals. In this context, drone system solutions have emerged that leverage vision sensors. Vision sensors are reliable, low-cost devices that can be fused with IMU data to provide effective translational velocity information and effectively localize the drone relative to its environment. Applications of drone vision systems include obstacle avoidance, attitude estimation, simultaneous localization and mapping, and path planning.

[0004] Vision-based UAV control primarily includes two modes: position-based visual servoing (PBVS) and image-based visual servoing (IBVS). PBVS estimates the three-dimensional rectangular coordinate space of a target object using visual sensors and uses this information to determine the UAV's trajectory. IBVS directly utilizes image information of target features to form a closed-loop control loop, which is robust to camera models and does not require three-dimensional reconstruction. Compared to PBVS, IBVS does not calculate the target position in three-dimensional coordinates, but instead relies on the projective relationship between the camera and the target for calculation and control. IBVS can better adapt to rapid scene motion and the uncertainty of target motion direction and distance. Furthermore, given the underactuated nature of quadrotors and coupled dynamics in the image plane, IBVS approaches for quadrotors are more challenging than those for fully actuated systems. Within the IBVS framework, the rotational and translational degrees of freedom need to be decoupled and independently controlled. In some studies, the IBVS framework achieves kinematic decoupling by selecting passive image features, and then designs simple controllers for translation and yaw motion, respectively.

[0005] The IBVS control mode based on virtual cameras has gradually become a research hotspot in recent years. In order to solve the problem of visual targets leaving the camera's field of view (FOV), researchers proposed an input saturation control law to ensure that roll, pitch and thrust are within the specified range. In order to deal with visibility constraints, researchers developed a robust nonlinear model predictive control scheme for UAV IBVS control, so that the visual target is kept within the camera FOV. Related researchers designed a hierarchical homography-based visual servoing scheme for posture extraction to solve the under-actuation problem of UAVs. To solve the problem of manual operation to observe visual targets, researchers designed an integrated IBVS flight system, including takeoff, target search and IBVS stages.

[0006] In the IBVS framework, the linear velocity of a drone is typically measured using auxiliary devices such as an IMU, but the measurement accuracy is susceptible to error accumulation and large vibrations. To address this issue, researchers designed a controller based on optical flow on the image plane that does not require information about the drone's linear velocity. Using a simplified model, the researchers designed a virtual camera-based visual servoing method, in which the drone's linear velocity is provided by an external motion capture system. They also designed a Lie group-based full-state observer to control drone motion without requiring angular and linear velocity measurements.

[0007] Depth information needs to be considered when designing an IBVS controller, and when the geometric model of the target is unavailable, the depth information is an unknown parameter. Some researchers have proposed an adaptive IBVS scheme to compensate for depth uncertainty, in which a linear observer is designed to estimate the image feature velocity of a stationary target. Related researchers have proposed an IBVS strategy based on layered images, using distance sensors to estimate the depth of objects. Other researchers have designed a visual servo controller using the backstepping method, and set the depth as part of the control gain within a certain range, thus eliminating the need to know the depth information of the feature points. However, how to effectively improve the robustness of drones in unknown depth environments and how to solve the problem of low quality or even unusable linear velocity obtained by the IMU remains a challenge. Summary of the Invention

[0008] The present invention proposes a UAV image visual servoing method based on robust observation and output feedback, which effectively improves the robustness of the UAV in an unknown depth environment. At the same time, a smoother image path can be obtained on both the virtual image plane and the real image plane, and the image moment error and UAV speed can be observed more accurately.

[0009] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0010] A UAV image visual servoing method based on robust observation and output feedback includes:

[0011] S1. Model the UAV's dynamics, use image moments as visual features, map translational optical flow to image errors, and derive the system dynamics formula that includes the UAV's translation and image features;

[0012] S2. Construct an image moment error observer and a robust observer for the translational optical flow of virtual image features;

[0013] S3. Aiming at the uncertainty of image feature depth, an adaptive control law is proposed to make the controller robust to unknown depth information;

[0014] S4. Establish an IBVS controller with output feedback.

[0015] Furthermore, step S1 specifically includes:

[0016] S101. Perform dynamic modeling on the UAV, analyze the transformation of its position and attitude in space under external forces and torques, and establish the Jacob matrix between the Euler angle vector change and the angular velocity vector change;

[0017] S102. Use the roll angle and pitch angle of the drone to project the image data onto a virtual image plane parallel to the object; select an image moment in the virtual image plane, define the translational optical flow on the virtual image plane as a variable vector related to the linear velocity, and derive a system dynamics formula that includes the drone's translation and image characteristics.

[0018] Furthermore, step S103 includes:

[0019] Reference the virtual camera model, whose optical center and parameters are the same as the drone's bottom camera;

[0020] Define the image moment vector for controlling the translation of the drone, the image moment vector for controlling the yaw movement of the drone, and the expected image moment vector;

[0021] Define the translational optical flow on the virtual image plane as a variable vector related to the linear velocity;

[0022] Derive the system dynamics formula that includes the UAV translation and image characteristics,

[0023] Auxiliary vectors about image moment errors are constructed, and the auxiliary vectors include translation motion image moment estimation error auxiliary vectors, image moment error vector filter auxiliary vectors, translation optical flow image moment error auxiliary vectors, and convergence analysis auxiliary vectors.

[0024] Furthermore, constructing an image moment error observer based on the image filter in step S2 includes:

[0025] S201, constructing a nonlinear observer based on the image filter as an image moment error observer;

[0026] S202. Construct a nonlinear observer based on the linear velocity of the UAV as a robust observer of the translational optical flow of the virtual image features.

[0027] Furthermore, step S3 includes:

[0028] A variable representing the constant depth from the target plane to the camera center when the drone is in a desired posture is defined, and the estimated value of the variable is adaptively updated based on the image moment error vector filter auxiliary vector, the translational optical flow image moment error auxiliary vector, and the convergence analysis auxiliary vector as an adaptive control law.

[0029] Furthermore, step S4 includes: based on the system dynamics formula, the image moment error observer and the robust observer of the virtual image feature translation optical flow, and the adaptive control law, establishing a controller that adjusts the posture and position of the drone according to the actual image moment and the expected image moment, and realizes output feedback control according to the image moment error.

[0030] On the other hand, the present invention also proposes a UAV image visual servoing device based on robust observation and output feedback, comprising:

[0031] Modeling and derivation module: This module performs dynamic modeling of the drone, uses image moments as visual features, maps translational optical flow to image errors, and derives system dynamics formulas that include the drone's translation and image features.

[0032] Observer module: constructs an image moment error observer and a robust observer for the translational optical flow of virtual image features;

[0033] Adaptive module: To address the uncertainty of image feature depth, an adaptive control law is proposed to make the controller robust to unknown depth information;

[0034] Controller module: Builds an IBVS controller with output feedback.

[0035] Furthermore, the modeling derivation module includes:

[0036] Conduct dynamic modeling of the drone, analyze the transformation of its position and attitude in space under external forces and torques, and establish the Jacob matrix between the Euler angle vector change and the angular velocity vector change;

[0037] Using the roll and pitch angles of the drone, the image data is projected onto a virtual image plane parallel to the object. Image moments are selected in the virtual image plane, and the translational optical flow on the virtual image plane is defined as a variable vector related to the linear velocity. A system dynamics formula that incorporates the drone's translation and image characteristics is derived. Here,

[0038] Reference the virtual camera model, whose optical center and parameters are the same as the drone's bottom camera;

[0039] Define the image moment vector for controlling the translation of the drone, the image moment vector for controlling the yaw movement of the drone, and the expected image moment vector;

[0040] Define the translational optical flow on the virtual image plane as a variable vector related to the linear velocity;

[0041] Derive the system dynamics formula that includes the UAV translation and image characteristics,

[0042] Auxiliary vectors about image moment errors are constructed, and the auxiliary vectors include translation motion image moment estimation error auxiliary vectors, image moment error vector filter auxiliary vectors, translation optical flow image moment error auxiliary vectors, and convergence analysis auxiliary vectors.

[0043] The present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the above-mentioned drone image visual servoing method based on robust observation and output feedback.

[0044] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned drone image visual servoing method based on robust observation and output feedback.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The proposed method converges slightly slower on the drone's position error, but provides smaller control inputs and less position fluctuation. This results in a smoother image path on both the virtual and real image planes, which means the drone's flight is stable and its thrust and attitude are not subject to sudden adjustments. Furthermore, the proposed method provides more precise observations of image moment errors and drone velocity.

[0047] 2. This invention introduces a depth-adaptive mechanism into the design of the IBVS controller, effectively improving the adaptability and robustness of the visual servo controller in unknown depth environments. The proposed method not only enables the drone to stably hover at the desired position and attitude, but also achieves high-precision velocity estimation through a robust observer design. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic flow chart of the method of the present invention.

[0049] Figure 2 The simulation results of Example 2 of the present invention are shown: UAV state change and system error;

[0050] in: Figure 2 (a) represents the position of the drone, Figure 2 (b) represents the Euler angle of the drone, Figure 2 (c) represents the virtual image plane error vector norm, Figure 2 (d) represents the total external force on the UAV.

[0051] Figure 3 The simulation results of the method of the present invention are shown as follows: the path of the image features in the IBVS stage;

[0052] in: Figure 3 (a) represents the path of the feature point on the real image plane, Figure 3 (b) represents the path of the feature points on the virtual image plane.

[0053] Figure 4The simulation results of the method of the present invention are shown as follows: the observed value of the image moment error on the virtual image plane;

[0054] [Solid line: estimated value, dotted line: true value].

[0055] Figure 5 The simulation results of the method of the present invention are shown as follows: the observed value of the UAV speed;

[0056] [Solid line: estimated value, dotted line: true value].

[0057] Figure 6 Represents the simulation results on the comparison method: UAV state and system error changes;

[0058] in: Figure 6 (a) represents the position of the drone, Figure 6 (b) represents the Euler angle of the drone, Figure 6 (c) represents the error vector norm on the virtual image plane, Figure 6 (d) represents the total external force on the UAV.

[0059] Figure 7 Represents the simulation results on the comparison method: the path of image features in the IBVS stage;

[0060] in: Figure 7 (a) represents the path on the real image plane, Figure 7 (b) represents the path on the virtual image plane.

[0061] Figure 8 Represents the simulation results of the comparison method: the image moment error observation results on the virtual image plane;

[0062] [Solid line: estimated value, dotted line: true value].

[0063] Figure 9 Represents the simulation results on the comparison method: UAV speed observations;

[0064] [Solid line: estimated value, dotted line: true value].

[0065] Figure 10 The experimental results of the method of the present invention are shown as follows: the position change of the UAV during the entire flight process;

[0066] [Dotted line: desired position, solid line: current position].

[0067] Figure 11 The experimental results of the method of the present invention are shown as follows: Euler angles and thrust control inputs of the UAV;

[0068] Figure 12The experimental results of the method of the present invention are shown as follows: the norm of the image error vector on the virtual image plane;

[0069] [dashed line: expected zero value, solid line: current value of the error vector norm].

[0070] Figure 13 The experimental results of the method of the present invention are shown as follows: the path of image features in the IBVS stage;

[0071] in: Figure 13 (a) represents the path on the real image plane, Figure 13 (b) represents the path on the virtual image plane.

[0072] Figure 14 The experimental results of the method of the present invention are shown as follows: the image moment error observation value on the virtual image plane;

[0073] [Solid line: estimated value, dotted line: true value].

[0074] Figure 15 The experimental results of the method of the present invention are shown as follows: the observed value of the UAV speed;

[0075] [Solid line: estimated value, dotted line: true value].

[0076] Figure 16 The experimental results of the comparison method are shown as follows: the position change of the UAV during the overall flight;

[0077] [Dashed line: expected position, solid line: actual position].

[0078] Figure 17 Represents the experimental results on the comparison method: Euler angles and thrust control inputs of the UAV.

[0079] Figure 18 Represents the experimental results on the comparison method: image moment error observation on the virtual image plane;

[0080] Values ​​[solid line: estimated value, dashed line: true value].

[0081] Figure 19 Represents the experimental results on the comparison method: UAV speed observation value;

[0082] [Solid line: estimated value, dotted line: true value]. DETAILED DESCRIPTION

[0083] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0084] The method proposed by the present invention is as follows Figure 1 Shown, including:

[0085] S1. Model the UAV's dynamics, use image moments as visual features, map translational optical flow to image errors, and derive the system dynamics formula that includes the UAV's translation and image features;

[0086] S2. Construct an image moment error observer and a robust observer for the translational optical flow of virtual image features;

[0087] S3. Aiming at the uncertainty of image feature depth, an adaptive control law is proposed to make the controller robust to unknown depth information;

[0088] S4. Establish an IBVS controller with output feedback.

[0089] In order to make the purpose and features of the present invention more obvious and easy to understand, the present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0090] Example 1:

[0091] In this embodiment, a quadrotor drone model is taken as an example to illustrate the drone image visual servoing method based on robust observation and output feedback proposed by the present invention.

[0092] 1. To model the kinematics and dynamics of a quadrotor drone, we make the following assumptions: the quadrotor is a uniform, symmetrical rigid body, its mass and moment of inertia do not change during flight, and it is subject only to gravity and propeller lift. The purpose of establishing the quadrotor kinematic and dynamic models is to analyze how its position and attitude change in space when subjected to external forces and torques.

[0093] The dynamic model of a quadrotor drone can be divided into linear velocity and angular velocity parts:

[0094]

[0095] F=-U1e3+mgR T e3 (5)

[0096] A dot above the parameter represents the first-order derivative of the parameter, and two dots represent the second-order derivative of the parameter.

[0097] m is the mass of the drone, J is a constant inertia matrix, and g is the acceleration due to gravity;

[0098] The rotation matrix R(t) is composed of the Euler angle components They represent the roll angle, pitch angle and yaw angle of the drone respectively.

[0099] v(t)=[v x (t),v y (t),vz (t)] T and w(t)=[w x (t),w y (t),w z (t)] T The drone is in the body coordinate system Linear and angular velocities.

[0100] ξ(t)=[x(t),y(t),z(t)] T The drone is in the world coordinate system The position below indicates that and are the net external forces and moments acting on the drone in the body coordinate system. U1(t) is the total thrust of the four propellers. The symbol sk(·) represents a skew-symmetric matrix that satisfies sk(a)b = a×b, where a and b are arbitrary three-dimensional vectors. e3 = [0,0,1] T is a unit vector.

[0101] The Jacob matrix between the Euler angle vector change and the angular velocity vector change is:

[0102]

[0103] 2. Controller and observer design.

[0104] The perspective projection model is widely used to discuss visual servoing for drones, primarily employing appropriately defined image moments as visual features. Using the drone's roll and pitch angles, image data is projected onto a virtual image plane parallel to the object. Image moments mitigate the effects of environmental noise and dynamic disturbances on single-point features, providing stable image feature measurements and a mathematical form that can be directly used in control law design. Based on the image moments selected in the virtual image plane, an image-based fully actuated controller can be designed for the drone, ensuring well-behaved control in Cartesian space.

[0105] A virtual camera model is introduced, whose optical center and parameters are the same as those of the lower camera, and its coordinate system is recorded as the virtual coordinate system Considering that there are N stationary point features located at on the horizontal plane, and The depth is the same. In order to realize the translation control of the UAV, it is necessary to select an image moment vector q(t) = [q x (t),q y (t),q z (t)] T , and its calculation formula is:

[0106]

[0107] in:

[0108]

[0109] in, v u k (t) and v n k (t) are the first and second components of the coordinates of the kth point on the virtual image plane, v u g (t) and v n g (t) is the coordinate of the target center point on the virtual image plane, v μ ij (t) corresponds to the virtual pixel distance between the center of the visual feature point and the center of the target. λ is the focal length of the camera, a * is the expected value of a(t) when the UAV reaches the desired posture.

[0110] Where N represents the total number of feature points involved in the calculation, and i and j represent the order of the image moment.

[0111] The dynamics on the virtual image plane can be obtained:

[0112]

[0113] Among them, z * represents the constant depth from the target plane to the camera center when the drone is in the desired pose. When the target model is unavailable, this parameter is unknown, so the proposed controller needs to handle the uncertainty. For general IBVS tasks, only ideal target images can be obtained.

[0114] Image features q used to control the yaw movement of unmanned aerial vehicles ψ (t) is defined as follows:

[0115]

[0116] v μ 11 It is the second-order mixing central moment, involving the mixed distribution characteristics in the horizontal and vertical directions, reflecting the correlation characteristics of the image between the two dimensions. v μ 20 It is the second-order central moment, which reflects the distribution characteristics of the image in the horizontal direction and describes the degree of discreteness of the image in the horizontal dimension. v μ 02 It is also the second-order central moment, which reflects the distribution characteristics of the image in the vertical direction and is used to characterize the discrete characteristics in the vertical dimension.

[0117] where q ψ The time derivative of (t) can be expressed as:

[0118]

[0119] Equation (10) means that in the translational motion of the UAV, there is a decoupled linear relationship between the virtual image space and the Cartesian coordinates. Therefore, the IBVS task can be completed by simply controlling the translation and yaw motion of the UAV.

[0120] The control task is to adjust the drone's attitude and position based on the actual image moment and the desired image moment, and to implement output feedback control based on the image moment error. When designing the controller, it is necessary to ensure that the camera coordinate system coincides with the drone's body coordinate system. If the two coordinate systems do not coincide, only a constant transformation between the two coordinate systems is required. First, define the desired image moment vector as follows:

[0121]

[0122] This means that the ideal position of the drone is at a specific height above the center of the target plane.

[0123] The image moment error vector e(t) used to control the translational motion of the drone is defined as follows:

[0124] e=qq d (14)

[0125] Taking the time derivative of e(t) and using (10), we can obtain:

[0126]

[0127] In order to obtain the translational dynamics of image features on the virtual image plane, the translational dynamics of the drone (3) is calculated in the virtual coordinate system. The following is reformulated as:

[0128] in The drone is in the virtual coordinate system The net external force under is the rotation matrix obtained by rotating around the roll and pitch directions.

[0129] In order to design the IBVS controller related to the dynamic equations (15) and (16), it is necessary to v v(t) is measured. Translational optical flow reflects the motion pattern of pixels in an image. By extracting feature points and tracking their motion, the UAV's translational velocity can be estimated. Using the velocity information estimated by translational optical flow makes it possible to exploit the passivity of image error dynamics in the design of an IBVS controller.

[0130] The translational optical flow is directly related to the linear velocity of the camera motion, making it easy to incorporate energy flow into the controller design. By mapping the optical flow to the image error, the controller can handle the translational image error independently.

[0131] By differentiating Equation (15) and substituting it into Equation (16), the system dynamics including the UAV translation and image features can be expressed as follows:

[0132]

[0133] in

[0134] In order to facilitate the design of controller and observer, the variable vector related to linear velocity is v v s (t) is defined as:

[0135] v v s =ρ v v (18)

[0136] where ρ = 1 / z * It is worth noting that the symbol v v s (t) is the translational optical flow on the virtual image plane, which is also a scaled linear velocity. Then, update Equation (15) with Equation (18), and use Equation (16) and Equation (18) respectively to obtain the dynamics of the translational optical flow as follows:

[0137]

[0138] Additionally, the following auxiliary variables are built for controller design:

[0139]

[0140] in, and are e(t) and v v s (t). e1(t) = k1e(t), ζ1(t) = k1ζ(t), where k1 is a positive constant gain.

[0141] It is worth noting that the auxiliary vector ζ(t) is related to the image moment estimation error of translational motion, while ε(t) represents the filter of the image moment error vector. The vector s(t) includes the translational optical flow, image moment error and its derivative. The vector r(t) is designed for the convergence analysis of ζ(t), which includes the image moment error derivative and the image estimation error. In equations (23) and (24), The relevant proportionality coefficients are determined by the defined Lyapunov derivatives through the method of undetermined coefficients. Using the constructed auxiliary variables, the speed observer and the deep adaptive law can be incorporated into the controller design, and then the thrust of the UAV can be calculated.

[0142] This embodiment designs a nonlinear observer to estimate As shown in the first expression of the following equation. In addition, a nonlinear observer is designed to estimate As shown in the second and third expressions below:

[0143]

[0144] Where k2 is a positive constant.

[0145] Assume ||M1||≤μ1, ||M2||≤μ2, where μ1 and μ2 are positive constants. Assume ρ min and ρ max are the minimum and maximum values ​​of ρ, respectively.

[0146] Define the variable z * The estimated value is And update it using the following adaptive rules:

[0147]

[0148] Where k3 is a positive constant. It defines the depth estimation error

[0149] Based on the above analysis, the following IBVS controller with output feedback is proposed:

[0150]

[0151] In addition, the designed controller (27) does not require the desired depth information z * , but replace it with the updated depth of formula (26) Therefore, the controller is robust to unknown image depth information without the need to replace z with a certain value. * .

[0152] Theorem: Consider the dynamic system defined by Equations (17), (19) and (20), with states e(t), ε(t), ζ(t), s(t), r(t) and control input f(t). By the designed controller Equation (27) and the adaptive law Equation (26), the system states e(t), ε(t), ζ(t), s(t), r(t) are UUB, and the depth estimation error It is bounded, and the states e(t),ε(t),ζ(t),s(t),r(t) converge asymptotically to 0.

[0153] Example 2:

[0154] Two simulation experiments were conducted on the MATLAB platform. The first simulation experiment was conducted to verify the feasibility of the proposed observer-based visual servo controller. Then, the second simulation experiment showed the results of the comparison method.

[0155] Simulation 1: World Coordinate System The initial coordinates of the feature points on the lower horizontal target plane are set to (0.25m, 0.2m, 0m), (0.25m, -0.2m, 0m), (-0.25m, 0.2m, 0m), and (-0.25m, -0.2m, 0m). The camera focal length is 1.8mm, and the sampling time is 0.01s. The mass of the drone is 4.55kg, and g = 9.81m / s. 2 Parameter ρ min and ρ max 0.08m respectively -1 and 2m -1 , corresponding to distances of 12.5 m and 0.5 m from the target, respectively. The controller and observer gains are chosen to be k r =8, k1=0.2, k2=20, k3=2 and k4=0.4. The initial values ​​of the adaptive rule (26) Set as

[0156] The initial hovering position of the drone is F w The following settings are (-0.4m, -0.1m, 7m), the Euler angle is (0, 0, 10deg), and the target is expected to be within the camera field of view (FOV). The desired image moment feature is set to At the same time, parameter a * =8.3×10 -8 , meaning the drone's final position is (0m, 0m, 4m) and the Euler angles are zero. Furthermore, 5dBW white Gaussian noise was added to the target location to simulate real-world interference. This noise addition validated the system's ability to withstand interference from image data and target locations.

[0157] Figure 2 The changes in the drone's state and error vector are shown, where the upper left corner shows the drone's position change, the upper right corner gives the drone's Euler angle change, the lower left corner provides the image moment error and the norm of the image moment estimation error on the virtual image plane, and the lower right corner shows the resultant external force on the drone. Figure 3 The paths of the feature points on the real image plane and the virtual image plane are given. Figure 3 (a) represents the path of the feature point on the real image plane, Figure 3 (b) shows the path of the feature point on the virtual image plane; the red circle represents the starting pixel and the red five-pointed star represents the final pixel. Figure 4 and Figure 5 The effectiveness of the proposed observer in image error and drone velocity estimation is demonstrated respectively. As shown in these two figures, the proposed observer can make the image error estimate and drone velocity estimate approach their true values ​​in about 5 seconds.

[0158] Simulation 2: The second simulation experiment is a comparative experiment, which aims to evaluate the performance difference between the proposed method and other methods. The same target parameter settings and initial conditions as the first simulation experiment are maintained to ensure the fairness and comparability of the experimental results. The control gain is selected as k r =8, k1=0.25, k2=25 and k4=0.4 to obtain better performance. Figure 6 It shows the changes of the UAV status and system error over time, and presents the evolution curves of the position, attitude angle, error and net external force related to the UAV through four sub-graphs. Figure 6 (a) represents the position of the drone, Figure 6 (b) represents the Euler angle of the drone, Figure 6 (c) represents the error vector norm on the virtual image plane, Figure 6 (d) represents the total external force on the UAV. Figure 7 The paths of image features on the virtual and real image planes are shown. Figure 7 (a) represents the path on the real image plane, Figure 7 (b) shows the path on the virtual image plane; the red circle represents the starting pixel and the red five-pointed star represents the final pixel. Figure 8 and Figure 9 The observation results of image moment error and UAV velocity are provided, and it can be seen that the image feature error and UAV velocity also converge to the true value.

[0159] Comparative simulation results show that the proposed method converges slightly slower on the drone's position error, but provides smaller control inputs and less position fluctuation. The proposed method also produces smoother image paths on both the virtual and real image planes, which means the drone's flight is stable and its thrust and attitude are not subject to sudden adjustments. Furthermore, the proposed method provides more accurate observations of image moment errors and drone velocity.

[0160] Example 3:

[0161] A dual-camera DJI-M100 drone was used as the experimental platform. The experiment consisted of three phases: takeoff, target search, and IBVS. Both the proposed method and the comparative method were considered to be in the IBVS phase. The front camera was used to record the drone's attitude throughout the flight, while the bottom camera was used during the IBVS phase. During the takeoff and target search phases, the geometric controller controlled the drone's flight using state information provided by the VINS-Mono algorithm. The proposed controller then controlled the drone during the IBVS phase. The first experiment verified the effectiveness of the proposed controller, and the second experiment demonstrated the results of the comparative method.

[0162] Experiment 1: The target plane is on the ground, world coordinate system The origin is set to the center of the four circles. The center coordinates are arbitrarily set to (0.25m, 0.25m, 0m), (-0.25m, 0.25m, 0m), (-0.25m, -0.25m, 0m) and (0.25m, -0.25m, 0m). The camera focal length is 1.8mm and the drone mass is m = 4.5kg. The desired image moment is set to and a * =9.1×10 -7 , the position of the drone in Cartesian space is (0,0,1.0m) and the Euler angle is zero. Parameter ρ min and ρ max Set to 0.08m respectively -1 and 2m -1 , corresponding to distances of 12.5m and 0.5m from the target, respectively. The closed-loop sampling time is 0.05 seconds, meeting real-time requirements.

[0163] The control gain constant is selected as k r =12.2, k1=0.25, k2=24.6, k3=5.8, k4=0.4. The initial value of the adaptive law (26) Set as The initial flight position is The following settings are (-1.2m, 0.1m, 0m) and the Euler angle is zero.

[0164] Figure 10 The figure shows the position change of the UAV. The proposed controller formula (27) can stabilize the UAV at the desired position. The first 10 seconds are the take-off and target search phase, which is controlled by the UAV nonlinear geometric controller. The following time is the IBVS phase, which is controlled by the proposed controller formula (27). Figure 11The Euler angle and thrust control inputs for the drone are shown. The control inputs for the IBVS phase are calculated by the designed controller, while the control outputs for the other two phases are calculated by the geometric controller. The drone's position and attitude during the three phases are recorded from the front camera using the VINS-Mono algorithm, eliminating the need for a motion capture system. Figure 12 The image moment error e(t) and the image moment estimation error ζ(t) on the virtual image plane are shown. Figure 13 The paths of feature points on the real image plane and virtual image plane in the IBVS stage are provided respectively, where Figure 13 (a) represents the path on the real image plane, Figure 13 (b) shows the path on the virtual image plane, the red circle represents the starting pixel, and the red five-pointed star represents the final pixel.

[0165] The true value of the UAV velocity is obtained by the VINS-Mono strategy and used to evaluate the observation accuracy. Figure 14 and Figure 15 The two figures show the effectiveness of the proposed observer on the output estimation of image moment error and UAV velocity, respectively. As shown in these two figures, the proposed observer can make the estimated signals converge to their true values ​​accurately in a short time.

[0166] Experiment 2: The target parameters and initial conditions are the same as those in Experiment 1. The control gain is chosen to be k r =15, k1=0.4, k2=25 and k4=0.4 to ensure the optimal system performance.

[0167] Figure 16 The drone's location is displayed. Figure 17 The Euler angles and thrust control inputs for the drone are shown. Comparison shows that the proposed method requires less thrust than the comparison method, which not only reduces energy consumption but also reduces system load. Furthermore, the yaw rate input curve is smoother, demonstrating that the proposed method offers improved stability and robustness in attitude control. Figure 18 and Figure 19 They represent the observation results of image moment error and UAV speed respectively. Figure 19 It shows that the proposed observer can estimate the UAV's speed information more accurately, providing reliable data support for the real-time adjustment of the control system.

[0168] In summary, this paper introduces a depth-adaptive mechanism into the design of the IBVS controller, effectively improving the adaptability and robustness of the visual servo controller in unknown depth environments. Experimental results demonstrate that the proposed method not only enables the drone to hover stably at the desired position and attitude, but also achieves high-precision velocity estimation through a robust observer design.

[0169] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A UAV image visual servoing method based on robust observation and output feedback, characterized in that: include: S1. Model the UAV's dynamics, use image moments as visual features, map translational optical flow to image errors, and derive the system dynamics formula that includes the UAV's translation and image features; S2. Construct an image moment error observer and a robust observer for the translational optical flow of virtual image features; S3. Aiming at the uncertainty of image feature depth, an adaptive control law is proposed to make the controller robust to unknown depth information; S4. Establish an IBVS controller with output feedback.

2. The UAV image visual servoing method based on robust observation and output feedback according to claim 1 is characterized in that: Step S1 specifically includes: S101. Perform dynamic modeling on the UAV, analyze the transformation of its position and attitude in space under external forces and torques, and establish the Jacob matrix between the Euler angle vector change and the angular velocity vector change; S102. Use the roll angle and pitch angle of the drone to project the image data onto a virtual image plane parallel to the object; select an image moment in the virtual image plane, define the translational optical flow on the virtual image plane as a variable vector related to the linear velocity, and derive a system dynamics formula that includes the drone's translation and image characteristics.

3. The UAV image visual servoing method based on robust observation and output feedback according to claim 2 is characterized in that: Step S103 includes: Reference the virtual camera model, whose optical center and parameters are the same as the drone's bottom camera; Define the image moment vector for controlling the translation of the drone, the image moment vector for controlling the yaw movement of the drone, and the expected image moment vector; Define the translational optical flow on the virtual image plane as a variable vector related to the linear velocity; Derive the system dynamics formula that includes the UAV translation and image characteristics, Auxiliary vectors about image moment errors are constructed, and the auxiliary vectors include translation motion image moment estimation error auxiliary vectors, image moment error vector filter auxiliary vectors, translation optical flow image moment error auxiliary vectors, and convergence analysis auxiliary vectors.

4. The UAV image visual servoing method based on robust observation and output feedback according to claim 1 is characterized in that: Constructing an image moment error observer based on the image filter in step S2 includes: S201, constructing a nonlinear observer based on the image filter as an image moment error observer; S202. Construct a nonlinear observer based on the linear velocity of the UAV as a robust observer of the translational optical flow of the virtual image features.

5. The UAV image visual servoing method based on robust observation and output feedback according to claim 4 is characterized in that: Step S3 includes: A variable representing the constant depth from the target plane to the camera center when the drone is in a desired posture is defined, and the estimated value of the variable is adaptively updated based on the image moment error vector filter auxiliary vector, the translational optical flow image moment error auxiliary vector, and the convergence analysis auxiliary vector as an adaptive control law.

6. The UAV image visual servoing method based on robust observation and output feedback according to claim 1 is characterized in that: Step S4 includes: based on the system dynamics formula, the image moment error observer and the robust observer of the virtual image feature translation optical flow, and the adaptive control law, establishing a controller that adjusts the attitude and position of the drone according to the actual image moment and the expected image moment, and implements output feedback control according to the image moment error.

7. A UAV image visual servoing device based on robust observation and output feedback, characterized in that: include: Modeling and derivation module: This module performs dynamic modeling of the drone, uses image moments as visual features, maps translational optical flow to image errors, and derives system dynamics formulas that include the drone's translation and image features. Observer module: constructs an image moment error observer and a robust observer for the translational optical flow of virtual image features; Adaptive module: To address the uncertainty of image feature depth, an adaptive control law is proposed to make the controller robust to unknown depth information; Controller module: Builds an IBVS controller with output feedback.

8. The UAV image visual servoing device based on robust observation and output feedback according to claim 7 is characterized in that: The modeling and derivation modules include: Conduct dynamic modeling of the drone, analyze the transformation of its position and attitude in space under external forces and torques, and establish the Jacob matrix between the Euler angle vector change and the angular velocity vector change; Using the roll and pitch angles of the drone, the image data is projected onto a virtual image plane parallel to the object. Image moments are selected in the virtual image plane, and the translational optical flow on the virtual image plane is defined as a variable vector related to the linear velocity. A system dynamics formula that incorporates the drone's translation and image characteristics is derived. Here, Reference the virtual camera model, whose optical center and parameters are the same as the drone's bottom camera; Define the image moment vector for controlling the translation of the drone, the image moment vector for controlling the yaw movement of the drone, and the expected image moment vector; Define the translational optical flow on the virtual image plane as a variable vector related to the linear velocity; Derive the system dynamics formula that includes the UAV translation and image characteristics, Auxiliary vectors about image moment errors are constructed, and the auxiliary vectors include translation motion image moment estimation error auxiliary vectors, image moment error vector filter auxiliary vectors, translation optical flow image moment error auxiliary vectors, and convergence analysis auxiliary vectors.

9. A computer-readable storage medium storing a computer program, characterized in that: The computer program is used to execute the drone image visual servoing method based on robust observation and output feedback as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the drone image visual servoing method based on robust observation and output feedback as described in any one of claims 1 to 6 is implemented.

Images